Giters

v-i-s-h / ai-tools

An index of openly available AI models and tools

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AI tools

This page contains a collection of opensource AI models and tools available for various use cases

Vision

Generative modeling

1. Stable Diffusion (2022 Aug 10)

2. Grounded-SAM ()

  • Summary: Marrying Grounding DINO with Segment Anything & Stable Diffusion & BLIP - Automatically Detect , Segment and Generate Anything with Image and Text Inputs
  • Resources:
  • Projects
    • Semantic-SAM - Segment and Recognize Anything at Any Granularity

3. AnimateDiff

4. PhotoMaker (2024 Jan)

5. DragGAN (2023 May)

Image Inpainting

1. lama-cleaner (2022 Nov)

  • Summary: Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
  • Resources

Object detection

1. YOLOv8

  • Summary: YOLOv8 in PyTorch > ONNX > CoreML > TFLite. Can do detection, segmentation and much more.
  • Resources

2. Face Recognition

Image Segmentation

1. SAM (2023 Apr 5) (License: Apache 2.0)

2. Detic (2021 Jan)

Image embeddings

1. DINO

Video

Object tracking

1. TrackHQ (2023 Jul)

Feature matching

1. LightGlue (2023 June 26)

Speech

Speech recognition

1. OpenAI Whisper (2022 Sept 21)

Text

Text generation

1. BLOOM (2022 July)

2. GALACTICA (2022 Nov)

  • Summary: A general-purpose scientific language model. It is trained on a large corpus of scientific text and data. It can perform scientific NLP tasks at a high level, as well as tasks such as citation prediction, mathematical reasoning, molecular property prediction and protein annotation.
  • Resources

3. GPT-GJT (Dec 2022)

4. PubMed GPT 2.7B (2022 Dec)

Summary: A language model trained on biomedical literature which delivers an improved state of the art for medical question answering.

5. nanoGPT (2022 Dec)

  • Summary: The simplest, fastest repository for training/finetuning medium-sized GPTs
  • Resources

7. Petals (2022 Dec)

8. Chat-RWKV (Jan 2023)

9. LLaMA (Feb 24, 2023)

11. Falcon

12. FinGPT (2023 Jun)

13. Llama2

14. Mistral 7B

15. Gemma (2024 Feb 21)

Embeddings

1. StarSpace (2017)

  • Summary: Learning embeddings for classification, retrieval and ranking.
  • Resources:
    1. Paper
    2. Github

2. Jina Embeddings-v2

Image - Language

1. OpenCLIP ()

2. IF ()

3. TinyGPT-V

4. LLaVa (2023 Apr)

5. moondream (2024 Jan)

6. Large World Model (2024 Feb)

Speech - Language

Text to Speech

1. Coqui-TTS

2. TorToiSe

  • Summary: A multi-voice TTS system trained with an emphasis on quality
  • Resources

3. AudioGPT

  • Summary: Understanding and Generating Speech, Music, Sound, and Talking Head
  • Resources:

4. suno-ai/bark

  • Summary: Text-Prompted Generative Audio Model
  • Resources

5. EmotiVoice

  • Summary: a powerful and modern open-source text-to-speech engine. EmotiVoice speaks both English and Chinese, and with over 2000 different voices. The most prominent feature is emotional synthesis, allowing you to create speech with a wide range of emotions, including happy, excited, sad, angry and others.
  • Resources

6. MeloTTS

  • Summary: High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.
  • Resources:

Speech to Text

1. Coqui - STT

  • Summary: An open-source deep-learning toolkit for training and deploying speech-to-text models.
  • Resources:

Tabular data

Transformers

1. Tab Transformers

3D rendering

NeRF

1. NVIDIA Instant-NGP

2. Shap-E (2023, May 3)

3. Neuralangelo (Jun 2023)

AI Tools

Language

1. langchain

2. xturing

3. LocalAI

  • Summary: Self-hosted, community-driven simple local OpenAI-compatible API written in go
  • Resources:

4. Lamini

5. CodeTF

  • Summary: One-stop Transformer Library for State-of-the-art Code LLM
  • Resources

6. MLC-LLM (2023 Mar)

7. GPT4All

8. OpenChatKit (2023 Mar 10)

  • Summary: OpenChatKit provides a powerful, open-source base to create both specialized and general purpose models for various applications. The kit includes an instruction-tuned language models, a moderation model, and an extensible retrieval system for including up-to-date responses from custom repositories
  • Resources:

9. FreedomGPT

  • Summary: A React and Electron-based app that executes the FreedomGPT LLM locally (offline and private) on Mac and Windows using a chat-based interface (based on Alpaca Lora)
  • Resources

10. Open-Assistant

11. SuperAGI

  • Summary: A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
  • Resources

12. exllamav2

  • Summary: A fast inference library for running LLMs locally on modern consumer-class GPUs
  • Resources

13. QAnything

  • Summary: a local knowledge base question-answering system designed to support a wide range of file formats and databases, allowing for offline installation and use
  • Resources:

14. llmware

  • Summary: Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models.
  • Resources:

Vision

1. PixelLib

2. StreamDiffusion

3. Supervision

4. InvokeAI

  • Summary: InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media
  • Resources:

Video

1. Roop (2023 Jun)

2. ShortGPT (2023 Jul)

  • Summary: ShortGPT is a powerful framework for automating content creation. It simplifies video creation, footage sourcing, voiceover synthesis, and editing tasks.
  • Resources:

Audio

1. JARVIS

  • Summary: a voice assistant made as an experiment using neural networks with Rust
  • Resources:

Multi-modal

1. TaskMatrix

2. Transformer Agents

3. LibreChat

  • Summary: Enhanced ChatGPT Clone: Features OpenAI, GPT-4 Vision, Bing, Anthropic, OpenRouter, Google Gemini, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. More features in development
  • Resources:

4. open-interpreter

Dataset management

1. fiftyone

  • Summary: The open-source tool for building high-quality datasets and computer vision models
  • Resources:

AI Libraries

General

  1. ColossalAI - Making large AI models cheaper, faster and more accessible

Vision

  1. monai - medical imaging with deep learning
  2. supervision - We write your reusable computer vision tools

Audio

  1. SpeechBrain - An Open-Source Conversational AI Toolkit

Language

  1. OpenNMT - An open source neural machine translation system
  2. outlines - Neuro Symbolic Text Generation
  3. llm-foundry - LLM training code for MosaicML foundation models
  4. chainlit - Build Python LLM apps in minutes!
  5. languagemodels - Explore large language models on any computer with 512MB of RAM
  6. lit-gpt - Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Multi-modal

  1. rasa - Open source machine learning framework to automate text- and voice-based conversations
    1. RasaGPT - headless LLM chatbot platform

Miscellaneous

Model Zoo

  1. modelzoo.co - Discover open source deep learning code and pretrained models.
  2. OpenVINO Model Zoo - Model zoo from multiple sources
  3. replicate - easy to use setup for popular models
  4. modelscope - bring the notion of Model-as-a-Service to life
  5. https://civitai.com/
  6. open-llms - A list of open LLMs available for commercial use.

AI in the wild

  1. AI Product Index - A curated index to track AI-powered products.
  2. awesome-generative-ai - A curated list of modern Generative Artificial Intelligence projects and services
  3. LinkedIn Post - Commercial use LLMs - List of commercially usable LLMs
  4. ai-collection - A Collection of Awesome Generative AI Applications
  5. tuning-playbook - A playbook for systematically maximizing the performance of deep learning models.
  6. ollama - Get up and running with large language models, locally.
  7. inference - Replace OpenAI GPT with another LLM in your app by changing a single line of code
  8. llama-embeddings-fastapi-service - designed to facilitate and optimize the process of obtaining text embeddings using different LLMs

About

An index of openly available AI models and tools