aiflash

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

Awesome-LLMOps

An awesome & curated list of best LLMOps tools for developers

fauxpilot

FauxPilot - an open-source alternative to GitHub Copilot server

litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

TaskWeaver

A code-first agent framework for seamlessly planning and executing data analytics tasks.

gpt4all

gpt4all: run open-source LLMs anywhere

h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/

private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks

llm

Access large language models from the command-line

continue

⏩ Continue enables you to create your own AI code assistant inside your IDE. Keep your developers in flow with open-source VS Code and JetBrains extensions

llama-cpp-python

Python bindings for llama.cpp

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

llama.cpp

LLM inference in C/C++

codegpt

VSCode extension that allows you to use GPT3 inside the IDE

ollama-autocoder

A simple to use Ollama autocompletion engine with options exposed and streaming functionality

magicoder

Magicoder: Source Code Is All You Need

ollama

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

open-local

cloud-native local storage management system for stateful workload, low-latency with simplicity

HummerRisk

HummerRisk 是云原生安全平台，包括混合云安全治理和云原生安全检测。

isotope

AWS superpowers with the power of bedrock and speed of rust

ChatGPT-Next-Web

A cross-platform ChatGPT/Gemini UI (Web / PWA / Linux / Win / MacOS). 一键拥有你自己的跨平台 ChatGPT/Gemini 应用。

ChatGLM2-6B

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

flaxkv

🗲 A high-performance on-disk dictionary.

LocalAI

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

openai-forward

🚀 大语言模型高效转发服务 · An efficient forwarding service designed for LLMs. · OpenAI API Reverse Proxy

blog

kubefire

KubeFire 🔥, creates and manages Kubernetes Clusters using Firecracker microVMs

vscodium

binary releases of VS Code without MS branding/telemetry/licensing

k0sctl

A bootstrapping and management tool for k0s clusters.