llama-cpp

There are 9 repositories under llama-cpp topic.

getumbrel / llama-gpt
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
ai chatgpt code-llama codellama gpt gpt-4 gpt4all llama llama-2 llama-cpp llama2 llamacpp llm localai openai self-hosted
Language:TypeScript 10995
SciSharp / LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
chatbot gpt llama llama-cpp llama2 llama3 llamacpp llava llm multi-modal semantic-kernel
Language:C# 3427
maid
Mobile-Artificial-Intelligence / maid
Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
android android-ai chatbot chatgpt facebook flutter free-chatgpt gguf large-language-models llama llama-cpp llama2 llamacpp local-ai mistral mobile-ai mobile-artificial-intelligence ollama openai openorca
Language:Dart 2197
node-llama-cpp
withcatai / node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
ai bindings catai llama llama-cpp llm nodejs prebuilt-binaries grammar gguf cuda metal json-schema cmake cmake-js self-hosted embedding function-calling gpu vulkan
Language:TypeScript 1740
gotzmann / llama.go
llama.go is like llama.cpp in pure Golang!
alpaca chatgpt dalai gpt gpt3 gpt4 gpt4all llama llama-cpp llm vicuna
Language:Go 1388
undreamai / LLMUnity
Create characters in Unity with LLMs!
ai chat chatbot conversational-ai dialogue generative-ai llama llm npc unity unity2d unity3d character game-development gamedev llama-cpp rag
Language:C# 1338
mybigday / llama.rn
React Native binding of llama.cpp
android ios llama llama-cpp llm react-native
Language:C 712
docker / compose-for-agents
Build and run AI agents using Docker Compose. A collection of ready-to-use examples for orchestrating open-source LLMs, tools, and agent runtimes.
agentic-workflows ai-agents docker docker-compose examples large-language-models llama-cpp openai-gym self-hosted
Language:TypeScript 708
the-crypt-keeper / can-ai-code
Self-evaluating interview for AI coders
ai ggml langchain llama-cpp llm humaneval transformers
Language:Python 597
withcatai / catai
Run AI ✨ assistant locally! with simple API for Node.js 🚀
chatgpt ai dalai openai chatbot chatui llama-cpp ai-assistant vicuna vicuna-installation-guide wizardlm catai llm ggmlv3 local-llm localai gguf node-llama-cpp nodejs
Language:TypeScript 479
mdrokz / rust-llama.cpp
LLama.cpp rust bindings
api-bindings cpp crates-io ffi llama llama-cpp machine-learning model rust
Language:Rust 407
dipampaul17 / KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
apple-silicon generative-ai kv-cache llama-cpp llm m1 m2 m3 memory-optimization metal optimization quantization
Language:Python 360
jlonge4 / local_llama
This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.
llamaindex python artificial-intelligence langchain machinelearning llama-cpp offline
Language:Python 293
gpustack / gguf-parser-go
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
gguf go llama-box llama-cpp stable-diffusion-cpp
Language:Go 215
lucasjinreal / Crane
A Pure Rust based LLM (Any LLM based MLLM such as Spark-TTS) Inference Engine, powering by Candle framework.
llama-cpp mllm qwen2-vl qwen3 rust spark-tts
Language:Rust 190
ptsochantaris / emeltal
Local ML voice chat using high-end models.
ai llama-cpp machine-learning macos ml natural-language-processing speech-recognition swift swiftui user-interface whisper-cpp
Language:C++ 178
phronmophobic / llama.clj
Run LLMs locally. A clojure wrapper for llama.cpp.
clojure llama llama-cpp llm
Language:Clojure 166
gotzmann / booster
Booster - open accelerator for LLM models. Better inference and debugging for AI hackers
chatgpt exllama ggml gpt llama llama-cpp llamacpp llm ollama oobabooga openai vllm
Language:C++ 163
shady.ai
BrutalCoding / shady.ai
Making offline AI models accessible to all types of edge devices.
android cross-platform dart fastlane flutter gguf ios linux linux-desktop llama-cpp llama-dart llvm macos material-design rwkv serverpod shady-ai web whisper-cpp windows
Language:Dart 142
nuance1979 / llama-server
LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.
chatbot-ui llama-cpp llamacpp llama
Language:Python 130
1038lab / ComfyUI-MiniCPM
A custom ComfyUI node for MiniCPM vision-language models, supporting v4, v4.5, and v4 GGUF formats, enabling high-quality image captioning and visual analysis.
comfyui custom-nodes gguf llama-cpp stable-diffusion minicpm minicpm-v muti-models
Language:Python 126
nrl-ai / CustomChar
Your customized AI assistant - Personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2.
cpp llama-cpp llm stt tts whisper-cpp ggml llama llama-v2
Language:C++ 118
thushan / olla
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
ai llm-inference lmstudio ollama proxy vllm golang llamacpp llm-proxy llm-router llm-routing self-hosted self-hosted-ai amd intel llama-cpp mlx nvidia sglang local-ai
Language:Go 110
R3gm / InsightSolver-Colab
InsightSolver: Colab notebooks for exploring and solving operational issues using deep learning, machine learning, and related models.
ai-ops aiops autogpt colab-notebook colorization computer-vision deep-learning llama-2 llama-cpp llm machine-learning object-detection stable-diffusion text-to-speech
Language:Jupyter Notebook 101
vtuber-plan / langport
Langport is a language model inference service
api chatgpt chatgpt-api fauxpilot langchain language-model llama llama-cpp llm openai tabby
Language:Python 95
robiwan303 / babyagi
BabyAGI-🦙: Enhanced for Llama models (running 100% local) and persistent memory, with smart internet search based on BabyCatAGI and document embedding in langchain based on privateGPT
agi task-based chatgpt reasoning llm ai-agents artificial-general-intelligence artificial-intelligence autonomous-agent openai-api python babyagi google-search-api llama llama-cpp serpapi langchain
Language:Python 90
OpenCSGs / llm-inference
llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.
deepspeed llama-cpp llm-inference ray transformer vllm
Language:Python 88
Abhi5h3k / PrivateDocBot
📚 Local PDF-Integrated Chat Bot: Secure Conversations and Document Assistance with LLM-Powered Privacy
chatgpt llm ai gpt gpt-4 gpt4all llama llama-2 llama-cpp llama2 llamacpp localai openai self-hosted generative pdf private privategpt vectorstore langchain
Language:Python 87
greynewell / musegpt
Local LLMs in your DAW!
ai ai-music daw juce juce-plugins llama-cpp llamacpp llm music-production vst vst-plugin vst3
Language:C++ 82
ImpAI
rbourgeat / ImpAI
😈 ImpAI is an advanced role play app using large language and diffusion models.
llm roleplay stable-diffusion ai character-ai chat game ggml gguf llama llama-cpp linux macos windows docker
Language:JavaScript 63
ystemsrx / code-atlas
A C++ implementation of Open Interpreter. / Open Interpreter 的 C++ 实现
ai api cpp cross-platform fine-tuning interpreter lightweight llama-cpp llm local open-interpreter
Language:C++ 63
fboulnois / llama-cpp-docker
Run llama.cpp in a GPU accelerated Docker container
chatgpt llama llama2 mistral llama-cpp llm docker docker-compose llama3
Language:Dockerfile 55
hyparam / hyllama
llama.cpp gguf file parser for javascript
gguf javascript llamacpp llama-cpp machine-learning ml parser llm js hyperparam
Language:JavaScript 50
iacopPBK / llama.cpp-gfx906
llama.cpp-gfx906
amd-gpu flash-attention kernel llama-cpp rocm gfx906 mi50 vega20
Language:C++ 50
lordmathis / llamactl
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
llama-cpp llama-server llamacpp llm llm-inference llm-router localllama localllm mlx mlx-lm openai-api self-hosted vllm
Language:Go 50
blueraai / universal-intelligence
◉ Universal Intelligence: AI made simple.
agent-framework agentic agentic-framework agentic-workflow ai-agents ai-models ai-tools artificial-intelligence javascript language-model llama-cpp llm mcp mlx python transformers typescript uin web-llm universal-intelligence
Language:Python 46

llama-cpp

getumbrel / llama-gpt

SciSharp / LLamaSharp

Mobile-Artificial-Intelligence / maid

withcatai / node-llama-cpp

gotzmann / llama.go

undreamai / LLMUnity

mybigday / llama.rn

docker / compose-for-agents

the-crypt-keeper / can-ai-code

withcatai / catai

mdrokz / rust-llama.cpp

dipampaul17 / KVSplit

jlonge4 / local_llama

gpustack / gguf-parser-go

lucasjinreal / Crane

ptsochantaris / emeltal

phronmophobic / llama.clj

gotzmann / booster

BrutalCoding / shady.ai

nuance1979 / llama-server

1038lab / ComfyUI-MiniCPM

nrl-ai / CustomChar

thushan / olla

R3gm / InsightSolver-Colab

vtuber-plan / langport

robiwan303 / babyagi

OpenCSGs / llm-inference

Abhi5h3k / PrivateDocBot

greynewell / musegpt

rbourgeat / ImpAI

ystemsrx / code-atlas

fboulnois / llama-cpp-docker

hyparam / hyllama

iacopPBK / llama.cpp-gfx906

lordmathis / llamactl

blueraai / universal-intelligence