gguf

There are 8 repositories under gguf topic.

LostRuins / koboldcpp
Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
koboldcpp llamacpp llm koboldai llama ggml gguf gemma language-model mistral
Language:C++ 8193
Sidekick
johnbean393 / Sidekick
A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.
agentic-ai agents ai ai-agents aichat chatbot deep-research deepseek deepseek-r1 gemma3 llama llama4 llm macos qwen qwen3 rag swift swiftui
Language:Swift 3074
menloresearch / cortex.cpp
Local AI API Platform
gguf llamacpp onnx onnxruntime
Language:C++ 2761
maid
Mobile-Artificial-Intelligence / maid
Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
android chatbot chatgpt facebook flutter gguf large-language-models llama llama-cpp llama2 llamacpp mistral openai openorca local-ai ollama mobile-ai android-ai mobile-artificial-intelligence free-chatgpt
Language:Dart 2152
heshengtao / comfyui_LLM_party
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG
comfyui openai workflow agent dify graphrag gemini llama ollama omost o1 gpt-sovits linux flux gguf vlm ocr mcp janus-pro
Language:Python 1905
node-llama-cpp
withcatai / node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
ai bindings catai llama llama-cpp llm nodejs prebuilt-binaries grammar gguf cuda metal json-schema cmake cmake-js self-hosted embedding function-calling gpu vulkan
Language:TypeScript 1656
av / harbor
Effortlessly run LLM backends, APIs, frontends, and services with one command.
cli docker docker-compose llm tools ai self-hosted tool bash container exl2 gguf local npm package pypi safetensors mcp
Language:Python 1541
datawhalechina / handy-ollama
动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollama/
agent gguf langchain large-language-models llamaindex llm ollama rag tutorial
Language:Jupyter Notebook 1530
edwko / OuteTTS
Interface for OuteTTS models.
gguf llama text-to-speech transformers tts
Language:Python 1113
gollama
sammcj / gollama
Go manage your Ollama models
ai ggml gguf linux llm macos models ollama tui
Language:Go 1004
mukel / llama3.java
Practical Llama 3 inference in Java
java llama llama3 llm llm-inference llms chatgpt genai gguf huggingface llamacpp openai simd transformers
Language:Java 781
kitops
jozu-ai / kitops
An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.
ai code datasets devops devops-tools gguf hacktoberfest kubernetes kubernetes-deployment ml mlops mlops-tools model-interpretability model-serving models opensource platform-engineering pytorch sklearn tensorflow
Language:Go 736
eastriverlee / LLM.swift
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
ios llm llm-inference macos swift tvos visionos watchos gguf
Language:C 726
withcatai / catai
Run AI ✨ assistant locally! with simple API for Node.js 🚀
chatgpt ai dalai openai chatbot chatui llama-cpp ai-assistant vicuna vicuna-installation-guide wizardlm catai llm ggmlv3 local-llm localai gguf node-llama-cpp nodejs
Language:TypeScript 472
kelindar / search
Go library for embedded vector search and semantic embeddings using llama.cpp
ai bert embeddings gguf gpu llamacpp search-engine semantic-search simd vector-search
Language:Go 428
antirez / gguf-tools
GGUF implementation in C as a library and a tools CLI program
ai gguf llm ml
Language:C 265
Webscout
OE-LUCIFER / Webscout
Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!
ai free api ml python freeai youtube youtube-api g4f openai openinterpreter tempmail tempnumber gguf llamacpp localgpt text-generation websearch ollama
Language:Python 204
mgonzs13 / llama_ros
llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2
cpp gpt llama llm ros2 ggml gguf llamacpp llava vlm llavacpp langchain embeddings rerank reranking
Language:C++ 195
ShelbyJenkins / llm_client
The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes
ai candle gguf llm rust
Language:CSS 194
gpustack / llama-box
LM inference server implementation based on *.cpp.
openai-compatible-api cpp gguf diffusion transformer
Language:C++ 165
shady.ai
BrutalCoding / shady.ai
Making offline AI models accessible to all types of edge devices.
flutter llvm rwkv android cross-platform dart fastlane ios linux-desktop macos material-design web windows llama-cpp whisper-cpp linux gguf shady-ai llama-dart serverpod
Language:Dart 145
gpustack / gguf-parser-go
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
gguf go llama-cpp llama-box stable-diffusion-cpp
Language:Go 140
sinanuozdemir / oreilly-hands-on-gpt-llm
Mastering the Art of Scalable and Efficient AI Model Deployment
deepseek distillation docker gguf gpt groq k8s llamacpp llm mlops quantization
Language:Jupyter Notebook 133
Aesthisia / LLMinator
Gradio based tool to run opensource LLM models directly from Huggingface
chatbot cpu cuda gguf gradio huggingface langchain llama-cpp-python llamacpp llm llm-inference model-conversion ollama openai opensource python safetensors streaming websockets-chat
Language:Python 90
AstraBert / PrAIvateSearch
Own your AI, search the web with it🌐😎
ai docker gguf llamacpp llm local-search-engine open-source phi-3-mini streamlit web-search-engine
Language:Python 84
Mobile-Artificial-Intelligence / lcpp
lcpp is a dart implementation of llama.cpp used by the mobile artificial intelligence distribution (maid)
facebook flutter-ai gemma ggml gguf llama llama2 llamacpp llm llm-inference local-ai meta mistral mixtral mobile-ai
Language:Dart 73
akx / ollama-dl
Download models from the Ollama library, without Ollama
downloader gguf llm ollama
Language:Python 68
ImpAI
rbourgeat / ImpAI
😈 ImpAI is an advanced role play app using large language and diffusion models.
llm roleplay stable-diffusion ai character-ai chat game ggml gguf llama llama-cpp linux macos windows docker
Language:JavaScript 59
mikecvet / nl-sh
The Natural Language Shell integrates OpenAI's GPTs, Anthropic's Claude, or local GGUF-formatted LLMs directly into the terminal experience, allowing operators to describe their tasks in either POSIX commands or fluent human language
gguf gpt-4 llama2 llm rust rust-lang shell shell-script terminal
Language:Rust 58
dreadnode / tensor-man
A utility to inspect, validate, sign and verify machine learning model files.
gguf onnx pytorch safetensors
Language:Rust 55
monk1337 / auto-ollama
run ollama & gguf easily with a single command
gguf inference llm llm-inference lora mistral ollama autogguf autoollama mergelora llama openai
Language:Shell 50
ahoylabs / gguf.js
A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.
ggml gguf large-language-models llamacpp llm machine-learning
Language:TypeScript 47
ShelbyJenkins / llm_utils
llm_utils: Basic LLM tools, best practices, and minimal abstraction.
gguf llm rust segmentation tokenizer nlp
Language:Rust 44
latestissue / AltaeraAI
A set of bash scripts to automate deployment of GGML/GGUF models [default: RWKV] with the use of KoboldCpp on Android - Termux
ai android ggml koboldai koboldcpp rwkv rwkv4 termux llamacpp mistral rwkvcpp phi gguf vicuna mamba tinyllama tinydolphin tinyvicuna
Language:Shell 42
jndiogo / sibila
Extract structured data from local or remote LLM models
ai dataclasses gpt local-models openai pydantic python structured-data structured-extraction gguf large-language-models llamacpp llm-inference local-ai structured-generation
Language:Python 41
hyparam / hyllama
llama.cpp gguf file parser for javascript
gguf javascript llamacpp llama-cpp machine-learning ml parser llm js hyperparam
Language:JavaScript 34

gguf

LostRuins / koboldcpp

johnbean393 / Sidekick

menloresearch / cortex.cpp

Mobile-Artificial-Intelligence / maid

heshengtao / comfyui_LLM_party

withcatai / node-llama-cpp

av / harbor

datawhalechina / handy-ollama

edwko / OuteTTS

sammcj / gollama

mukel / llama3.java

jozu-ai / kitops

eastriverlee / LLM.swift

withcatai / catai

kelindar / search

antirez / gguf-tools

OE-LUCIFER / Webscout

mgonzs13 / llama_ros

ShelbyJenkins / llm_client

gpustack / llama-box

BrutalCoding / shady.ai

gpustack / gguf-parser-go

sinanuozdemir / oreilly-hands-on-gpt-llm

Aesthisia / LLMinator

AstraBert / PrAIvateSearch

Mobile-Artificial-Intelligence / lcpp

akx / ollama-dl

rbourgeat / ImpAI

mikecvet / nl-sh

dreadnode / tensor-man

monk1337 / auto-ollama

ahoylabs / gguf.js

ShelbyJenkins / llm_utils

latestissue / AltaeraAI

jndiogo / sibila

hyparam / hyllama