language-model

There are 168 repositories under language-model topic.

transformers
huggingface / transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
audio deep-learning deepseek gemma glm hacktoberfest llm machine-learning model-hub natural-language-processing nlp pretrained-models python pytorch pytorch-transformers qwen speech-recognition transformer vlm
Language:Python 149801
generative-ai-for-beginners
microsoft / generative-ai-for-beginners
21 Lessons, Get Started Building with Generative AI
ai chatgpt dall-e generativeai gpt azure generative-ai llms openai prompt-engineering language-model semantic-search transformers microsoft-for-beginners
Language:Jupyter Notebook 97904
LLMs-from-scratch
rasbt / LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
ai artificial-intelligence chatgpt deep-learning from-scratch gpt language-model large-language-models llm machine-learning python pytorch transformer
Language:Jupyter Notebook 71013
gpt4free
xtekky / gpt4free
The official gpt4free repository | various collection of powerful language models | o4, o3 and deepseek r1, gpt-4.1, gemini 2.5
chatbot chatbots chatgpt chatgpt-4 chatgpt-api chatgpt-free chatgpt4 deepseek deepseek-api deepseek-r1 gpt gpt-4 gpt-4o gpt4 gpt4-api language-model openai openai-api openai-chatgpt reverse-engineering
Language:Python 65109
dair-ai / Prompt-Engineering-Guide
🐙 Guides, papers, lecture, notebooks and resources for prompt engineering
chatgpt deep-learning generative-ai language-model openai prompt-engineering
Language:MDX 61864
LLaMA-Factory
hiyouga / LLaMA-Factory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
agent ai deepseek fine-tuning gemma gpt instruction-tuning large-language-models llama llama3 llm lora moe nlp peft qlora quantization qwen rlhf transformers
Language:Python 58281
LAION-AI / Open-Assistant
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
ai assistant chatgpt discord-bot language-model machine-learning nextjs python rlhf
Language:Python 37468
tatsu-lab / stanford_alpaca
Code and documentation to train Stanford's Alpaca models, and generate the data.
deep-learning instruction-following language-model
Language:Python 30148
haystack
deepset-ai / haystack
AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
nlp question-answering pytorch semantic-search information-retrieval summarization transformers machine-learning ai python large-language-models generative-ai llm rag retrieval-augmented-generation agents agent gemini gpt-4 orchestration
Language:Python 22487
mlc-ai / mlc-llm
Universal LLM Deployment Engine with ML Compilation
language-model llm machine-learning-compilation tvm
Language:Python 21333
yamadashy / repomix
📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.
ai llm generative-ai chatgpt claude developer-tools chatbot gpt openai javascript nodejs typescript anthropic gemini language-model artificial-intelligence genai deepseek llama mcp
Language:TypeScript 19201
vercel / ai
The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents
anthropic artificial-intelligence gemini generative-ai generative-ui javascript language-model llm nextjs openai react svelte typescript vercel vue
Language:TypeScript 17739
mlc-ai / web-llm
High-performance In-browser LLM Inference Engine
chatgpt deep-learning language-model llm tvm webgpu webml
Language:TypeScript 16442
arc53 / DocsGPT
DocsGPT is an open-source genAI tool that helps users get reliable answers from knowledge source, while avoiding hallucinations. It enables private and reliable information retrieval, with tooling and agentic system capability built in.
ai chatgpt docsgpt hacktoberfest information-retrieval language-model llm machine-learning natural-language-processing python pytorch rag react semantic-search transformers web-app
Language:TypeScript 15529
BlinkDL / RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
attention-mechanism deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn transformer transformers rwkv chatgpt
Language:Python 13969
mlfoundations / open_clip
An open source implementation of CLIP.
computer-vision contrastive-loss deep-learning language-model multi-modal-learning pretrained-models pytorch zero-shot-classification
Language:Python 12606
microsoft / LoRA
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
adaptation deberta deep-learning gpt-2 gpt-3 language-model lora low-rank pytorch roberta
Language:Python 11712
txtai
neuml / txtai
💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
ai artificial-intelligence embeddings information-retrieval language-model large-language-models llm machine-learning nlp python rag retrieval-augmented-generation search search-engine semantic-search sentence-embeddings transformers txtai vector-database vector-search
Language:Python 10702
tokenizers
huggingface / tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
bert gpt language-model natural-language-processing natural-language-understanding nlp transformers
Language:Rust 10071
speechbrain / speechbrain
A PyTorch-based Speech Toolkit
asr audio audio-processing deep-learning huggingface language-model pytorch speaker-diarization speaker-recognition speaker-verification speech-enhancement speech-processing speech-recognition speech-separation speech-to-text speech-toolkit speechrecognition spoken-language-understanding transformers voice-recognition
Language:Python 9660
brightmart / nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
chinese-dataset chinese-corpus pretrain word2vec nlp bert language-model wiki news question-answering chinese corpus chinese-nlp dataset text-classification
9658
BlinkDL / ChatRWKV
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
chatbot chatgpt language-model pytorch rnn rwkv
Language:Python 9472
EleutherAI / lm-evaluation-harness
A framework for few-shot evaluation of language models.
evaluation-framework language-model transformer
Language:Python 8590
OptimalScale / LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
chatgpt deep-learning instruction-following language-model pretrained-models pytorch transformer
Language:Python 8395
EleutherAI / gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
gpt gpt-2 gpt-3 language-model transformers
Language:Python 8289
EleutherAI / gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
deepspeed-library gpt-3 language-model transformers
Language:Python 7301
LostRuins / koboldcpp
Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
gemma ggml gguf koboldai koboldcpp language-model llama llamacpp llm mistral
Language:C++ 7015
OpenNMT / OpenNMT-py
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
deep-learning language-model llms machine-translation neural-machine-translation pytorch
Language:Python 6857
THUDM / CogVLM
a state-of-the-art-level open visual language model | 多模态预训练模型
cross-modality language-model multi-modal pretrained-models visual-language-models
Language:Python 6466
codertimo / BERT-pytorch
Google AI 2018 BERT pytorch implementation
bert transformer pytorch nlp language-model
Language:Python 6366
ramon-victor / freegpt-webui
GPT 3.5/4 with a Chat Web UI. No API key required.
apifree chatgpt chatgpt-api chatgpt-clone gpt-4-api gpt-interface gpt3 gpt3-api gpt3-turbo gpt4 gpt4all gptfree jailbreak language-model website freegpt freegpt4 gpt-free gpt-4 gpt-4-free
Language:Python 5673
aiwaves-cn / agents
An Open-source Framework for Data-centric, Self-evolving Autonomous Language Agents
autonomous-agents language-model llm
Language:Python 5557
NexaAI / nexa-sdk
On device AI inference in minutes—now for MLX & GGUF and Qualcomm NPU, with Android and iOS coming soon.
edge-computing llm on-device-ai on-device-ml sdk stable-diffusion transformers vlm language-model go
Language:Go 4785
yizhongw / self-instruct
Aligning pretrained language models with instruction data generated by themselves.
general-purpose-model instruction-tuning language-model
Language:Python 4475
xlang-ai / OpenAgents
[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild
llm code-generation executable-langauge-grounding language-model semantic-parsing language-model-agent hacktoberfest agent assistant-chat-bots gpt tool-learning ui
Language:Python 4229
CLUEbenchmark / CLUE
中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
nlu benchmark chinese corpus dataset bert albert chineseglue glue roberta language-model pretrained-models transformers tensorflow pytorch
Language:Python 4108

language-model

huggingface / transformers

microsoft / generative-ai-for-beginners

rasbt / LLMs-from-scratch

xtekky / gpt4free

dair-ai / Prompt-Engineering-Guide

hiyouga / LLaMA-Factory

LAION-AI / Open-Assistant

tatsu-lab / stanford_alpaca

deepset-ai / haystack

mlc-ai / mlc-llm

yamadashy / repomix

vercel / ai

mlc-ai / web-llm

arc53 / DocsGPT

BlinkDL / RWKV-LM

mlfoundations / open_clip

microsoft / LoRA

neuml / txtai

huggingface / tokenizers

speechbrain / speechbrain

brightmart / nlp_chinese_corpus

BlinkDL / ChatRWKV

EleutherAI / lm-evaluation-harness

OptimalScale / LMFlow

EleutherAI / gpt-neo

EleutherAI / gpt-neox

LostRuins / koboldcpp

OpenNMT / OpenNMT-py

THUDM / CogVLM

codertimo / BERT-pytorch

ramon-victor / freegpt-webui

aiwaves-cn / agents

NexaAI / nexa-sdk

yizhongw / self-instruct

xlang-ai / OpenAgents

CLUEbenchmark / CLUE