language-model

There are 132 repositories under language-model topic.

transformers
huggingface / transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
bert deep-learning flax hacktoberfest jax language-model language-models machine-learning model-hub natural-language-processing nlp nlp-library pretrained-models python pytorch pytorch-transformers seq2seq speech-recognition tensorflow transformer
Language:Python 126123
gpt4free
xtekky / gpt4free
The official gpt4free repository | various collection of powerful language models
chatbot chatbots chatgpt chatgpt-4 chatgpt-api chatgpt-free chatgpt4 free-gpt gpt gpt-3 gpt-4 gpt3 gpt4 gpt4-api language-model openai openai-api openai-chatgpt python reverse-engineering
Language:Python 58021
dair-ai / Prompt-Engineering-Guide
🐙 Guides, papers, lecture, notebooks and resources for prompt engineering
chatgpt deep-learning language-model openai prompt-engineering
Language:MDX 44438
generative-ai-for-beginners
microsoft / generative-ai-for-beginners
18 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/
ai azure chatgpt dall-e generative-ai generativeai gpt language-model llms openai prompt-engineering semantic-search transformers
Language:Jupyter Notebook 43850
LAION-AI / Open-Assistant
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
ai assistant chatgpt discord-bot language-model machine-learning nextjs python rlhf
Language:Python 36711
tatsu-lab / stanford_alpaca
Code and documentation to train Stanford's Alpaca models, and generate the data.
deep-learning instruction-following language-model
Language:Python 28911
LLaMA-Factory
hiyouga / LLaMA-Factory
Unify Efficient Fine-Tuning of 100+ LLMs
agent ai chatglm fine-tuning gpt instruction-tuning language-model large-language-models llama llama3 llm lora mistral moe peft qlora quantization qwen rlhf transformers
Language:Python 22150
mlc-ai / mlc-llm
Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
language-model llm machine-learning-compilation tvm
Language:Python 17186
haystack
deepset-ai / haystack
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
nlp question-answering bert language-model pytorch semantic-search squad information-retrieval summarization transformers machine-learning ai python chatgpt gpt-3 large-language-models generative-ai
Language:Python 13870
BlinkDL / RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
attention-mechanism deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn transformer transformers rwkv chatgpt
Language:Python 11747
mlc-ai / web-llm
Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.
chatgpt deep-learning language-model llm tvm webgpu webml
Language:TypeScript 9795
BlinkDL / ChatRWKV
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
chatbot chatgpt language-model pytorch rnn rwkv
Language:Python 9304
microsoft / LoRA
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
gpt-2 adaptation language-model gpt-3 low-rank pytorch deep-learning roberta deberta lora
Language:Python 9235
brightmart / nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
chinese-dataset chinese-corpus pretrain word2vec nlp bert language-model wiki news question-answering chinese corpus chinese-nlp dataset text-classification
9207
mlfoundations / open_clip
An open source implementation of CLIP.
deep-learning pytorch computer-vision language-model multi-modal-learning contrastive-loss zero-shot-classification pretrained-models
Language:Jupyter Notebook 8610
tokenizers
huggingface / tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
nlp natural-language-processing natural-language-understanding language-model transformers bert gpt
Language:Rust 8508
EleutherAI / gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
gpt gpt-2 gpt-3 language-model transformers
Language:Python 8150
OptimalScale / LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
chatgpt deep-learning instruction-following language-model pretrained-models pytorch transformer
Language:Python 8045
speechbrain / speechbrain
A PyTorch-based Speech Toolkit
speech-recognition speech-toolkit speaker-recognition speech-to-text speech-enhancement speech-separation audio audio-processing speech-processing speechrecognition asr voice-recognition spoken-language-understanding speaker-diarization speaker-verification pytorch huggingface transformers language-model deep-learning
Language:Python 7964
vercel / ai
Build AI-powered applications with React, Svelte, Vue, and Solid
artificial-intelligence generative-ai generative-ui huggingface javascript language-model llm nextjs openai react solidjs svelte typescript vercel vue
Language:TypeScript 7923
txtai
neuml / txtai
💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
python search machine-learning nlp semantic-search neural-search vector-search txtai llm vector-database language-model transformers sentence-embeddings large-language-models information-retrieval search-engine vector-search-engine embeddings retrieval-augmented-generation rag
Language:Python 7096
EleutherAI / gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
deepspeed-library gpt-3 language-model transformers
Language:Python 6624
OpenNMT / OpenNMT-py
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
deep-learning language-model llms machine-translation neural-machine-translation pytorch
Language:Python 6600
codertimo / BERT-pytorch
Google AI 2018 BERT pytorch implementation
bert transformer pytorch nlp language-model
Language:Python 6031
ramonvc / freegpt-webui
GPT 3.5/4 with a Chat Web UI. No API key required.
apifree chatgpt chatgpt-api chatgpt-clone freegpt freegpt4 gpt-4 gpt-4-api gpt-4-free gpt-free gpt-interface gpt3 gpt3-api gpt3-turbo gpt4 gpt4all gptfree jailbreak language-model website
Language:Python 5722
EleutherAI / lm-evaluation-harness
A framework for few-shot evaluation of language models.
evaluation-framework language-model transformer
Language:Python 5230
THUDM / CogVLM
a state-of-the-art-level open visual language model | 多模态预训练模型
cross-modality language-model multi-modal pretrained-models visual-language-models
Language:Python 5209
aiwaves-cn / agents
An Open-source Framework for Autonomous Language Agents
autonomous-agents language-model llm
Language:Python 4565
CLUEbenchmark / CLUE
中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
nlu benchmark chinese corpus dataset bert albert chineseglue glue roberta language-model pretrained-models transformers tensorflow pytorch
Language:Python 3848
ravenscroftj / turbopilot
Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
code-completion cpp language-model machine-learning
Language:C++ 3832
yizhongw / self-instruct
Aligning pretrained language models with instruction data generated by themselves.
general-purpose-model language-model instruction-tuning
Language:Python 3821
spark-nlp
JohnSnowLabs / spark-nlp
State of the Art Natural Language Processing
nlp natural-language-processing spark pyspark named-entity-recognition sentiment-analysis lemmatizer spell-checker entity-extraction part-of-speech-tagger bert transformers albert tensorflow language-detection machine-translation text-classification language-model llm question-answering
Language:Scala 3713
xlang-ai / OpenAgents
OpenAgents: An Open Platform for Language Agents in the Wild
agent assistant-chat-bots code-generation executable-langauge-grounding gpt hacktoberfest language-model language-model-agent llm semantic-parsing tool-learning ui
Language:Python 3613
hiyouga / ChatGLM-Efficient-Tuning
Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调
chatglm chatgpt fine-tuning lora alpaca peft huggingface language-model transformers pytorch rlhf chatglm2 qlora
Language:Python 3611
mlfoundations / open_flamingo
An open-source framework for training large multimodal models.
computer-vision deep-learning flamingo in-context-learning language-model multimodal-learning pytorch
Language:Python 3493
xtekky / chatgpt-clone
ChatGPT interface with better UI
chatgpt chatgpt-api chatgpt-app chatgpt-clone gpt-4 gpt-4-api gpt3 gpt4 gpt4-api gpt4all interface language language-model site ui gpt-interface
Language:Python 3381