Kyriection

grok-1

Grok open release

LLaMA-Factory

Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)

llm.c

LLM training in simple, raw C/CUDA

SWE-agent

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges.

mistral-src

Reference implementation of Mistral AI 7B v0.1 model.

LWM

FlagEmbedding

Retrieval and Retrieval-augmented LLMs

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

transformer-debugger

dbrx

Code examples and resources for DBRX, a large language model developed by Databricks

schedule_free

Schedule-Free Optimization in PyTorch

Awesome-Graph-LLM

A collection of AWESOME things about Graph-Related LLMs.

TimeSformer

The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

pyreft

ReFT: Representation Finetuning for Language Models

Triton-Puzzles

Puzzles for learning Triton

Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Neural-Network-Diffusion

We introduce a novel approach for parameter generation, named neural network parameter diffusion (p-diff), which employs a standard latent diffusion model to synthesize a new set of parameters

recurrentgemma

Open weights language model from Google DeepMind, based on Griffin.

long-llms-learning

A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarks

LLMs-Finetuning-Safety

We jailbreak GPT-3.5 Turbo’s safety guardrails by fine-tuning it on only 10 adversarially designed examples, at a cost of less than $0.20 via OpenAI’s APIs.

scattermoe

Triton-based implementation of Sparse Mixture of Experts.

tinyBenchmarks

Evaluating LLMs with fewer examples

SVD-LLM

Official Code for "SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression"

GRIFFIN

resta

Restore safety in fine-tuned language models through task arithmetic

Ms-PoE

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.