syncdoth

llm.c

LLM training in simple, raw C/CUDA

pyhgf

PyHGF: A neural network library for predictive coding

predictive-forward-forward

Implementation/simulation of the predictive forward-forward credit assignment algorithm for training neurobiologically-plausible recurrent neural network models.

mesop

LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

Awesome-LLM-KG

Awesome papers about unifying LLMs and KGs

pykan

Kolmogorov Arnold Networks

litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

miner-release

Stable Diffusion and LLM miner for Heurist

grok-1

Grok open release

Sophia

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

safari

Convolutions for Sequence Modeling

axolotl

Go ahead and axolotl questions

BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

mergekit

Tools for merging pretrained large language models.

nanotron

Minimalistic large language model 3D-parallelism training

datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

DoLa

Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"

pretraining

Pretraining

TransformerLens

A library for mechanistic interpretability of GPT-style language models

lmql

A language for constraint-guided and efficient LLM programming.

outlines

Structured Text Generation

guidance

A guidance language for controlling large language models.

NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

mamba

Mamba SSM architecture

FilmstripMaker

c++ Program for adding images together into a film strip. Useful in creating GUIs

honest_llama

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model

Knowledge-Constrained-Decoding

Official Code for EMNLP2023 Main Conference paper: "KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection"