jqhoogland

pygments

Pygments is a generic syntax highlighter written in Python

polars

Dataframes powered by a multithreaded, vectorized query engine, written in Rust

nanotron

Minimalistic large language model 3D-parallelism training

lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

CircuitsVis

Mechanistic Interpretability Visualizations using React

arrow2

Transmute-free Rust library to work with the Arrow format

torcharrow

High performance model preprocessing library on PyTorch

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

diskhash

Diskbased (persistent) hashtable

kneser-ney

Kneser-Ney implementation in Python

kenlm

KenLM: Faster and Smaller Language Model Queries

opt_einsum

⚡️Optimizing einsum functions in NumPy, Tensorflow, Dask, and more with contraction order optimization.

xla

Enabling PyTorch on XLA Devices (e.g. Google TPU)

pyvene

Stanford NLP Python Library for Understanding and Improving PyTorch Models via Interventions

flash-attention

Fast and memory-efficient exact attention

algorithmica

A computer science textbook

penzai

A JAX research toolkit for building, editing, and visualizing neural networks.

promptsource

Toolkit for creating, sharing and using natural language prompts.

kronfluence

Influence Functions with (Eigenvalue-corrected) Kronecker-Factored Approximate Curvature

slt-of-dd

Singular Learning of Double Descent

cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

ml-engineering

Machine Learning Engineering Open Book

mistral-inference

Official inference library for Mistral models

mistral

Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging Face 🤗 Transformers.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

lm-evaluation-harness

A framework for few-shot evaluation of language models.

anonymous_github

Anonymous Github is a proxy server to support anonymous browsing of Github repositories for open-science code and data.

ARENA_3.0

aider

aider is AI pair programming in your terminal

gd2md-html

Convert a Google Doc to Markdown or HTML. This Docs add-on converts a Google Doc to simple Markdown and/or HTML.