Sebastian Müller's repositories
rebasin-results
Results for snimu/rebasin
dspy-redteam-tests
Red-Teaming Language Models with DSPy
hlb-gpt-cli
CLI controllable version of hlb-gpt by tysam-code
attention-experiments
I'm playing around with Attention mechanisms
dspy
DSPy: The framework for programming—not prompting—foundation models
etbl-vision
Embracing the bitter lesson (vision)
gradient-rounding
Round the gradient during LLM training to different degrees; compare "scaling" of rounding to different significant digits to parameter scaling
guarantees
Python: guarantee test coverage, guarantee type and runtime-guarantees
hlb-CIFAR10
Train to 94% on CIFAR-10 in ~6.84 seconds on a single A100, the current world speed record. Or ~95.78% in ~114 seconds (or less!)
hlb-gpt
Minimalistic, fast, and experimentation-friendly researcher's toolbench for GPT-like models in ~<365 lines of code. Reaches <3.8 validation loss on wikitext-103 on a single A100 in ~138 seconds.
llm-parameter-stats
How do parameter statistics change over training in LLMs?
neuralsort
Sort lists with the help of an ANN to allow maximal parallelism in execution.
parameter-checks
Extend typehints to include dynamic checks (that might otherwise be dealt with by assertions) in Python.
px4-simulation-ignition
Fix issue #19981 on PX4-Autopilot
torch-benchmarks
Performance benchmark for PyTorch models
hlb-gpt-value-activation
Check out how much of a difference the activation of the value makes vs. keeping it linear as in standard attention
llm-small-to-large
1. Train small LLM; 2. Use its outputs on the training data as labels for training large LLM, where their argmax agrees with the training data.
mask
Some experiments with Attention masks
plan-act
A better way for LLMs to plan before acting.
sglang
SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with LLMs faster and more controllable.
torch-nested
Easily manipulate torch.Tensors inside highly nested data-structures.
torchinfo
View model summaries in PyTorch!
typing-exe
Executable typehints for Python: make assertions about and/or modify parameters & return values
ul2
How much information can we extract from one token?