Rumen Dangovski's starred repositories
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
mistral-src
Reference implementation of Mistral AI 7B v0.1 model.
consistency_models
Official repo for consistency models.
Single-Player-MCTS
🌳 Python implementation of single-player Monte-Carlo Tree Search.
contrastive-phase-transitions
Studying Phase Transitions in Contrastive Learning with Physics-Inspired Datasets
ModelStitching_LookingForFunctionalSimilarityBetweenRepresentations
Contrastive Learning SuperUROP Fall 2021 - Spring 2022
state-spaces
Sequence Modeling with Structured State Spaces