L's starred repositories
LLMTest_NeedleInAHaystack
Doing simple retrieval from LLM models at various context lengths to measure accuracy
InfiniteBench
Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718
RAGChecker
RAGChecker: A Fine-grained Framework For Diagnosing RAG
VideoLLaMA2
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Megatron-Kwai
[USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism
ring-flash-attention
Ring attention implementation with flash attention
MedicalGPT
MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。
UltraFeedback
A large-scale, fine-grained, diverse preference dataset (and models).
RedPajama-Data
The RedPajama-Data repository contains code for preparing large datasets for training large language models.
long-context-attention
Sequence Parallel Attention for Long Context LLM Model Training and Inference
awesome-multi-modal-reinforcement-learning
A curated list of Multi-Modal Reinforcement Learning resources (continually updated)
llm-autoeval
Automatically evaluate your LLMs in Google Colab
persona-hub
Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"