Sagor Sarker's starred repositories
LLaMA-Factory
Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)
LLMs-from-scratch
Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step
llama3-from-scratch
llama3 implementation one matrix multiplication at a time
text-generation-inference
Large Language Model Text Generation Inference
bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
Awesome-LLMOps
An awesome & curated list of best LLMOps tools for developers
datasketch
MinHash, LSH, LSH Forest, Weighted MinHash, HyperLogLog, HyperLogLog++, LSH Ensemble and HNSW
helm
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in Holistic Evaluation of Text-to-Image Models (HEIM) (https://arxiv.org/abs/2311.04287).
Awesome-LLMs-Datasets
Summarize existing representative LLMs text datasets.
text-clustering
Easily embed, cluster and semantically label text datasets
llm_distillation_playbook
Best practices for distilling large language models.
IndicLLMSuite
A blueprint for creating Pretraining and Fine-Tuning datasets for Indic languages
LLMEvaluation
A comprehensive guide to LLM evaluation methods designed to assist in identifying the most suitable evaluation techniques for various use cases, promote the adoption of best practices in LLM assessment, and critically assess the effectiveness of these evaluation methods.
Awesome_Bangla_Datasets
Awesome Bangla Datasets