adamlin120

Open-Reasoning-Tasks

A comprehensive repository of reasoning tasks for LLMs (and beyond)

text-clustering

Easily embed, cluster and semantically label text datasets

flash-attention

Fast and memory-efficient exact attention

TMLU

Taiwanese Mandarin Language Modeling

zh-tw-embedding-model-benchmark

使用繁體中文資料集做的 Embedding 模型評測

SystemChat

TWLLM-Tutor

Taiwan-LLM Tutor: Large Language Models for Taiwanese Secondary Education

DataDreamer

DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤

cluster-health

ai-workshop-code

Code I wrote for my AI & LLM workshops

llm.c

LLM training in simple, raw C/CUDA

tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

awesome-synthetic-datasets

awesome synthetic (text) datasets

cohere-toolkit

Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.

shisa-v2

Japanese / English Bilingual LLM

text-dedup

All-in-one text de-duplication

SemDeDup

Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically similar, but not exactly identical).

chat-ui

Open source codebase powering the HuggingChat app

axolotl

Go ahead and axolotl questions

research-career-tools

BTX

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

composer

Supercharge Your Model Training

lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

echo

Sensei

Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI

ml-engineering

Machine Learning Engineering Open Book

unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

tabbyAPI

An OAI compatible exllamav2 API that's both lightweight and fast

LLaMA-Factory

A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)