XuexII

promptsource

Toolkit for creating, sharing and using natural language prompts.

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

sglang

SGLang is yet another fast serving framework for large language models and vision language models.

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

dclm

DataComp for Language Models

mem0

The memory layer for Personalized AI

LESS

[ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning

reasoning-teacher

Official code for "Large Language Models Are Reasoning Teachers", ACL 2023

LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs

consistent-CoT-distillation

sqllineage

SQL Lineage Analysis Tool powered by Python

simulated-trial-and-error

mm-cot

Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)

conformer

[Unofficial] PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

GenAI-System-2-Attention-S2A-by-Meta

datasets from the paper "Towards Understanding Sycophancy in Language Models"

auto-cot

Official implementation for "Automatic Chain of Thought Prompting in Large Language Models" (stay tuned & more will be updated)

SymbCoT

Codes and Data for ACL 2024 Paper "Faithful Logical Reasoning via Symbolic Chain-of-Thought".

translation-agent

deduplicate-text-datasets

Platypus

Code for fine-tuning Platypus fam LLMs using LoRA

cc_net

Tools to download and cleanup Common Crawl data

alpagasus

Unofficial implementation of AlpaGasus

open-instruct

SimPO

SimPO: Simple Preference Optimization with a Reference-Free Reward

perspectiveapi

Perspective is an API that uses machine learning models to score the perceived impact a comment might have on a conversation. See https://developers.perspectiveapi.com for more information.

AlpacaDataCleaned

Alpaca dataset from Stanford, cleaned and curated

DPR

Dense Passage Retriever - is a set of tools and models for open domain Q&A task.

orpo

Official repository for ORPO

AdvancedLiterateMachinery

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.