jorirsan

UTMOSv2

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

RapidFuzz

Rapid fuzzy string matching in Python using various string metrics

jiwer

Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

outlines

Structured Text Generation

epitran

A tool for transcribing orthographic text as IPA (International Phonetic Alphabet)

WMT-Biomed-Test

toLLMatch

toLLMatch🔪: Context-aware LLM-based simultaneous translation

simul_whisper

Code for our INTERSPEECH paper Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detection

NAT_vs_AT

ccextractor

CCExtractor - Official version maintained by the core team

whisper-timestamped

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

MLVU

🔥🔥MLVU: Multi-task Long Video Understanding Benchmark

VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

NAST-S2x

A fast speech-to-any translation model that supports simultaneous decoding and offers 28× speedup.

paella-core

Paella Player core library

mbrs

A library for minimum bayes risk (MBR) decoding

eole

Open language modeling toolkit based on PyTorch

eamt24-linguistic-mt

A repo for resources for our EAMT 2024 tutorial

llm-foundry

LLM training code for Databricks foundation models

ParaDocs

CroCoAlign

A Cross-Lingual, Context-Aware and Fully-Neural Sentence Alignment System for Long Texts.

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

apptainer

Apptainer: Application containers for Linux

compare-mt

A tool for holistic analysis of language generations systems

lingua-py

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

tensorrt_backend

The Triton backend for TensorRT.

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

konoha

🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.