imvladikon

pylate

Late Interaction Models Training & Retrieval

Liger-Kernel

Efficient Triton Kernels for LLM Training

relik

Retrieve, Read and LinK: Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget (ACL 2024)

outlines

Structured Text Generation

Bend

A massively parallel, high-level programming language

transformer-heads

Toolkit for attaching, training, saving and loading of new heads for transformer models

neural-tree

Tree-based indexes for neural-search

SONAR

SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders.

transformers-CFG

🤗 A specialized library for integrating context-free grammars (CFG) in EBNF with the Hugging Face Transformers

Filtered-Semi-Markov-CRF

Code for our paper accepted at EMNLP 2023 (Findings)

whisper.cpp

Port of OpenAI's Whisper model in C/C++

MedTator

A Serverless Text Annotation Tool for Corpus Development

focus

[EMNLP'23] Official Code for "FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models"

distil-whisper

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

MEGA

Multilingual Evaluation of LLMs

masakhane-ner

voyager

🛰️ An approximate nearest-neighbor search library for Python and Java with a focus on ease of use, simplicity, and deployability.

self-rag

This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi.

alignment-handbook

Robust recipes to align language models with human and AI preferences

DiffusionLLM

Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"

dspy

DSPy: The framework for programming—not prompting—foundation models

prompt2model

prompt2model - Generate Deployable Models from Natural Language Instructions

usearch

Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

riveter-nlp

Package to extract connotation frames

charred

CHARacter-awaRE Diffusion: Multilingual Character-Aware Encoders for Font-Aware Diffusers That Can Actually Spell

webie

Dataset for web-scaled information extraction.

ScalableVectorSearch

UD_Hebrew-IAHLTKnesset

factorizer

Hebrew-Question-Answering-Dataset

A question answering dataset in Modern Hebrew, containing 30,147 questions.