gentaiscool

miners

MINERS ⛏️: The semantic retrieval benchmark for evaluating multilingual language models.

gentaiscool.github.io

My website

metametrics

acl-anthology

Data and software for building the ACL Anthology.

code-switching-papers

A curated list of research papers and resources on code-switching

human-preference-papers

distfuse

A library to calculate similarity scores between two collections of text sequences encoded using transformer models for bitext mining, dense retrieval, retrieval-based classification, and retrieval-augmented generation (RAG).

mteb

MTEB: Massive Text Embedding Benchmark

calcs2023_ingest

indonesian-nlp

A curated list of research papers and resources on Indonesian languages

aclpub2

matrix_fact

Matrix Factorization Library

mt-metrics-eval

Tools for evaluating the performance of MT metrics on data from recent WMT metrics shared tasks.

calcs2023

calcs2023_test

do-we-need-attention

lstm-attention

Attention-based bidirectional LSTM for Classification Task (ICASSP)

DataLab

The unified platform for data-related resources.

meta-emb

Multilingual Meta-Embeddings for Named Entity Recognition (RepL4NLP & EMNLP 2019)

lm-evaluation-harness

A framework for few-shot evaluation of autoregressive language models.

promptsource

Toolkit for creating, sharing and using natural language prompts.

few-shot-lm

The source code of "Language Models are Few-shot Multilingual Learners" (MRL @ EMNLP 2021)

nusa-datasets

end2end-asr-pytorch

End-to-End Automatic Speech Recognition on PyTorch

BIG-bench

Beyond the Imitation Game collaborative benchmark for enormous language models

xnli-dataset

PromptPapers

Must-read papers on prompt-based tuning for pre-trained language models.

NER-datasets

Datasets to train supervised classifiers for Named-Entity Recognition in different languages (Portuguese, German, Dutch, French, English)

NL-Augmenter

NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations

mesh-transformer-jax

Model parallel transformers in JAX and Haiku