thunlp / OpenMatch

An Open-Source Package for Information Retrieval.

This repo is deprecated. Please check out our new repo at https://github.com/OpenMatch/OpenMatch.

OpenMatch v2

An all-in-one toolkit for information retrieval. Under active development.

Install

git clone https://github.com/thunlp/OpenMatch.git
cd OpenMatch
pip install -e .

-e means editable, i.e. you can change the code directly in your directory.

We do not include all the requirements in the package. You may need to manually install torch, tensorboard.

You may also need faiss for dense retrieval. You can install either faiss-cpu or faiss-gpu, according to your enviroment. Note that if you want to perform search on GPUs, you need to install the version of faiss-gpu compatible with your CUDA. In some cases (usually CUDA >= 11.0) pip installs a wrong version. If you encounter errors during search on GPUs, you may try installing it from conda.

Features

• Human-friendly interface for dense retriever and re-ranker training and testing
• Various PLMs supported (BERT, RoBERTa, T5...)
• Native support for common IR & QA Datasets (MS MARCO, NQ, KILT, BEIR, ...)
• Deep integration with Huggingface Transformers and Datasets
• Efficient training and inference via stream-style data loading

Docs

See docs folder.

Project Organizers

• Zhiyuan Liu
  • Tsinghua University
  • Homepage
• Zhenghao Liu
  • Northeastern University
  • Homepage
• Chenyan Xiong
  • Microsoft Research AI
  • Homepage
• Maosong Sun
  • Tsinghua University
  • Homepage

Acknowledgments

Our implementation uses Tevatron as the starting point. We thank its authors for their contributions.

Contact

Please email to [e-mail].