Giters

thiyanesh / The-NLP-Pandect

A comprehensive reference for all topics related to Natural Language Processing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The-NLP-Pandect

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find almost anything related to Natural Language Processing that is available online.

The-NLP-Resources

Compendiums and awesome lists on the topic of NLP:

NLP Conferences, Paper Summaries and Paper Compendiums:

Papers and Paper Summaries
Conferences

NLP Progress and NLP Tasks:

NLP Datasets:

Word and Sentence embeddings:

Notebooks, Scripts and Repositories

Non-English resources and compendiums

Pre-trained NLP models

NLP History

General
2020 Year in Review

The-NLP-Podcasts

NLP-only podcasts

Many NLP episodes

Some NLP episodes

The-NLP-Newsletter

The-NLP-Meetups

The-NLP-Youtube

The-NLP-Benchmarks

General NLU

  • GLUE - General Language Understanding Evaluation (GLUE) benchmark
  • SuperGLUE - benchmark styled after GLUE with a new set of more difficult language understanding tasks
  • decaNLP - The Natural Language Decathlon (decaNLP) for studying general NLP models
  • RACE - ReAding Comprehension dataset collected from English Examinations
  • dialoglue - DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue
  • DynaBench - Dynabench is a research platform for dynamic data collection and benchmarking

Summarization

  • WikiAsp - WikiAsp: Multi-document aspect-based summarization Dataset

Question Answering

  • SQuAD - Stanford Question Answering Dataset (SQuAD)
  • XQuad - XQuAD (Cross-lingual Question Answering Dataset) for cross-lingual question answering
  • GrailQA - Strongly Generalizable Question Answering (GrailQA)
  • CSQA - Complex Sequential Question Answering

Multilingual and Non-English Benchmarks

  • XTREME - Massively Multilingual Multi-task Benchmark
  • GLUECoS - A benchmark for code-switched NLP
  • IndoNLU Benchmark - collection of resources for training, evaluating, and analyzing NLP for Bahasa Indonesia
  • IndicGLUE - Natural Language Understanding Benchmark for Indic Languages
  • LinCE - Linguistic Code-Switching Evaluation Benchmark
  • Russian SuperGlue - Russian SuperGlue Benchmark

Bio, Law, and other scientific domains

  • BLURB - Biomedical Language Understanding and Reasoning Benchmark
  • BLUE - Biomedical Language Understanding Evaluation benchmark

Transformer Efficiency

Speech Processing

  • SUPERB - Speech processing Universal PERformance Benchmark

Other

  • CodeXGLUE - A benchmark dataset for code intelligence
  • CrossNER - CrossNER: Evaluating Cross-Domain Named Entity Recognition
  • MultiNLI - Multi-Genre Natural Language Inference corpus

The-NLP-Research

General

Embeddings

Repositories

Blogs

Cross-lingual Word and Sentence Embeddings

  • vecmap - VecMap (cross-lingual word embedding mappings) [GitHub, 553 stars]
  • sentence-transformers - Multilingual Sentence & Image Embeddings with BERT [GitHub, 5498 stars]

Byte Pair Encoding

  • bpemb - Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE) [GitHub, 969 stars]
  • subword-nmt - Unsupervised Word Segmentation for Neural Machine Translation and Text Generation [GitHub, 1735 stars]
  • python-bpe - Byte Pair Encoding for Python [GitHub, 147 stars]

Transformer-based Architectures

General

Transformer

BERT

Other Transformer Variants

T5
BigBird
Reformer / Linformer / Longformer / Performers
Switch Transformer

GPT-family

General
GPT-3
Learning Resources
Applications
  • Aweseome GPT-3 - list of all resources related to GPT-3 [GitHub, 3219 stars]
  • GPT-3 Projects - a map of all GPT-3 start-ups and commercial projects
  • GPT-3 Demo Showcase - GPT-3 Demo Showcase, 180+ Apps, Examples, & Resources
  • OpenAI API - API Demo to use GPT-3 for commercial applications
Open-source Efforts

Other

Distillation, Pruning and Quantization

Automated Summarization

Rule-based NLP

  • LemmInflect - A python module for English lemmatization and inflection

The-NLP-Industry

Best Practices for NLP

Transformer-based Architectures

General
Multi-GPU Transfomers

Embeddings as a Service

NLP Recipes Industrial Applications:

NLP Applications in Bio, Finance, Legal and other industries

  • Blackstone - A spaCy pipeline and model for NLP on unstructured legal text [GitHub, 492 stars]
  • Sci spaCy - spaCy pipeline and models for scientific/biomedical documents [GitHub, 954 stars]
  • FinBERT: Pre-Trained on SEC Filings for Financial NLP Tasks [GitHub, 147 stars]
  • LexNLP - Information retrieval and extraction for real, unstructured legal text [GitHub, 458 stars]
  • NerDL and NerCRF - Tutorial on Named Entity Recognition for Healthcare with SparkNLP
  • Legal Text Analytics - A list of selected resources dedicated to Legal Text Analytics [GitHub, 294 stars]
  • BioIE - A curated list of resources relevant to doing Biomedical Information Extraction [GitHub, 155 stars]

Model and Data testing

  • WildNLP - Corrupt an input text to test NLP models' robustness [GitHub, 65 stars]
  • Great Expectations - Write tests for your data [GitHub, 4653 stars]
  • CheckList - Beyond Accuracy: Behavioral Testing of NLP models [GitHub, 1414 stars]
  • TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1538 stars]

Deployability

The-NLP-Speech

General Speech Recognition

  • wav2letter - Automatic Speech Recognition Toolkit [GitHub, 5798 stars]
  • DeepSpeech - Baidu's DeepSpeech architecture [GitHub, 17664 stars]
  • Acoustic Word Embeddings by Maria Obedkova [Blog, 2020]
  • kaldi - Kaldi is a toolkit for speech recognition [GitHub, 10595 stars]
  • awesome-kaldi - resources for using Kaldi [GitHub, 429 stars]
  • ESPnet - End-to-End Speech Processing Toolkit [GitHub, 3918 stars]
  • HuBERT - Self-supervised representation learning for speech recognition, generation, and compression [Blog, June 2021]

Text to Speech

  • FastSpeech - The Implementation of FastSpeech based on pytorch [GitHub, 630 stars]
  • TTS - a deep learning toolkit for Text-to-Speech [GitHub, 1883 stars]

The-NLP-Topics

Blogs

Frameworks for Topic Modeling

  • gensim - framework for topic modeling [GitHub, 12224 stars]
  • Spark NLP [GitHub, 2234 stars]

Repositories

Keyword-Extraction

Text Rank

  • PyTextRank - PyTextRank is a Python implementation of TextRank as a spaCy pipeline extension [GitHub, 1565 stars]
  • textrank - TextRank implementation for Python 3 [GitHub, 1037 stars]

RAKE - Rapid Automatic Keyword Extraction

  • rake-nltk - Rapid Automatic Keyword Extraction algorithm using NLTK [GitHub, 831 stars]
  • yake - Single-document unsupervised keyword extraction [GitHub, 704 stars]
  • RAKE-tutorial - A python implementation of the Rapid Automatic Keyword Extraction [GitHub, 352 stars]
  • rake-nltk - Rapid Automatic Keyword Extraction algorithm using NLTK [GitHub, 831 stars]

Other Approaches

  • flashtext - Extract Keywords from sentence or Replace keywords in sentences [GitHub, 4837 stars]
  • BERT-Keyword-Extractor - Deep Keyphrase Extraction using BERT [GitHub, 190 stars]
  • keyBERT - Minimal keyword extraction with BERT [GitHub, 683 stars]

Further Reading

Responsible-NLP

NLP and ML Interpretability

  • Language Interpretability Tool (LIT) [GitHub, 2589 stars]
  • WhatLies - Toolkit to help visualise - what lies in word embeddings [GitHub, 274 stars]
  • Interpret-Text - Interpretability techniques and visualization dashboards for NLP models [GitHub, 257 stars]
  • InterpretML - Fit interpretable models. Explain blackbox machine learning [GitHub, 3924 stars]
  • ecco - Tools to visuals and explore NLP language models [GitHub, 815 stars]
  • NLP Profiler - A simple NLP library allows profiling datasets with text columns [GitHub, 196 stars]
  • transformers-interpret - Model explainability that works seamlessly with transformers [GitHub, 405 stars]
  • Awesome-explainable-AI - collection of research materials on explainable AI/ML [GitHub, 316 stars]

Ethics, Bias, and Equality in NLP

Adversarial Attacks for NLP

The-NLP-Frameworks

General Purpose

  • spaCy by Explosion AI [GitHub, 20858 stars]
  • flair by Zalando [GitHub, 10550 stars]
  • AllenNLP by AI2 [GitHub, 10311 stars]
  • stanza (former Stanford NLP) [GitHub, 5508 stars]
  • spaCy stanza [GitHub, 540 stars]
  • nltk [GitHub, 9978 stars]
  • gensim - framework for topic modeling [GitHub, 12224 stars]
  • pororo - Platform of neural models for natural language processing [GitHub, 941 stars]
  • NLP Architect - A Deep Learning NLP/NLU library by Intel® AI Lab [GitHub, 2697 stars]
  • FARM [GitHub, 1263 stars]
  • gobbli by RTI International [GitHub, 256 stars]
  • headliner - training and deployment of seq2seq models [GitHub, 229 stars]
  • SyferText - A privacy preserving NLP framework [GitHub, 178 stars]
  • DeText - Text Understanding Framework for Ranking and Classification Tasks [GitHub, 1135 stars]
  • TextHero - Text preprocessing, representation and visualization [GitHub, 2239 stars]
  • textblob - TextBlob: Simplified Text Processing [GitHub, 7725 stars]
  • AdaptNLP - A high level framework and library for NLP [GitHub, 326 stars]
  • textacy - NLP, before and after spaCy [GitHub, 1703 stars]
  • texar - Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow [GitHub, 2177 stars]
  • jiant - jiant is an NLP toolkit [GitHub, 1282 stars]

Data Augmentation

Tools

  • WildNLP Text manipulation library to test NLP models [GitHub, 65 stars]
  • snorkel Framework to generate training data [GitHub, 4692 stars]
  • NLPAug Data augmentation for NLP [GitHub, 2193 stars]
  • SentAugment Data augmentation by retrieving similar sentences from larger datasets [GitHub, 323 stars]
  • faker - Python package that generates fake data for you [GitHub, 12753 stars]
  • textflint - Unified Multilingual Robustness Evaluation Toolkit for NLP [GitHub, 470 stars]
  • Parrot - Practical and feature-rich paraphrasing framework [GitHub, 244 stars]
  • AugLy - data augmentations library for audio, image, text, and video [GitHub, 3600 stars]

Papers & Blogs

Adversarial NLP Attacks

  • TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1538 stars]
  • CleverHans - adversarial example library for constructing NLP attacks and building defenses [GitHub, 5184 stars]

Non-English oriented

  • textblob-de - TextBlob: Simplified Text Processing for German [GitHub, 86 stars]
  • Kashgari Transfer Learning with focus on Chinese [GitHub, 2141 stars]
  • Underthesea - Vietnamese NLP Toolkit [GitHub, 863 stars]

Transformer-oriented

  • transformers by HuggingFace [GitHub, 48261 stars]
  • Adapter Hub and its documentation - Adapter modules for Transformers [GitHub, 470 stars]
  • haystack - Transformers at scale for question answering & neural search. [GitHub, 2067 stars]

Dialog Systems and Speech

  • DeepPavlov by MIPT [GitHub, 5287 stars]
  • ParlAI by FAIR [GitHub, 7300 stars]
  • rasa - Framework for Conversational Agents [GitHub, 11670 stars]
  • wav2letter - Automatic Speech Recognition Toolkit [GitHub, 5798 stars]
  • ChatterBot - conversational dialog engine for creating chat bots [GitHub, 11280 stars]

Word/Sentence-embeddings oriented

  • MUSE A library for Multilingual Unsupervised or Supervised word Embeddings [GitHub, 2812 stars]
  • vecmap A framework to learn cross-lingual word embedding mappings [GitHub, 553 stars]
  • sentence-transformers - Multilingual Sentence & Image Embeddings with BERT [GitHub, 5498 stars]

Social Media Oriented

  • Ekphrasis - text processing tool, geared towards text from social networks [GitHub, 479 stars]

Phonetics

  • DeepPhonemizer - grapheme to phoneme conversion with deep learning [GitHub, 51 stars]

Multi-lingual tools

  • polyglot - Multi-lingual NLP Framework [GitHub, 1864 stars]
  • trankit - Light-Weight Transformer-based Python Toolkit for Multilingual NLP [GitHub, 488 stars]

Distributed NLP / Multi-GPU NLP

Machine Translation

  • COMET -A Neural Framework for MT Evaluation [GitHub, 65 stars]
  • marian-nmt - Fast Neural Machine Translation in C++ [GitHub, 808 stars]
  • argos-translate - Open source neural machine translation in Python [GitHub, 646 stars]
  • Opus-MT - Open neural machine translation models and web services [GitHub, 139 stars]
  • dl-translate - A deep learning-based translation library built on Huggingface transformers [GitHub, 152 stars]

Entity and String Matching

  • PolyFuzz - Fuzzy string matching, grouping, and evaluation [GitHub, 356 stars]
  • pyahocorasick - Python module implementing Aho-Corasick algorithm for string matching [GitHub, 619 stars]
  • fuzzywuzzy - Fuzzy String Matching in Python [GitHub, 8292 stars]
  • jellyfish - approximate and phonetic matching of strings [GitHub, 1489 stars]
  • textdistance - Compute distance between sequences [GitHub, 1997 stars]
  • DeepMatcher - Compute distance between sequences [GitHub, 347 stars]

Discourse Analysis

  • ConvoKit - Cornell Conversational Analysis Toolkit [GitHub, 278 stars]

PII scrubbing

  • scrubadub - Clean personally identifiable information from dirty dirty text [GitHub, 240 stars]

The-NLP-Learning

General

Books

Courses

Tutorials

The-NLP-Communities

Other-NLP-Topics

Tokenization

  • tokenizers - Fast State-of-the-Art Tokenizers optimized for Research and Production [GitHub, 4680 stars]
  • SentencePiece - Unsupervised text tokenizer for Neural Network-based text generation [GitHub, 5191 stars]
  • SoMaJo - A tokenizer and sentence splitter for German and English web and social media texts [GitHub, 91 stars]

Data Augmentation and Weak Supervision

Libraries and Frameworks
  • WildNLP Text manipulation library to test NLP models [GitHub, 65 stars]
  • snorkel Framework to generate training data [GitHub, 4692 stars]
  • NLPAug Data augmentation for NLP [GitHub, 2193 stars]
  • SentAugment Data augmentation by retrieving similar sentences from larger datasets [GitHub, 323 stars]
  • TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1538 stars]
  • skweak - software toolkit for weak supervision applied to NLP tasks [GitHub, 310 stars]
Blogs and Tutorials

Named Entity Recognition (NER)

Relation Extraction

  • tacred-relation TACRED: position-aware attention model for relation extraction [GitHub, 286 stars]
  • tacrev TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [GitHub, 42 stars]
  • tac-self-attention Relation extraction with position-aware self-attention [GitHub, 59 stars]

Coreference Resolution

Domain Adaptation

Low Resource NLP

Spell Correction

  • Gramformer - ramework for detecting, highlighting and correcting grammatical errors [GitHub, 588 stars]
  • NeuSpell - A Neural Spelling Correction Toolkit [GitHub, 211 stars]
  • SymSpellPy - Python port of SymSpell [GitHub, 448 stars]
  • Speller100 by Microsoft [Blog, Feb 2021]

Style Transfer for NLP

  • Styleformer - Neural Language Style Transfer framework [GitHub, 189 stars]

Automata Theory for NLP

  • pyahocorasick - Python module implementing Aho-Corasick algorithm for string matching [GitHub, 619 stars]

Obscene words detection

  • LDNOOBW - List of Dirty, Naughty, Obscene, and Otherwise Bad Words [GitHub, 1443 stars]

Reinforcement Learning for NLP

  • nlp-gym - NLPGym - A toolkit to develop RL agents to solve NLP tasks [GitHub, 91 stars]

AutoML / AutoNLP

  • AutoNLP - Faster and easier training and deployments of SOTA NLP models [GitHub, 591 stars]
  • TPOT - Python Automated Machine Learning tool [GitHub, 8109 stars]
  • Auto-PyTorch - Automatic architecture search and hyperparameter optimization for PyTorch [GitHub, 1263 stars]
  • HungaBunga - Brute-Force all sklearn models with all parameters using .fit .predict [GitHub, 625 stars]
  • AutoML Natural Language - Google's paid AutoML NLP service

Text Generation

License CC0

Attributions

Resources

  • All linked resources belong to original authors

Icons

Fonts

The Pandect Series also includes

     

About

A comprehensive reference for all topics related to Natural Language Processing

License:Creative Commons Zero v1.0 Universal

Languages

Language:Python 100.0%