German-NLP
Curated list of open-access/open-source/off-the-shelf resources and tools developed with a particular focus on German
Resources and tools which can be used either off-the-shelf or with minor adjustments and which are currently maintained are primarily chosen for this list. It is deliberately biased in terms of usability and user-friendliness.
Pull requests and suggestions are welcome! See contributing guidelines.
Table of Contents
- Corpora
- Generic resources
- Linguistic processing
- Semantic analysis
- Speech NLP
- Machine Translation
- Teaching resources and tutorials
- More lists
Corpora
General-purpose
- Araneum Germanicum
- COW
- Digitales Wörterbuch der deutschen Sprache (DWDS)
- IDS Corpora
- Leipzig Corpora Collection
- SdeWaC
Historical
- Anselm (14th-16th centuries)
- Deutsches Textarchiv
- Elektronische Texte (Thomas Gloning)
- German Drama Corpus
- Lesekorpus Altdeutsch (750-1050)
- Referenzkorpus Altdeutsch (750-1050)
- Referenzkorpus Mittelhochdeutsch (1050-1350)
- Referenzkorpus Mittelniederdeutsch/Niederrheinisch (1200-1650)
- Referenzkorpus Frühneuhochdeutsch (1350-1650)
- Thesaurus Indogermanischer Text- und Sprachmaterialien (TITUS)
Specialized
- arg-microtexts
- auto-hMDS (multi-document summarization)
- Dortmunder Chat Korpus
- German EUROPARL data w/ NE annotation
- German Political Speeches Corpus
- GermaParl (Bundestag)
- One Million Posts Corpus
- Open Speech Data Corpus
- Potsdam Commentary Corpus (PCC)
- TTLab StadtWiki Corpus
Swiss German
- ArchiMob Corpus
- NOAH's Corpus of Swiss German Dialects
- SpinningBytes Swiss German Sentiment Corpus
- Swiss SMS Corpus
Learner and Error Corpora
Word lists
- DeReWo
- DiMLex (lexicon of German discourse markers)
- German Compound Database
- German derivational lexicons
- German nouns from Wiktionary
- Morphological Dictionaries (DEMorphy)
- German Wiktionary Lexicon Graph
- German word list for GNU Aspell
- OpenThesaurus
Lists
- CLARIN-D list
- Corpora at the IMS
- CorpusExplorer's list of corpora
- Korpora am IMS
- Parallel corpora (see below)
- Treebanks (see below)
- ZAS list
Data acquisition
Generic resources
Frameworks
- AmbiverseNLU
- CLARIN-D web tools
- CorpusExplorer
- DKPro Core
- DKPro Similarity
- DKPro Text Classification (TC)
- DKPro Word Sense Disambiguation (WSD)
- flair
- ixa pipes
- Mate Tools, webservice via WebLicht
- spaCy
- Stanford CoreNLP
- textblob-de
- TextImager
Treebanks
- German Universal Dependency Treebank
- Hamburg Dependency Treebank
- NEGRA
- TIGER Corpus
- TGermaCorp (literary texts)
- TüBa-D/Z
Annotation
Standards
Linguistic processing
Tokenization
Stemming
Lemmatization
Morphological analysis
- CharSplit
- DEMorphy
- Durm Lemmatizer
- HypheNN-de
- jwordsplitter
- MarMoT
- Morphy
- morphisto
- SECOS (unsupervised compound splitter)
- SFST
- SMOR, webservice via WebLicht
- zmorge
Normalization
POS-tagging
- clevertagger
- hunpos
- moot
- pattern.de
- RFTagger, webservice via WebLicht
- SoMeWeTa
- TnT
- TreeTagger
Syntactical parsing
Named Entity Recognition
- AmbiverseNLU KnowNER
- flair
- GermaNER
- LSTM+CRF+FastText with models for (historic) German
- microNER
- ner-corpora
- (Faruqui & Pado 2010) Components and evaluation data
Text generation
Industry/Applications
Evaluation
Semantic analysis
Datasets
- Complex Word Identification (DE, EN, ES, FR)
- Distributional memories: DM.de TransDM.de
- Distributional thesauri (includes German)
- Lexical Chains
- Logical metonymy database
- schulteimwalde.de/resources.html
- Semantic Relations in Context
- UKP Darmstadt data list
Word embeddings and senses
- disco (semantic similarity)
- GermaNet
- german2vec
- GermanWordEmbeddings
- Open German WordNet
- sensegram
- SpinningBytes word embeddings (tweets)
- UBY Linked Lexical Resource
Sentiment analysis datasets / polarity clues
- GermanPolarityClues
- HeiST – Heidelberg Sentiment Treebank
- Potsdam Twitter Sentiment Corpus (PotTS)
- Sentiment Lexicon (Univ. Zurich)
- SentimentWortschatz
- SpinningBytes Swiss German Sentiment Corpus
Sentiment detection
GermEval
(category to improve)
- Official GermEval tools list
- GermEval 2015 data (Lexical Substitution)
- Germeval Task 2017
- GermEval-2018 data
- germeval-rug
- IWG_hatespeech_public
- jpadillamontani/germeval2018
- uhh-lt/GermEval2017-Baseline
- UKP embeddings for GermEval 2017
Discourse
- Bilingual formality (T/V) corpus (EN/DE)
- Bilingual FrameNet frame embeddings (EN/DE)
- Bilingual parallel frame-semantic annotation (EN/DE)
- CorZu (coreference resolution)
- Discourse Segmenter
- Frame Identification
- German social media textual entailment dataset
- PropS-DE (proposition structures)
Summarization
Psycholinguistics
Speech NLP
- Archiv für gesprochenes Deutsch
- BAS ressources
- Bochumer Korpus der gesprochenen Sprache im Ruhrgebiet
- Database for Spoken German (IDS Mannheim)
- (D)iscourse (I)nformation (R)adio (N)ews (D)atabase for (L)inguistic Analysis
- Hamburger Zentrum für Sprachkorpora
- kaldi-tuda-de
Machine Translation
Parallel corpora
Teaching resources and tutorials
- bubenhofer.com/korpuslinguistik/kurs/
- CorpusExplorer v2.0 – Seminartauglich in einem halben Tag
- deeplearning4nlp-tutorial
- Uni Zürich: Sprachtechnologie in den Digital Humanities – MOOC Youtube & Coursera
More lists
German
- CLARIN VLO (DE+public)
- computerlinguistik.org
- LRE Map
- MetaShare Language Resources
- Peter Kolb's list
- Swiss German Language Processing
General
- GitHub topics corpus-linguistics & nlp
- nlp-datasets
- NLP-progress
- /r/LanguageTechnology/
Comparable lists
- awesome-nlp
- Awesome Community-Curated NLP List
- awesome-chinese-nlp
- awesome-hungarian-nlp
- Indonesian NLP
- awesome-spanish-nlp
- M. Weisser's list of NLP/Computational Linguistics Resources
- NLP tools (Saarland University)
- W. Roberts' Computational Linguistics Links
