corpus

There are 20 repositories under corpus topic.

brightmart / nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
chinese-dataset chinese-corpus pretrain word2vec nlp bert language-model wiki news question-answering chinese corpus chinese-nlp dataset text-classification
9777
dariusk / corpora
A collection of small corpuses of interesting data for the creation of bots and similar stuff.
bots corpus language words
Language:JavaScript 5025
CLUEbenchmark / CLUEDatasetSearch
搜索所有中文NLP数据集，附常用英文NLP数据集
nlp datasets chinese ner qa match text-classification machine-translation knowledge-graph corpus machine-reading-comprehension sentiment-analysis text-similarity text-summarization
Language:Python 4369
CLUEbenchmark / CLUE
中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
nlu benchmark chinese corpus dataset bert albert chineseglue glue roberta language-model pretrained-models transformers tensorflow pytorch
Language:Python 4188
wainshine / Chinese-Names-Corpus
中文人名语料库。人名生成器。中文姓名,姓氏,名字,称呼,日本人名,翻译人名,英文人名。可用于中文分词、人名实体识别。
corpus dataset dict names ner
4183
endymecy / awesome-deeplearning-resources
Deep Learning and deep reinforcement learning research papers and some codes
deep-learning reinforcement-learning neural-network paper code video modelzoo corpus nlp
2934
lucasjinreal / weibo_terminater
Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator
chatbot chinese corpus scraper sina weibo
Language:Python 2314
fendouai / Awesome-Chatbot
Awesome Chatbot Projects,Corpus,Papers,Tutorials.Chinese Chatbot =>:
chatbot tensorflow awesome seq2seq seq2seq-chatbot seq2seq-model corpus tutorial
Language:Python 2125
candlewill / Dialog_Corpus
用于训练中英文对话系统的语料库 Datasets for Training Chatbot System
dataset dialog system corpus chatbot
Language:Python 2048
gunthercox / chatterbot-corpus
A multilingual dialog corpus
chatterbot corpus dialog language yaml
Language:Python 1403
NiuTrans / Classical-Modern
非常全的文言文（古文）-现代文平行语料
corpus parallel-corpus traditional-and-simplified-chinese traditional-chinese
Language:Python 1307
wainshine / Company-Names-Corpus
公司名语料库。机构名语料库。公司简称,缩写,品牌词,企业名。可用于中文分词、机构名实体识别。
company corpus dataset dict ner
1276
chatopera / insuranceqa-corpus-zh
:helicopter: 保险行业语料库，聊天机器人
corpus chatbot qasystem natural-language-processing natural-language-understanding machine-learning dataset question-answering insurance insuranceqa-corpus-zh
Language:Python 1043
CLUEbenchmark / CLUECorpus2020
Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料
chinese chinese-corpus datasets pretrain corpus nlp bert roberta albert
982
PlexPt / chatgpt-corpus
ChatGPT 中文语料库对话语料小说语料客服语料用于训练大模型
corpus awesome corpus-data question-answering
913
OYE93 / Chinese-NLP-Corpus
Collections of Chinese NLP corpus
chinese-nlp datasets corpus
Language:Python 911
quanteda / quanteda
An R package for the Quantitative Analysis of Textual Data
r text-analytics natural-language-processing corpus quanteda
Language:R 863
soskek / bookcorpus
Crawl BookCorpus
bookcorpus corpus crawler nlp scraper
Language:Python 845
tensorlayer / seq2seq-chatbot
Chatbot in 200 lines of code using TensorLayer
tensorlayer tensorflow chatbot rnn lstm bot nlp chat corpus python
Language:Python 841
CLUEbenchmark / CLUEPretrainedModels
高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型
pretrained-models chinese bert roberta albert sentence-classification sentence-pairs semantic-similarity sentence-analysis text-classification distillation corpus dataset
Language:Python 815
CBLUEbenchmark / CBLUE
中文医疗信息处理基准CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
benchmark chineseblue corpus dataset chinese biomedical-tasks evaluation acl2022
Language:Python 773
mhbashari / awesome-persian-nlp-ir
Curated List of Persian Natural Language Processing and Information Retrieval Tools and Resources
corpus dependency-parser embeddings information-retrieval language-detection morphological-analysis named-entity-recognition natural-language-processing normalizer part-of-speech-tagger persian-language persian-nlp persian-stemmer shallow-parser spell-check stemmer
762
BLKSerene / Wordless
An Integrated Corpus Tool With Multilingual Support for the Study of Language, Literature, and Translation
corpus corpus-analysis corpus-linguistics corpus-processing corpus-search corpus-statistics corpus-tools dependency-parser lemmatizer linguistics literature stopword tagger tokenizer translation
Language:Python 734
ko-nlp / Korpora
Korean corpus repository
corpus korean-natural-language
Language:Python 718
chatopera / efaqa-corpus-zh
❤️Emotional First Aid Dataset, 心理咨询问答、聊天机器人语料库
natural-language-processing natural-language-understanding corpus psychology
Language:Python 707
nonamestreet / weixin_public_corpus
微信公众号语料库
corpus nlp chinese-nlp natural-language-processing linguistics weixin weixin-data wei-xin corpora yu-liao yu-liao-ku
589
crownpku / Small-Chinese-Corpus
Some useful Chinese corpus datasets 中文语料小数据
chinese-nlp corpus
540
louisowen6 / NLP_bahasa_resources
A Curated List of Dataset and Usable Library Resources for NLP in Bahasa Indonesia
bahasa-indonesia corpus corpus-linguistics dataset indonesian indonesian-language library natural-language-processing nlp nlp-bahasa-resources packages sentiment-analysis sentiment-analysis-dataset
508
guhhhhaa / 4675-scifi
chinese NLP corpus of chinese science fiction,chinese science fiction corpus : About 4675 Chinese science fiction novels 大约有4675本科幻小说，中文科幻小说自然语言处理语料库，中文科幻小说文本语料库，中文科幻小说文本数据库，科幻小说语料
chinese-nlp corpus corpus-data datasets nlp nlp-datasets nlp-machine-learning nlp-resources science-fiction scifi
421
GAIR-NLP / MathPile
[NeurlPS D&B 2024] Generative AI for Math: MathPile
corpus language-model large-language-models math pre-training
Language:Python 417
flairNLP / fundus
A very simple news crawler with a funny name
cc-news commoncrawl corpus crawler news-crawler news-scraping nlp python rss scraper sitemap text-extraction web-corpus web-scraping corpus-tools datasets image-classification image-extraction
Language:Python 401
several27 / FakeNewsCorpus
A dataset of millions of news articles scraped from a curated list of data sources.
artificial-intelligence corpus database dataset fakenews machine-learning natural-language-processing nlp
391
ko-ichi-h / khcoder
KH Coder: for Quantitative Content Analysis or Text Mining
content-analysis text-mining visualization kwic corpus
Language:Perl 315
malaysian-dataset
mesolitica / malaysian-dataset
We gather Malaysian dataset! https://malaysian-dataset.readthedocs.io/
text-mining corpus malaysia malay-dataset bahasa-melayu manglish
Language:Jupyter Notebook 314
strongcourage / fuzzing-corpus
My fuzzing corpus
fuzzing vulnerability testsuite corpus file-format
Language:JavaScript 281
lil-lab / nlvr
Cornell NLVR and NLVR2 are natural language grounding datasets. Each example shows a visual input and a sentence describing it, and is annotated with the truth-value of the sentence.
computer-vision corpus machine-learning natural-language-processing
Language:HTML 262

corpus

brightmart / nlp_chinese_corpus

dariusk / corpora

CLUEbenchmark / CLUEDatasetSearch

CLUEbenchmark / CLUE

wainshine / Chinese-Names-Corpus

endymecy / awesome-deeplearning-resources

lucasjinreal / weibo_terminater

fendouai / Awesome-Chatbot

candlewill / Dialog_Corpus

gunthercox / chatterbot-corpus

NiuTrans / Classical-Modern

wainshine / Company-Names-Corpus

chatopera / insuranceqa-corpus-zh

CLUEbenchmark / CLUECorpus2020

PlexPt / chatgpt-corpus

OYE93 / Chinese-NLP-Corpus

quanteda / quanteda

soskek / bookcorpus

tensorlayer / seq2seq-chatbot

CLUEbenchmark / CLUEPretrainedModels

CBLUEbenchmark / CBLUE

mhbashari / awesome-persian-nlp-ir

BLKSerene / Wordless

ko-nlp / Korpora

chatopera / efaqa-corpus-zh

nonamestreet / weixin_public_corpus

crownpku / Small-Chinese-Corpus

louisowen6 / NLP_bahasa_resources

guhhhhaa / 4675-scifi

GAIR-NLP / MathPile

flairNLP / fundus

several27 / FakeNewsCorpus

ko-ichi-h / khcoder

mesolitica / malaysian-dataset

strongcourage / fuzzing-corpus

lil-lab / nlvr