benpeloquin7 / diachronic_entropy

Diachronic language entropy as measure of language change.

Diachronic measures of language entropy

Current datasets

Historical corpora

• Coca - https://corpus.byu.edu/coha/
• Coha
• http://martinweisser.org/corpora_site/historical_corpora.html

Modern (diachronic)

• arxiv - https://www.kaggle.com/neelshah18/arxivdataset/kernels
• IRC - https://wiki.ubuntu.com/IRC/ChannelList
• Reddit - https://archive.org/download/2015_reddit_comments_corpus/reddit_data/

Other resources

• Jurafksy's language change course - * https://docs.google.com/spreadsheets/d/1Q_1ib4gqbHgHeFJPcR9JfwrSzkqMDX2TQr0mmbzPi7I/edit#gid=0

---