OnurArdaB / NLP-Toolkit

Some useful NLP tools and their implementations.

NLP-Toolkit

This is an explonatary repository which contains some useful NLP tools and their implementations.

Functions

• Plots

  • WordCloud
  • Heaps Law
  • Zipfs Law

• Language Models

  • MLE
  • KneserNeyInterpolated

• word2vec (gensim)

  • CBOW
  • skip-gram

Setup

1.Download the source code.

virutalenv venv

or

python -m virtualenv venv

2.Lauch the below command in order to download the requirements.

pip install -r requirements.txt

3.Check for the examples folder in order to learn how to use the existing modules.

Sample Datasets

• T_sample5000.pkl
• T_sample20000.pkl

Acknowledgements

Many thanks to Reyyan Yeniterzi for providing us the data set with function definitions.