swy0601 / DevOps

LDA topic modeling

Contents

1. Introduction
2. Train
3. Test
4. Results
5. Citation
6. Acknowledgements

Usage

Please see code comments for more details

Quick start

Pre-process

1. Download stack overflow dataset from dataset.

2. Put the data file under the data directory

3. Run the following script.

    python pre-process.py

Train

1. (Optional) Run the following script to get perplexity.

    python perplexity.py

2. Run the following script to get LDA model.

    python train_LDA_model.py

Detail Package Version

    numpy==1.21.6
    pandas==1.3.5
    
    pickle==1.1.1
    gensim==3.8.3