zyuanlim / singlish-manglish-nlp

Resources for Singlish and Manglish NLP.

Singlish Manglish NLP Resources

Getting started

1. Install anaconda or miniconda
2. Install the requirements conda env create -f environment.yml

Masked Language Modelling

1. SingBERT models mask-filling- BERT base and large models with pre-training finetuned on singlish corpus. Its efficacy of masked language modelling (filling up blanks or masks between words in a sentence) is demonstrated here.

Text Summarization

1. SingBERT models extractive summarization demonstrated here.

Text Classification

1. Classification of the speaker/user of SMS messages from the NUS SMS Corpus, with the processed json data obtained from Kaggle demonstrated here.