pavlosdais / Deep-Learning-NLP

Deep learning for Natural Language Processing

General Elections Classifier

Deep learning models for sentiment analysis classification of greek general election tweets into 3 categories:

• Positive
• Neutral
• Negative

Four different classifiers have been developed on the same datasets using real twitter data:

Softmax Regression

In short, experimented with:

• Data preprocessing
• Different vectorizers (count, BoW, TF-IDF)
• Removal of features with very low/high frequency by adding max-df / min-df parameters
• Hyperparameter optimization using the Optuna framework

Place 36/114 at the course's kaggle competition.

Feed Forward Neural Network

In short, experimented with:

• Data preprocessing
• Different number of layers, neurons per layer & learning rates
• Activation functions (Linear & non-linear)
• Dropout layers
• Different optimizers (Stochastic Gradient Descent, Adam)
• Early stopping
• Hyperparameter optimization using the Optuna framework

Place 1/95 at the course's kaggle competition.

Recurrent Neural Network

In short, experimented with:

• Data preprocessing
• Bidirectional stacked RNN's with LSTM/GRU cells
• Gradient Clipping
• Early stopping
• Skip connections
• Attention to the best model
• Hyperparameter optimization using the Optuna framework

Place 1/84 at the course's kaggle competition.

BERT

In short, experimented with:

• Data preprocessing
• BertModel
• DistilBertModel
• Adding dropout layers
• Hyperparameter optimization using the Optuna framework

Place 6/70 at the course's kaggle competition.

About

The models were created for the Artificial Intelligence II course under prof. Manolis Koubarakis.