k4black / uds-2024-coli

UdS 2024: Final project for Computational Linguistics course

POS-Aware Data Augmentation for NLP Tasks

This project investigates the impact of Part-Of-Speech (POS)-aware data augmentation on the performance of the roberta-base model across four Super GLUE tasks. Our experiments showed that POS-aware augmentation techniques outperformed random augmentation methods, and introduced a more stable training process on challenging tasks like WiC and RTE. However, it is not a silver bullet, and the augmentation requires a task-specific parameter tuning to achieve the best performance (or just improvement).

How Install

To run the training script, you need to have Python 3.12 and the required packages installed.

pip install -r requirements.txt

Additionally, you need to fill .env file with your Neptune.ai NEPTUNE_PROJECT and NEPTUNE_API_TOKEN to log the experiments.

Main Libraries used

• transformers for obtaining the checkpoints, training loop and evaluation
• datasets for loading the Super GLUE datasets
• fast-aug - our custom library for random data augmentation - written on rust with python bindings
• neptune for logging the experiments (runs available)

Run

To get all the available options, run:

python main.py --help

For example, to train the roberta-base model on the WiC task with POS-aware substitution augmentation, run:

python main.py --task_name super_glue/wic --model_name roberta-base --augmentation words-pos-sub