MiniXC / masked-prosody-modeling

A masked prosody model.

Machine Leraning Template

Template for my machine learning projects.

It's quite opinionated in the following (and probably more) ways:

• uses accelerate
• splits up config, model and scripts
• assumes one is always using a huggingface dataset (this can also be done using load_dataset('some_dataset.py'))
• uses collators as the main way to process data that hasn't been preprocessed by the dataset
• uses separate configs for training (everything not shipped with the model), model and collator

Architecture

The following updates automatically every time scripts/train.py is run.

Click to expand

First Batch

The following updates automatically every time scripts/train.py is run.

Click to expand

Random Masking Command

accelerate launch scripts/train.py configs/default.yml --bin_size 128 --mask_length 1 --mask_length_max 128 --run_name "bin128_mask_random" --wandb_mode=offline