Plan is to pre-train RoBERTa, ViT & Wav2Vec2 model using the objective described in Data2Vec paper. Connect with me over LinkedIn/Twitter for updates on this project.
- Google Developers Expert program for providing GCP credits
- TPU Cloud Research for providing free TPU resources