shan18 / Perceiver-Resampler-XAttn-Captioning

Generating Captions via Perceiver-Resampler Cross-Attention Networks

shan18/Perceiver-Resampler-XAttn-Captioning Issues

Create a mapping network to change CLIP embeddings in GPTs distribution
Closed a year ago1
Update dataloader to output multilingual sign language videos
Closed a year ago
Look for attention masking for padded frames in resampler
Closed 2 years ago
Remove pad tokens from BLEU calculation
Closed 2 years ago1
Create script for model checkpoint averaging
Closed a year ago
Ignore pad idx from cross entropy loss function
Closed 2 years ago
Add support for multilingual text generation
Closed a year ago
Add option to resume training from a previous checkpoint
Closed 2 years ago
Inspect usage of ff_mult in resampler
Closed 2 years ago2
Add support for training with 8 num_workers
Closed a year ago1
Optimizing the number of parameters in the model
Closed 2 years ago1
Hydra integration
Closed 2 years ago1
Add support for wandb logging
Closed a year ago
Add inference mode to the trainer class
Closed 2 years ago
Add BLEU score metric
Closed 2 years ago
Docstrings for Perceiver Resampler
Closed 2 years ago
Generate masks for the time embeddings in the perceiver to mask out the paddings
Closed 2 years ago
Add the checkpointing functionality to the trainer
Closed 2 years ago
Build the text generator
Closed 2 years ago1
Build the trainer function
Closed 2 years ago
Send max timesteps length for positional encoding from the dataset
Closed 2 years ago1