The codes for paper MAP: Modality-Agnostic Uncertainty-Aware Vision-Language Pre-training Model.
The code is based on pytorch_lightning 1.3.2, torch 1.8.0, cuda 11.1.
The start file is run.py.
The model, dataset, and datamodule files are under map.
The config file is map/config.py. Choose different task config to run the program.
Please find more details of used datasets in ViLT and METER.