Please use requirements.txt to install the necessary python packages:
pip3 install -r requirements.txt
Please create following folders:
models: To store trained models in various experiments.
dataset: To store the various data.
runs: To store the tensorboard training summary.
- To get the dataset, please visit https://www.aicrowd.com/challenges/imageclef-2021-vqa-med-vqa and request for the dataset. Because of the agreement issues, we are not sure whether we are allowed to share them or not.
- After downloading all the zip files, including 2020-challenge train/val and 2021 challenge new validation set, extract them inside dataset directory.
- At last, follow the path requirements in
./data_preprocessing.ipynbto combine 2020 challenge's training and validation data single training data. This will create final training data. - Now, visit https://github.com/abachaa/VQA-Med-2021 to get the testing dataset and follow the same instruction as step 2.
Pre-trained model weights can be found at: https://drive.google.com/drive/folders/1K9f-huVsGUSgSVGfaYKL-XeygAm7IDMU?usp=sharing
Please refer the image id wise predicted answers by various models in results folder. One needs to run ./eval.ipynb to get the accuracy and bleu scores. Keep in mind that this part also requires the access to the testing dataset.
Please run folllowing command in exitsting project directory:
tensorboard --logdir=runs
Note: Consider the experiment results with highest run count. For example, xx_run_3 is the final version instead of xx_run_2.
Now, we have dataset prepared then run any .ipynb notebooks with following path specific constraints:
- All the models are stored in
modelsfolder for simplicity. However, keep in mind that model does not get overwirtten. This is ensured while performing the training.