LuoweiZhou / VLP

Vision-Language Pre-training for Image Captioning and Question Answering

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Not able to load pre-trained model on CC dataset

chenyez opened this issue · comments

Hi Luowei,

Thank you so much for sharing the pre-trained model and with such detailed instructions.

I followed the instructions to install VLP, I am using python3.6, pytorch 1.1.0. And I also downloaded the checkpoint file pre-trained on CC dataset (cc_g8_lr1e-4_batch512_s0.75_b0.25.tar.gz).

However, when I load the model in the code (line 344 in run_img2txt_dist.py):
model_recover = torch.load(args.model_recover_path)

I received the following error:
Traceback (most recent call last):
File "vlp/run_img2txt_dist.py", line 629, in
main()
File "vlp/run_img2txt_dist.py", line 362, in main
model_recover = torch.load(args.model_recover_path,encoding='latin1')
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/site-packages/torch/serialization.py", line 387, in load
return _load(f, map_location, pickle_module, **pickle_load_args)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/site-packages/torch/serialization.py", line 564, in _load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x1f'.

I tried to load the file(cc_g8_lr1e-4_batch512_s0.75_b0.25.tar.gz) using encoding="latin1" but still the error persists.

Could you please let me know where might be the error? Thank you!

Regards,

Chenye

And I also tried to first
gunzip cc_g8_lr1e-4_batch512_s0.75_b0.25.tar.gz
and load:
gunzip cc_g8_lr1e-4_batch512_s0.75_b0.25.tar.

I received the following error:

Traceback (most recent call last):
File "vlp/run_img2txt_dist.py", line 637, in
main()
File "vlp/run_img2txt_dist.py", line 370, in main
model = torch.load(args.model_recover_path, map_location=lambda storage, loc: storage, pickle_module=pickle)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/site-packages/torch/serialization.py", line 387, in load
return _load(f, map_location, pickle_module, **pickle_load_args)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/site-packages/torch/serialization.py", line 556, in _load
return legacy_load(f)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/site-packages/torch/serialization.py", line 470, in legacy_load
tar.extract('storages', path=tmpdir)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/tarfile.py", line 2041, in extract
tarinfo = self.getmember(member)
File "/home/ubuntu/anaconda3/envs/vlp/lib/python3.6/tarfile.py", line 1752, in getmember
raise KeyError("filename %r not found" % name)
KeyError: "filename 'storages' not found"

Some discussions on similar issues online say that this is because the file is corrupted. Please correct me if I am wrong, looking forward to your reply, thank you!

Stupid question, I need to extract the file first... Sorry.