Some questions about the result table

[Yixiao Ge]

Hi,

Thanks for your nice work!
I have several questions about the following table

53e0 Best epoch: 4 (out of 7)
===========================================
Recall@N      0001 0002 0003 0004 0005 0010 
===========================================
off-the-shelf 0.80 0.86 0.90 0.92 0.93 0.96 
our trained   0.89 0.94 0.95 0.96 0.97 0.98 
trained/shelf 1.12 1.09 1.06 1.05 1.05 1.02

1. Does the off-the-shelf here include (ImageNet pretrained) VGG16+VLAD+PCA?
2. What does trained/shelf mean?
3. I am now trying to implement your results by PyTorch, but the results seem similar to the raw NetVLAD. So do you have any suggestions?

Thank you very much!

[Liu Liu]

Hi,

Thank you for the questions.
For the questions 1 and 2, I would suggest to read the function pickBestNet in NetVLAD repository since the table is the output of the function pickBestNet.

For question 3, aha, it is a big question to answer since many aspects (mostly a small point) can affect it.

[Yixiao Ge]

Hi @Liumouliu

Thanks for your reply!

I have a small question,
do you use the same model for all the testing in the paper, including the same PCA weights?

[Yixiao Ge]

And another small questions,
do you train the model from the ImageNet pretraining weights or the NetVLAD pretraining weights?

Thanks a lot!

[Liu Liu]

Hi,

1. do you use the same model for all the testing in the paper, including the same PCA weights?

Yes. only one model is used.

2. do you train the model from the ImageNet pretraining weights or the NetVLAD pretraining weights?

I use ImageNet pretraining weights.

Best,

liu

[Yixiao Ge]

Hi, Thanks a lot for your help! I achieve similar results as yours on Pitts250k-test set now, however, I could not achieve your test results on 24/7 Tokyo. Do you have any suggestions about applying the trained model on 24/7 Tokyo in the inference stage? Liu Liu <[e-mail]>于2019年10月22日 周二15:45写道：

…

Hi, 1. do you use the same model for all the testing in the paper, including the same PCA weights? Yes. only one model is used. 1. do you train the model from the ImageNet pretraining weights or the NetVLAD pretraining weights? I use ImageNet pretraining weights. Best, liu — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=ACHSH2DOUD2C7577H3BJ7UTQP2VRTA5CNFSM4JCHX5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB42NAA#issuecomment-544843392>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHSH2ENFHI2VAKIEA72Q7DQP2VRTANCNFSM4JCHX5TA> .

[Liu Liu]

Good!

I assume the following steps have been done:

To do experiments on Tokyo 24/7 dataset, you need to resize query images (Only query, don't resize database images):
ims_= vl_imreadjpeg(thisImageFns, 'numThreads', opts.numThreads, 'Resize', 640);
You also need to convert all .png images to .jpg images to enable vl_imreadjpeg. You can use convertPngtoJPG.m to do that.

Since you have achieved good performance on Pitts250k-test. I think the reason could be the matlab function : vl_imreadjpeg.

Please read vl_imreadjpeg carefully.

For example,

ims_= vl_imreadjpeg(thisImageFns, 'numThreads', opts.numThreads, 'Resize', 640);

resize the short side of images to 640, while keeping the aspect ratio.

I hope the above solution can do some help.

If the above solution does not work. I would suggest you first download my pretrained model, and test it using matconvnet. See whether you can achieve the same results as mine. We shall start from here.

[Yixiao Ge]

Finally, I could achieve similar results as yours on Pitts and Tokyo datasets.
Thanks a lot for your quick replies!

I found that loading the same MatConvNet's pretrained VGG-16 weights and clustering centers is key for improving the final results.
I will look into why training with PyTorch pretrained weights performs worse.