Adaptation leads to lower precision.
TGISer opened this issue · comments
I changed the dataset（source data count:20000, target data count:2100）
Result:
source only:
mydata set: Average loss: 2.1571, Accuracy: 1311/2100 (62.00%)
domain adaptation:
mydata set: Average loss: 4.5971, Accuracy: 327/2100 (15.00%)
Because GPU has small memory , I set batchsize=16,Is this batchsize problem?
Thank you for your help!
I had the same problem. I used the original setup, and got very low precision.
=== Evaluating classifier for source domain ===
Avg Loss = 0.10151993528631294, Avg Accuracy = 99.120000%
=== Evaluating classifier for encoded target domain ===
source only
Avg Loss = 20191.21803042763, Avg Accuracy = 39.139785%
domain adaption
Avg Loss = 28.358844706886693, Avg Accuracy = 8.118280%
Try to lower the lr of target encoder and reduce the number of epoch.
My setting:
Dataset: Office 31 (A->W)
Network: ResNet-50 for source and target encoder. Same discriminator as in this repo.
lr for source encoder : 1e-3 this doesnot matter
lr for discriminator: 1e-3
lr for target encoder: 1e-5
trainin epoch : 6
Final result: 81% [better than the result in ADDA paper(doesnot know why)]
I have the same issue, thanks for your advice on training whether the result will be good enough.
@TGISer, @wgqtmac, @Tianlin-Gao - seems like you all got it to work. There's no change to main.py since you got it to work. Any pointers on changes you made for it to work?
I'm trying to reproduce the same, but getting an error
Traceback (most recent call last):
File "main.py", line 40, in
src_encoder, src_classifier, src_data_loader)
File "/data/users/vivek/trials/pytorch-adda/core/pretrain.py", line 32, in train_src
for step, (images, labels) in enumerate(data_loader):
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 560, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 560, in
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 95, in getitem
img = self.transform(img)
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 164, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/data/users/vivek/anaconda3/envs/pytorch-adda/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 28, 28] doesn't match the broadcast shape [3, 28, 28]
Try to lower the lr of target encoder and reduce the number of epoch.
My setting:
Dataset: Office 31 (A->W)
Network: ResNet-50 for source and target encoder. Same discriminator as in this repo.
lr for source encoder : 1e-3 this doesnot matter
lr for discriminator: 1e-3
lr for target encoder: 1e-5
trainin epoch : 6
Final result: 81% [better than the result in ADDA paper(doesnot know why)]
Hi, I try to train on the office31 dataset using your parameters, but i got very low precision. Could you share your code with me?Thanks!
@Tianlin-Gao, could you please post out the code snippet for the office 31 dataset, I use the same setting and cannot reproduce your reported result.
I got a similar result to @Tianlin-Gao with office31 dataset. I freezed 1~4th conv blocks of ResNet50 and trained only the 5th block and the classifier, same config (for discriminator: 1e-3, for target encoder: 1e-5) for lr. I resized images to 224 x 224.
Adaptation epochs 20
Final result 83.52%