Giters

Solacex / Domain-Consensus-Clustering

[CVPR2021] Domain Consensus Clustering for Universal Domain Adaptation

Home Page:https://openaccess.thecvf.com/content/CVPR2021/papers/Li_Domain_Consensus_Clustering_for_Universal_Domain_Adaptation_CVPR_2021_paper.pdf

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Huge gap between result and paper, CSDA

gavinatthu opened this issue · comments

commented

Hi Team,

Thanks for sharing the great work!

I've noticed in offcehome_run.py, CSDA is not provided.

Domain-Consensus-Clustering/offcehome_run.py

Line 28 in 5a626e1

if config.setting=='uda':

After I added:
elif config.setting=='cda': config.cls_share = 65 config.cls_src = 0 config.cls_total = 65

I noticed there is a huge gap between the result and paper you reported in support material Table C.
here is the result I've got:
Ar->Pr[best]: 0.6168955022899004 0.6168955022899004 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Ar->Pr[last]: 0.3347919933497906 0.3347919933497906 [H-Score]: 0.0 0.0 0.0
Ar->Cl[best]: 0.4183573762957866 0.4183573762957866 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Ar->Cl[last]: 0.3801841050529709 0.3801841050529709 [H-Score]: 0.0 0.0 0.0
Ar->Re[best]: 0.6801177620887756 0.6801177620887756 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Ar->Re[last]: 0.6801177620887756 0.6801177620887756 [H-Score]: 0.0 0.0 0.0
Pr->Ar[best]: 0.4206200426587692 0.4206200426587692 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Pr->Ar[last]: 0.3828089099090833 0.3828089099090833 [H-Score]: 0.0 0.0 0.0
Pr->Cl[best]: 0.3435283417598559 0.3435283417598559 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Pr->Cl[last]: 0.29526575485674234 0.29526575485674234 [H-Score]: 0.0 0.0 0.0
Pr->Re[best]: 0.6622800114636238 0.6622800114636238 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Pr->Re[last]: 0.6590736272243353 0.6590736272243353 [H-Score]: 0.0 0.0 0.0
Cl->Ar[best]: 0.44268233615618485 0.44268233615618485 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Cl->Ar[last]: 0.32432382525159764 0.32432382525159764 [H-Score]: 0.0 0.0 0.0
Cl->Pr[best]: 0.5993400445924355 0.5993400445924355 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Cl->Pr[last]: 0.5993400445924355 0.5993400445924355 [H-Score]: 0.0 0.0 0.0
Cl->Re[best]: 0.5868140732009823 0.5868140732009823 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Cl->Re[last]: 0.5521672795311763 0.5521672795311763 [H-Score]: 0.0 0.0 0.0
Re->Ar[best]: 0.5438899568067147 0.5438899568067147 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Re->Ar[last]: 0.48911217634494486 0.48911217634494486 [H-Score]: 0.0 0.0 0.0
Re->Pr[best]: 0.7569770986070999 0.7569770986070999 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Re->Pr[last]: 0.7569770986070999 0.7569770986070999 [H-Score]: 0.0 0.0 0.0
Re->Cl[best]: 0.44347345402034427 0.44347345402034427 [H-Score]: 0.0 0.0 0.0 0.0 0.0
Re->Cl[last]: 0.44347345402034427 0.44347345402034427 [H-Score]: 0.0 0.0 0.0

And here is result in the SM you've reported:
image

Except for this CSDA experiments, the other experiments go well, though a little unstable.

commented

And in SM, you've noticed table C twice as OSDA and CSDA. Could you please confirm which set is table C?
image

commented

And in SM, you've noticed table C twice as OSDA and CSDA. Could you please confirm which set is table C? image

Hi,
I also don't reproduce in the other setting. Do you reproduce the UDA setting? The gap is large.