Question about the table

Question

Question about the table

Cassiatora opened this issue 2 years ago · comments

Hi, I want to know if every column indicates leave-one-out cross-validations for the target domain accuracy. And oracle uses the small split of the test domain as the validation set (test-domain validation). Is this contradictory to leave-one-out?

Cassiatora · Answer 1 · Mon Aug 15 2022 16:29:43 GMT+0800 (China Standard Time)

And if the oracle uses the small split of the test domain as the validation set (test-domain validation), I don't understand why the result of SWAD is better than the result of the oracle? Could you help me to solve the confusion?

Junbum Cha · Answer 2 · Wed Aug 17 2022 15:54:56 GMT+0800 (China Standard Time)

You're right. The "oracle" is a kind of cheat setting. In DomainBed, the authors use it to see upper bound on performance by model selection method.

In this table, "oracle" indicates the oracle model selection performance of "ERM". Since SWAD produces only one model, SWAD does not require a model selection method (or, SWAD itself can be viewed as a model selection method).