How to use SWAD in PyTorch?

[ping-Huang]

Dear authors,

How can I integrate this method with torchvision model(e.g. ResNet)?
Do you have the plan to separate the SWAD method like the pytorch SWA method?

Thanks

[Junbum Cha]

Hi,

When you train your own ResNet, you can use SWAD just like we did in trainer.py.
If you know when your model converges and overfits, overfit-aware sampling in SWAD can be skipped by setting the sampling range manually. In this case, you can implement SWAD easily using pytorch SWA, by applying SWA update_parameters in every step (densely).