Clarification on Pretraining Process and Data Inclusion

[hushuguo]

Hi there,

I'm currently exploring the excellent work you've done, and I have a few questions regarding the pretraining process:

1.After the model has been pretrained, is it fine-tuned using the train.py script?
2.whether train.py is used for both the pretraining and fine-tuning phases, or is it specifically designed for one of these phases?
I appreciate the effort put into this project.

Thank you!

[wetliu]

Thanks for your interest. It is safe to use it as fine-tuning. You can refer https://github.com/hendrycks/outlier-exposure/tree/master to train from scratch. If I remembered correctly, we tried to do that before but the newly trained checkpoint seems not be able to reach the same performance as their baseline.