Does R-Drop need two models?

[zjh21]

Thank you for your great work!
However, I do have a problem concerning train_regularized_dropout_2D.py, which is an implementation of R-Drop. From what I know about R-Drop, it constrains two sub-models, i.e., the same model with different dropouts, to output the same results. That is, we can just put data x through the same model twice. The two forward propagations have two different sub-networks due to the randomness of nn.DropOut().
In your code implementation, however, there are model1 and model2. I wonder if it is necessary.

[KzRinga]

Hey,bro.I also want to know the answer to this question.In your implementation, what is the performance of using only one model.