PRBonn / TARL

Hi,

when I finetune your TARL pre-trained model on downsteam tasks, I notice that there are two models saved, model_q (online updated model) and model_k (momentum updated model). Which one do you load for down-stream tasks?

Hi! We load model_q you can check how we save it in this function:

TARL/tarl/models/models.py

Line 52 in 7f883d6

def save_backbone(self):

Thank you for your reply.

The code you quote is only used once at the begining of training. After that, the ckpts are saved by

TARL/tarl/tarl_train.py

Line 47 in 7f883d6

checkpoint_saver = ModelCheckpoint(every_n_epochs=10,

I think that is the reason why there are both model_q and model_k in the ckpts.

Which model do you load for downstream tasks? model_q or model_k?