Why using dummy data to calculate loss and update the meta model in MAML?
Xiaohan-Chen opened this issue · comments
Thanks for sharing this few-shot learning repository, which help me a lot.
I am curious and confused why using dummy data to calculate loss and update the meta model in line 110 of file few_shot/maml.py
, here I enclose the code:
# Dummy pass in order to create `loss` variable
# Replace dummy gradients with mean task gradients using hooks
logits = model(torch.zeros((k_way, ) + data_shape).to(device, dtype=torch.double))
loss = loss_fn(logits, create_nshot_task_label(k_way, 1).to(device))
loss.backward()
optimiser.step()
As I understand it, the test data should be applied to update the model, using dummy data to train the model seems unreasonable. I will be appreciated it if someone can answer this question.