Task formulation for meta-learning enquiry

[Amber Lee]

Hi there,
Firstly, thank you for your work.
I have read your paper and I have a few questions to clarify.
Mainly, how do you "... formulate a set of tasks such that the model
learns to generalize over unseen entities, which are simulated using seen entities." Your paper also mentioned sampling a task from the distribution p(T) but how is p(T) obtained? Is it predefined?

In other words, in the aspect of code, how did you pre-process your data such that it is split into meta-train/meta-valid/meta-test triplets?

self.filtered_triplets, self.meta_train_task_triplets, self.meta_valid_task_triplets, self.meta_test_task_triplets, \
        self.meta_train_task_entity_to_triplets, self.meta_valid_task_entity_to_triplets, self.meta_test_task_entity_to_triplets \
            = utils.load_processed_data('./Dataset/processed_data/{}'.format(args.data))

Other than that, do you mind elaborating on "our meta-learning framework can simulate the unseen entities during meta-training." cause I am still a confused by how your model works.

Thanks!!!

[Jinheon Baek]

Hello,
Thanks for your interest.

I answer your questions one by one as follows:

Regarding the formulation of a set of tasks:

• A task corresponds to a set of unseen entities, with a predefined number of instances N (See Meta-Learning Framework paragraph of page 4 in the paper).
• Then, a set of tasks is formulated with multiple sets for unseen entities, since each task corresponds to a single set of unseen entities.
• To be more precise, while it is possible to optimize the model with a set of tasks, we optimize the model with only one task, which contains a N number of entities, at every batch for simplicity (See Algorithm 1 of page 5 in the paper).

Regarding the task distribution p(T):

• Each task T is obtained by randomly sampling a set of unseen entities.
• Then, the role of the distribution p(T) is to randomly sample the set of unseen entities, with a predefined number of instances N.
• Thus, we sample a task with a distribution p(T) that denotes the random sample for N number of unseen entities.

Regarding the splitting of meta-train/valid/test:

• Each meta-{type} consists of key and value pairs, where a key denotes the entity, and a value denotes the corresponding triplets for the entity.
• Then, we sample a set of entities (task) using the randomly sampled N number of keys at every batch, in our code.
• For the details of the pre-processing steps, please see Appendix A.1 Datasets in the paper.
• Also, the meta-train/valid/test sets do not share the unseen entities, while it might share the seen entities in the self.filtered_triplets variable ("Once the model is trained with the meta-training tasks T_train, we can apply it to unseen meta-test tasks T_test, whose set of entities is disjoint from T_train, as shown in the center of Figure 1").

Regarding the question "our meta-learning framework can simulate the unseen entities during meta-training.":

• Our dataset consists of two parts, namely self.filtered_triplets and self.meta_{type}_task_triplets.
• Then, the variable self.filtered_triplets denotes triplets for the seen entities, and only self.meta_train_task_triplets denotes the triplets for simulated unseen entities during meta-training. With trained models, we finally evaluate the model performances with self.meta_test_task_triplets that is not used on the training.

Thanks, Author.

[Jinheon Baek]

Hello,
Thank you for your interest again, with an insightful question.

I close this issue with the above answer, and please let me know if you have further questions by opening the issue again.

Best wishes,
Author