nearai / program_synthesis

Program Synthesis

Home Page:http://near.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

NAPS dataset: distinguishing 'search' from 'test' IO examples

mtensor opened this issue · comments

Hello,
The NAPS paper says, "Each full solution is accompanied with 2-10 inputs/outputs each split into two groups. The first group can be used in search or can be included into the problem specification
as part of the model input. The second group can be used for the evaluation at the test time."
However in the NAPS dataset, it seems that there is a single list of IO examples with the key 'tests'. How may I distinguish 'search' examples from 'test' examples?

Thanks.

Added split tests pipe in 9e64e06

Use it to deterministically split tests upon reading the dataset.