google-research / FLAN

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How many samples were used to train Flan-T5?

rohan-mehta opened this issue · comments

Hey all, possibly silly question. I see that the huggingface collection has many millions of samples, and the google blog post suggests that the collection has 15M samples: https://ai.googleblog.com/2023/02/the-flan-collection-advancing-open.html

On the other hand mixtures.py suggests that ~350K samples is the default maximum: https://github.com/google-research/FLAN/blob/main/flan/v2/mixtures.py#L27

How many samples were actually used to fine tune T5 and produce Flan-T5?

Thanks!