base

Question

huseinzol05 opened this issue a year ago · comments

done

HUSEIN ZOLKEPLI · Answer 1 · Wed Oct 25 2023 07:58:40 GMT+0800 (China Standard Time)

Train using 2048 context length, 0.15 masking probability and 3.0 mean span length, does not converge, not sure why.

HUSEIN ZOLKEPLI · Answer 2 · Wed Oct 25 2023 07:59:03 GMT+0800 (China Standard Time)

Going to use default settings, 512 context length and same batch size as nanoT5.