Data difficulty thresholding
lalalune opened this issue · comments
The data needs to get harder incrementally. It's pointless to train a fresh model on something a 100 IQ person probably couldn't solve. Starting with ultra simple tasks until loss is down, and then introducing more and more complex tasks into the mix. Eventually taking out the tasks which are under the threshold difficulty of the real problem.