Which data points should be considered difficult? Those which are close to the other class, or those which are far away from the center of the class?
The data points in the "tail" are learned later, but the data points at the overlap have low confidence even after training.