Why does this method not work in SUN and aPY dataset？

[Yangzhu Kuang]

I have tried RN in SUN and aPY， but it did not work, so why?
Why does RN can get good performance in AWA and CUB but not SUN and aPY?

[Keerti Harpavat]

Same doubt, also why is the hidden layer size different for each dataset, and how is it chosen?