Run hash should give a warning for empty/all-zero inputs
rstojnic opened this issue · comments
rstojnic commented
In some cases the first batch might be too small, or simply contain no interesting predictions (eg empty or all zeros).
Run hash calculation should catch this, produce a warning and not try to retrieve the cached results.
Ross Taylor commented
This might be an issue for the frontends (sotabencheval, torchbench) and is task-specific.
I've solved for COCO in sotabencheval by doing the following:
- Check whether all the bboxs in a list of dicts are empty
- If they are all empty, then don't calculate a batch hash (wait until the first addition that has detections)