Investigate vectorizing metric computation
aaarrti opened this issue · comments
Description of the problem
- A lot of metrics do computations on a single instance.
- They often could be computed in batches.
Description of a solution
Apply vectorization (or batch computations) for
- Completeness
- Complexity
Investigate which other metrics can be batched/vectorized.
Minimum acceptance criteria
- Metric outputs did not change.
A typical instance-wise metric is currently implemented like this way:
return [self.evaluate_instance(x, a) for x, a in zip(x_batch, a_batch)]we want to get rid of these for-loops.