Wrong implementation of rmse_keras, ... difference_keras for model.fit_generator()
Tudor67 opened this issue · comments
Tudor Buzu commented
Wrong implementation of evaluation metrics for model.fit_generator():
- rmse_keras;
- underestimate_keras;
- overestimate_keras;
- difference_keras.
These functions/methods are from ./utils/evaluation/evaluation.py.
Keras will calculate these metrics per batch and will return their mean.
This is not ok for some evaluation metrics.
I think that one example is enough to get an idea about this:
rmse([pred_batch_1, pred_batch_2, ... pred_batch_k]) != mean (rmse(pred_batch_1), rmse(pred_batch_2), ..., rmse(pred_batch_k))
In other words:
sqrt((a + b + c + d) / 4) != (sqrt((a + b) / 2) + sqrt((c + d) / 2)) / 2.
We need sqrt((a + b + c + d) / 4), but the current implementation returns (sqrt((a + b) / 2) + sqrt((c + d) / 2)) / 2.