A Suspected Bug in `categorical_crossentropy`

[Shuyan Liao]

I found that the the calculation result of categorical_crossentropy loss function is different from other deep learning libraries, such as tensorflow.

repo code:
theano 1.0.4

import numpy as np
from theano import tensor as T
y_true = np.array([0., 1., 0.], dtype=np.float32)
y_pred = np.array([0.9999999, 0.9999999, 0.0000001], dtype=np.float32)
res = T.nnet.categorical_crossentropy(y_pred, y_true)
print("loss = ", res.eval())
# loss =  1.192093e-07

tensorflow 2.0.0

import numpy as np
import keras
y_true = np.array([0., 1., 0.], dtype=np.float32)
y_pred = np.array([0.9999999, 0.9999999, 0.0000001], dtype=np.float32)
res = keras.losses.categorical_crossentropy(y_true, y_pred)
print("loss = ", res.numpy())
# loss =  0.69314724