gradient checks do not match

[Vu Tran]

I checked the gradients you derived against the numerical gradients, and your implementation does not match. It looks like the error is in two places:

1. In calculate_loss, you average the total loss (including the regularization term) over the data batch. The correct implementation should average only the log loss, but not the regularization term.

2. In build_model, the gradients (dW1, dW2, db1, db2) during backprop should be averaged over the data batch. Again, the correct implementation should not include the regularization terms in the average over the data batch.

[Uri Peled]

Do you have or know a better implementation?
Can you explain or show to me how you checked it?

[Vu Tran]

@uripeled2 I have a method for gradient checking in my implementation here: https://github.com/vuptran/introduction-to-neural-networks