autograd.numpy.round() gives always 0 as gradient
Fabian188 opened this issue · comments
def foo(x):
return autograd.numpy.round(x**2,decimals=12)
grad_foo = autograd.grad(foo)
print(grad_foo(2.)) # gives 0.0 but should be 4.0
Maybe I'm missing something, but why do you think it should give a non-zero value? The function is flat almost everywhere.
It's a quadratic function, the gradient is linear (2*x)
Hm, I originally missed the decimal=12 part, and was going to agree with you, but I'm not sure now. Why do you need to round to the 12th decimal point? Even with this condition, the function is still flat AE. I'm guessing the decimal=12 is to test what happens in the limit, as decimal -> inf => round should like linear?
The round() was in a existing code I applied to autograd. Once I identified round() as the issue I got rid. The background was an expression to find the angle between two vectors by cos_alpha = v1.dot(v2) / (norm(v1) * norm(v2)) which numerically exceed slightly [-1,1] is special cases and then arccos(cos_alpha) failed. Clearly rounding was not the best way to fix it (but this was the code).
Now I see what you mean with flat - round(f) is a piecewise constant function an yes, having the gradient 0 is the correct answer. Thanks for pointing it out - I missed it, as finite differences did what I wanted.
So it's not a bug but a correct implementation :) I'll close the issue.
Was not a bug
For constraining a value to be between [-1, 1], take a look at np.clip!