theano.gradient.verify_grad lack of checking the validity of inputs

[Shuyan Liao]

problem:
theano.gradient.verify_grad doesnt check the validity of inputs. In following snippet code, input_1 and input_alpha should have same shape. When I input them into theano.tensor.nnet.relu, this API throws error. While in theano.gradient.verify_grad, it outputs None without error.

repo code:

import theano
import theano.tensor as T
import numpy as np
def custom_activation(x, alpha):
    return theano.tensor.nnet.relu(x, alpha)

x1 = T.tensor3('x1')
alpha = T.tensor4('alpha')

output = custom_activation(x1, alpha)
loss = T.sum(output ** 2)
grad_x1, grad_alpha = T.grad(loss, [x1, alpha])
input_1 = np.random.random((1, 2, 3)).astype('float32')
input_alpha = np.random.random((1, 2, 1, 1)).astype('float32')

rng = np.random.RandomState(123)
print(theano.gradient.verify_grad(custom_activation, pt=[input_1, input_alpha], rng=rng))

[Shuyan Liao]

Same problem can be found in theano.tensor.nnet.nnet.binary_crossentropy.
repo code:

import theano
import theano.tensor as T
import numpy as np
def custom_activation(output, target):
    return theano.tensor.nnet.nnet.binary_crossentropy(output, target)
output = T.tensor4('output')
target = T.tensor4('target')
output = custom_activation(output, target)
loss = T.sum(output ** 2)
grad_inputs, grad_gamma = T.grad(loss, [output, target])
output_data = np.random.random((1, 1, 1, 1)).astype('float32')
target_data = np.random.random((1, 2, 3, 1)).astype('float32')
rng = np.random.RandomState(123)
print(theano.gradient.verify_grad(custom_activation, pt=[output_data, target_data], rng=rng))