Avoid for-loops, create new elementwise-axis (trax) instead
adtzlr opened this issue · comments
The idea is to replace for-loops in the gradient and hessian calls by an additional dimension within the trailing elementwise-axes (trax). That means in gradient/hessian calls, the shape of a tensor must be modified:
(*shape, *trax)
to (*shape, newaxis, *trax)
with a size of newaxis=size(shape)
(without symmetry).
Each item of the newaxis consists of exactly one dual tensor. The output of a scalar-valued gradient is then simply reshaped
(*shape, newaxis, *trax)
to (*shape, *shape, *trax)
.