adtzlr / tensortrax

Differentiable Tensors based on NumPy Arrays

Home Page:https://tensortrax.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Avoid for-loops, create new elementwise-axis (trax) instead

adtzlr opened this issue · comments

The idea is to replace for-loops in the gradient and hessian calls by an additional dimension within the trailing elementwise-axes (trax). That means in gradient/hessian calls, the shape of a tensor must be modified:

(*shape, *trax) to (*shape, newaxis, *trax) with a size of newaxis=size(shape) (without symmetry).

Each item of the newaxis consists of exactly one dual tensor. The output of a scalar-valued gradient is then simply reshaped

(*shape, newaxis, *trax) to (*shape, *shape, *trax).