About gradients clipping in Tape
Leon-LXA opened this issue · comments
Hi,
I am currently doing the gradients clipping in Tape.py. At first I tried to use warp.clamp(), but it can only be used for a scalar.
Gradients in tape are in the form of array so I wanna use a for loop to clamp each element of the array. But it shows that 'RuntimeError: Item indexing is not supported on wp.array objects'.
I wanna know if there's any other method to do it, thx :D
Best regards,
Lei
Hi Lei, you can call wp.clamp()
for each array element in a @wp.kernel
function. Typically you can add that operation to the end of the kernel that computes the array values.
Hi Lei, you can call
wp.clamp()
for each array element in a@wp.kernel
function. Typically you can add that operation to the end of the kernel that computes the array values.
I am trying to modify tape.py. However in get_adjoint function, it seems like vector a would have different types and it is unable for me to use clamp for each element in the array. Could you please show me a simple modification example in tape.py? Thanks a lot!