Unable to free the intermediate values in the graph.

[Chandradeep Pokhariya]

Hey, I was integrating Fast-Snarf with another project. However, I am not able to free up the gradients even though I am detaching the tensors of broyden cuda. Are you familiar with this error?

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

[Chandradeep Pokhariya]

Wrapping the calculation of the gradients under torch.no_grad() solved the issue

[Abc11c]

Hi @coreqode,

I'm also having the same issue could you share the snippet for the above solution?

If we use @torch.no_grad() then at the loss.backward() step won't it throw an error?

Thanks!