clarify mode
barak opened this issue · comments
Based on a quick read, this appears to implement "forward accumulation mode automatic differentiation with stacked tangents" sometimes called "tangent vector mode" in the AD literature, since a vector of n tangents is attached to the primal values. (So calculation of the gradient of an n-input function has a factor of n overhead, in both space and time, for carrying around n stacked tangent values.) It would be good to clarify this in the documentation.
I updated the README. Thanks.