NTM documentation
EderSantana opened this issue · comments
In the NTM example you mention that
rprev_bhm: previous vector read from memory. [-1, 1]
but this vector is the result of a linear operation between the memory and the address vector. But the memory is not bounded, how the read vector got bounded? Shouldn't it be [-inf, inf] as well?
Hi Eder, you're right -- I'll update the docs.
(Of course, maybe we should apply a tanh after reading, but that's it's empirical question whether that makes a difference.)