joschu / cgt

Computation Graph Toolkit

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

NTM documentation

EderSantana opened this issue · comments

In the NTM example you mention that

rprev_bhm:          previous vector read from memory. [-1, 1]

but this vector is the result of a linear operation between the memory and the address vector. But the memory is not bounded, how the read vector got bounded? Shouldn't it be [-inf, inf] as well?

Hi Eder, you're right -- I'll update the docs.
(Of course, maybe we should apply a tanh after reading, but that's it's empirical question whether that makes a difference.)