How to write the RWKV in autogressive style like RNN
HaiFengZeng opened this issue · comments
instead of running in a teacher-forcing in training stage, can we run the model in free running style?:
for i in range(T):
X_{t}=RNN(X_{t-1},hidden_state)
since teacher-forcing may have an exposure bias problem.
yes you can