Do you consider adding rwkv
chaodreaming opened this issue · comments
Do you consider adding rwkv
no, i don't really believe in that approach anymore
lucidrains / x-transformers
A concise but complete full-attention transformer with a set of promising experimental features from various papers