[FEATURE] Settings related to common layers.
BoxiangW opened this issue · comments
BoxiangW commented
Describe the feature
Compared to vanilla PyTorch, Titans right now includes many unnecessary codes for example multiple files for MLP.
We could provide common MLP, Attention ... modules for users to avoid redundancy in both our repo and users' codes.
I have attached an simple example of PyTorch code as well.