GPT-RMT - An experiment to test Reccurent Memory Transformers in GPT models
- More info coming soon
Updates
- May 5th, 2023: Trained a demo model with mixed results. Unfortunately, adding an RMT makes training incredibly slow, which slows down hyper param tuning. Will be hopefully posting results within the next two weeks