Mamba training loss at 7B parameters
manuel-tran opened this issue · comments
Dear authors,
Very impressive work! I have just finished reading your paper and would be very curious about the training loss curve for a 7B Mamba model as shown in Figure S4 in the paper for Transformer++, Hyena, and Striped Hyena. Do you happen to have it?
Best wishes.
We will include a 7B comparison in the next version of the preprint
Mamba 7B included in preprint v2 now