A small GPT model that generates criminal stories.
The repository provides a base code for pre-training the model based on the novel The Murder of Roger Ackroyd by Agatha Christie.
Currently achieves a cross-entropy loss of 1.8 for training and validation splits.
Bottlenecked by hardware.