karpathy / minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

where did the self.bias get defined in the casual attention class

nebyu08 opened this issue 2 months ago · comments

nebiyu youhannes commented 2 months ago

Chase Lambert commented 13 days ago

It's here: https://github.com/karpathy/minGPT/blob/master/mingpt/model.py#L47