Giters

jaydeepthik / Nano-GPT

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

gptpytorch-implementationmultihead-attentionpytorchtransformers

Nano-GPT : Decoder only Transformer

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

Features

  1. Multi-headed self attention
  2. Layer Norm layers
  3. Skip connections
  4. Feed Forward layer

About

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

gptpytorch-implementationmultihead-attentionpytorchtransformers

Languages

Language:Jupyter Notebook 59.8%Language:Python 40.2%