pwegrzyn / llm-from-scratch

Repository from Github https://github.compwegrzyn/llm-from-scratchRepository from Github https://github.compwegrzyn/llm-from-scratch

llm-from-scratch

The goal of this project is to write a simple transformer from scratch.

Stretch goals:

  • Own tokenizer
  • GPU support
  • Instruction fine-tuning
  • RLHF
  • LoRA
  • Quantization
  • Text+Vision

About