karpathy / llm.c

LLM training in simple, raw C/CUDA

Repository from Github https://github.comkarpathy/llm.cRepository from Github https://github.comkarpathy/llm.c

Is there any pretrain weights for this project?

theodoruszq opened this issue · comments

I plan to perform style-focused supervised fine-tuning (SFT) on local private data, leveraging pretrained weights as the initialization to speed up convergence and preserve general knowledge.

Is there any pretrain weight trained on T/B level tokens?