karpathy / llm.c

LLM training in simple, raw C/CUDA

Repository from Github https://github.comkarpathy/llm.c

karpathy/llm.c Issues

Is there any pretrain weights for this project?
Updated a day ago
sel4 + llm.c > path to putting these llms in any mission critical system
Closed a year ago1
Why do I encounter a "Bad magic model file" error when running train_gpt2fp32cu?
Updated 2 months ago
[Feature Req]: Any plan on parallel training like deepseeed(zero3, offload nvme) and megatron(tensor parallem, layer pileline)?
Updated 2 months ago1
Error nvcc
Updated 2 months ago1
Some things that might be optimizations
Updated 4 months ago
PMLL.c/.h/.CUDA GPU TWINS
Updated 5 months ago1
The PMLL
Updated 5 months ago2
The persistent memory logic loop by Dr. Edwards
Updated 5 months ago
cu compile fail
Updated 6 months ago
MPI run with upto 3xH100 GPU works but starts giving an "illegal memory access" error when run using "mpirun -np 4 ./train_gpt2cu"
Updated 6 months ago1
link issue
Closed 7 months ago1
Runs under Windows
Updated 7 months ago1
Loss computation is wrong
Closed 7 months ago2
LLM.c in google colab
Closed a year ago2
how it's can support deepseek r1, Who is interested
Closed 8 months ago
Question concerning `float4`
Updated 8 months ago1
wrong MFU with blackwell
Closed 9 months ago1
[cudnn_frontend] Error: No execution plans support the graph.
Updated 9 months ago5
Can't train in FP16 on Turing
Updated 9 months ago2
miscalculating of head index and series index
Closed 9 months ago
Kernel out-of-bounds access in normalization_kernel
Updated 9 months ago
Dataloader shuffle_rng logic bug under multi-gpu settings?
Updated 10 months ago
Makefile incorrectly finds that `nccl` is installed for Linux systems with `libvncclclient`
Updated a year ago1
Token out of vocabulary at train_gpt2.cu:675
Updated a year ago1
hellaswag.py - Github Access to this site has been restricted.
Closed a year ago1
MPI run with 8 GPU fails
Updated a year ago2
Is there any way to make customized dataset?
Closed a year ago
Online Softmax is wrong
Updated a year ago
Problem when debugging cuda kernel functions.
Closed a year ago3
Will this repo update new documentation later?
Updated a year ago
llm.c for inference
Updated a year ago2
Error no instance of overloaded function "..." matches the argument list
Closed a year ago1
Suggestion: Test more Activation Functions
Updated a year ago
MPI run error
Updated a year ago
TypeError: normal_() got an unexpected keyword argument 'generator'
Updated a year ago1
Pretraining (with CPUs)
Updated a year ago4
Suggestion: Use smollm corpus
Updated a year ago3
Windows issue with Cuda Toolkit 12.5 and latest MSVC compiler 17.10
Closed a year ago2
Different batch_size results in different evaluation loss.
Updated a year ago
Larger Tokenizers
Updated a year ago
Getting "Floating point exception (core dumped)" Error
Updated a year ago4
image-gpt
Updated a year ago1
Is Multi-GPU config enabled even when I'm using one GPU?
Closed a year ago1
Specify torch version number in requirements.txt ?
Updated a year ago2
is max_seq_len configurable or hardcoded parameter?
Updated a year ago2
OSError: Memory mapping file failed: Cannot allocate memory
Closed a year ago1
Running `quick start on CPU` on Macbook Pro M2
Closed a year ago7
apparent compatibility issues with earlier c++ versions after recent pushes
Closed a year ago3
I can not understand the `cublasGemmStridedBatchedEx` call in the `attention_forward`
Closed a year ago