Giters

karpathy / llm.c

LLM training in simple, raw C/CUDA

Repository from Github https://github.comkarpathy/llm.cRepository from Github https://github.comkarpathy/llm.c

OSError: Memory mapping file failed: Cannot allocate memory

antonkratz opened this issue · comments

commented

When I run python dev/data/fineweb.py --version 10B , I get an error OSError: Memory mapping file failed: Cannot allocate memory. Full log attached.

--
note.txt

I am running this in a HPC context (ABCi cluster in Japan). I think each node has 384 GiB RAM.

commented

I could not get to the root of the matter, but when submitting as a grid engine job instead of running it directly on the node, it seems to work.
Closing this issue.