Giters

karpathy / llama2.c

Inference Llama 2 in one file of pure C

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Understanding "multiple_of"

akbayt opened this issue · comments

commented

Hi everyone, I am trying to understand the usage of the "multiple_of" parameter. I understand the purpose of the parameter. However, the code is not doing what it is supposed to do. e.g. when I set the dim: 64 and multiple_of: 32, the hidden_layer becomes 192 dimensional. Shouldn't it be 256? Am I missing something?