error line 200, in from_pretrained assert len(keys) == len(sd)

[Sandy4321]

error line 200, in from_pretrained assert len(keys) == len(sd)

len(keys)
581
len(sd)
629

mingpt-master\mingpt-master\mingpt\model.py", line 200, in from_pretrained
assert len(keys) == len(sd)

windows 11 OS
torch version 1.13.1
transformers version 4.36.2

[Sandy4321]

Python 3.10.11

[Sandy4321]

code
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel
from mingpt.model import GPT
from mingpt.utils import set_seed
from mingpt.bpe import BPETokenizer
set_seed(3407)

use_mingpt = True # use minGPT or huggingface/transformers model?
model_type = 'gpt2-xl'
device = 'cuda'
if use_mingpt:
model = GPT.from_pretrained(model_type)
else:
model = GPT2LMHeadModel.from_pretrained(model_type)
model.config.pad_token_id = model.config.eos_token_id # suppress a warning

ship model to device and set to eval mode

model.to(device)
model.eval();

q = 0

[Sandy4321]

mingpt-master\mingpt-master\s_mingpt_jan21.py", line 15, in
model = GPT.from_pretrained(model_type)
File "c:\mingpt-master\mingpt-master\mingpt\model.py", line 200, in from_pretrained
assert len(keys) == len(sd)

builtins.AssertionError

[XiaoyuXie]

I have the same issue.

[Sandy4321]

MAY @karpathy FIX IT ?

[derek]

is it fixed?

[LE VAN DUC]

It was fixed in here #120 and also the reason of the error.