Error when exporting gpt2

[Paul Garner]

I get this error when trying to convert gpt2

/site-packages/coremltools/converters/mil/mil/input_type.py", line 162, in validate_inputs
    raise ValueError(msg.format(name, var.name, input_type.type_str,
ValueError: Op "137" (op_type: fill) Input shape="136" expects tensor or scalar of dtype from type domain ['int32'] but got tensor[0,fp32]

I first tried:

python -m exporters.coreml --model=gpt2 --framework=pt --feature=causal-lm models/gpt2.mlpackage

Next I tried:

from exporters.coreml import export
from exporters.coreml.models import GPT2CoreMLConfig
from transformers import GPT2LMHeadModel, GPT2Tokenizer

model_ckpt = "gpt2"
base_model = GPT2LMHeadModel.from_pretrained(
    model_ckpt, torchscript=True
)
preprocessor = GPT2Tokenizer.from_pretrained(model_ckpt)

coreml_config = GPT2CoreMLConfig(
    base_model.config, 
    task="causal-lm",
)
mlmodel = export(
    preprocessor, base_model, coreml_config
)

mlmodel.save(f"models/{model_ckpt}.mlpackage")

But they both give the same error

I realise this repo is WIP, but I had seen the list here saying GPT2 model is supported: https://github.com/huggingface/exporters/blob/main/MODELS.md

[Paul Garner]

For more context... I noticed that some of the other LMHeadModel classes have a note:

works OK with coremltools commit 50c5569, breaks with later versions

It looks like that commit was added between coremltools versions 6.0 and 6.1

so I installed compatible package versions:

pip install torch==1.12.1
pip install numpy==1.23.5
pip install coremltools==6.0

also:

coreml_config = GPT2CoreMLConfig(
    base_model.config, 
    task="causal-lm",
    use_past=False,
)

but I still get:

ValueError: Op "137" (op_type: fill) Input shape="136" expects tensor or scalar of dtype from type domain ['int32'] but got tensor[0,fp32]