MPT models on the Hub not working with `transformers` main

[Younes Belkada]

Hi there!

Currently with transformers main loading MPT models from the Hub fails because it tries to import some private method (such as _expand_mask ) that has been recently removed: huggingface/transformers#27086

The simple loading script below should work

from accelerate import init_empty_weights
from transformers import AutoModelForCausalLM, AutoConfig

model_id = "mosaicml/mpt-7b"
config = AutoConfig.from_pretrained(
    model_id, trust_remote_code=True
)
with init_empty_weights():
    model = AutoModelForCausalLM.from_config(
        config, trust_remote_code=True
    )

[Daniel King]

Thanks for letting us know Younes, will look into this ASAP

[Younes Belkada]

Thanks @dakinggg !

[Daniel King]

@younesbelkada this should be resolved in the foundry code now, and I'm uploading the updated code to the hf hub as we speak.

[Daniel King]

Ok, this should be resolved completely now. Let me know if you see otherwise! Thanks again for the report :)

[Younes Belkada]

Works now like charm! Thanks for the quick fix @dakinggg