OpenNMT / CTranslate2

Fast inference engine for Transformer models

Home Page:https://opennmt.net/CTranslate2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for Phi3-Small, Medium, and Vision

BBC-Esq opened this issue · comments

Does ctranslate2 to have plans to support the recently released small, medium, and vision. I've tried running them with transformers (on Windows) and can't get past a Triton and compiler not found kind of errors). I've not had the same issues with phi3-4k so it stands to reason that the phi3 converter in ctranslate2 might need to be modified to handle these models...even though they're in the same family.

ALSO, as far as I know ctranslate2 has never supported any vision models. Phi-3-Vision might be a good place to start or maybe you guys don't plan to support that general category due to time constraints.

Here are the models. Let me know if I can help in any way:

https://huggingface.co/microsoft/Phi-3-small-8k-instruct
https://huggingface.co/microsoft/Phi-3-small-128k-instruct
https://huggingface.co/microsoft/Phi-3-medium-4k-instruct
https://huggingface.co/microsoft/Phi-3-medium-128k-instruct
https://huggingface.co/microsoft/Phi-3-vision-128k-instruct

Here's the error when I tried to convert the small, and I presume it'll be similar for the Medium , so seems to confirm they're not compatible with the current phi3 converter, just FYI:

ValueError: No conversion is registered for the model configuration Phi3SmallConfig (supported configurations are: BartConfig, BertConfig, BloomConfig, CodeGenConfig, DistilBertConfig, FalconConfig, GPT2Config, GPTBigCodeConfig, GPTJConfig, GPTNeoXConfig, GemmaConfig, LlamaConfig, M2M100Config, MBartConfig, MPTConfig, MT5Config, MarianConfig, MistralConfig, MixFormerSequentialConfig, OPTConfig, PegasusConfig, Phi3Config, PhiConfig, RWConfig, T5Config, Wav2Vec2Config, WhisperConfig, XLMRobertaConfig)