OpenNMT / CTranslate2

Fast inference engine for Transformer models

Home Page:https://opennmt.net/CTranslate2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for Zephyr and other "StableLmForCausalLM" models?

BBC-Esq opened this issue · comments

Any plans to support conversion of ```StableLmForCausalLM" models? I've noticed that they're very good; for example the new Zephyr model here:

https://huggingface.co/stabilityai/stablelm-zephyr-3b

Amazing performance for a 3B model, much better than Phi2 IMHO. Support was added into Transformers in version 4.38.2:

https://github.com/huggingface/transformers/releases/tag/v4.38.0

Here's the link to a description of the model architecture to help:

https://huggingface.co/docs/transformers/v4.38.2/en/model_doc/stablelm