Giters

UKPLab / EasyNMT

Easy to use, state-of-the-art Neural Machine Translation for 100+ languages

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can't access other models in docker image

4quen opened this issue · comments

commented

Hi,

I'm sorry for this noobish question/issue and maybe it is easy to resolve (I'm not experienced with docker). I've built a web app which uses easyNMT in the back via the docker images and REST. When translating from romanian to german I noticed that the docker image is only using the opus model which does not provide this language direction. But when executing the "/model_name" request it shows me only "opus" as part of the docker image.

So how can I get the other models? I have 3 docker images of easynmt (one with 7.7gb, one with 6.02 and one with 3.8 gb size) but it seems none of them contains the other models. Am I doing something wrong here?
And also when they are part of the image, is there some kind of auto selection if a language is not available in one of the packages?

I installed the docker images via the "build-docker-hub.sh" file.

Best regards,
André

commented

When you start the docker, you must set the env. variable EASYNMT_MODEL to control which model to load:
https://github.com/UKPLab/EasyNMT/tree/main/docker#environment-variables

So pass to docker that the env. variable EASYNMT_MODEL should be e.g. the M2M model and then the docker will load this specific model

commented

Hi,

ok, so it's not possible to load multiple models in the same container, right?

Best regards,
André

commented

No, that is not possible.

You can start multiple docker containers. The overhead is quite small compared to the model size.

commented

Hi, ok thanks for the quick help and response!

commented

@4quen I ran into the same issue. Instead of docker I went straight to Python and servered it all with fastapi, uvicorn and Nginx reverse proxy. I load the models in a list on startup then select the model I need depending on language pair:

all_models = [
  Model(source_lang="en", target_lang="de", model=EasyNMT(translator=models.AutoModel('Helsinki-NLP/opus-mt-en-de'),device="cpu")),
  Model(source_lang="de", target_lang="en", model=EasyNMT(translator=models.AutoModel('Helsinki-NLP/opus-mt-de-en'),device="cpu"))
]

You need enough GPU/RAM to do this but also has the bonus of giving you control over the language codes and over the Huggingface model names. For example the Portuguese opus-mt model has "tc-big" inserted in the name, or Japanese is not "jp" but "jap". Those can cause the autoloader to fail miserably!