Clarification on running the service on just cpu
goku2130 opened this issue · comments
Hi,
Thank you for this amazing API.
I am trying to run the service on a VM that does not have gpu. Based on the readme, I used the 0.5 version and removed --gpu settings in docker run as well as changed the device to 'cpu' in fapp.py. While the container starts running without issues, I cannot see the documentation on local host:18081
Am I missing any other changes? I am a bit new to docker. I do not know if this port is mapped correctly or not.
Once again, thank you for your help.
Gokul
Hi! Try running container in interactive mode, by removing -d flag in docker run command to see logs.
Also are you using deploy.sh or deploy_trt.sh to start container?
Hi SthPhoenix,
It is now working. Additionally, apart from the changes mentiioned before, you should also add one more change: ensure that import mxnet instead of import mxnet-cu... I totally missed this change. Thank you for your inputs. I am sure it would be a great idea to have this feature in-built as sometimes, gpu can be expensive and unavailable.
Just a final query, is it possible to override the default arcface model with something that is custom trained? I have a trained model. I replaced the default model outside the container in 'model' folder to have the custom model. I reran the deploy script. This time, it shows 'Downloaded file has different hash. Please try again.' I investigated and found out that the code tries to load the default arcface model by calling model_store() of insightface. However, the api calls the prepare() method with arcface model parameter. Am I missing something here? I would love to read some documents if they exist. Anyway, I liked it the API very much
Regards,
Gokul
Actually current version might be run on CPU with ONNX backend instead of TensorRT, though it require some bug fix, I'll commit it later.
Default model could be overriden in current latest version, but for now you should either add custom model to configs and add appropriate wrapper to model zoo, either use one of non standard models, such as subcenter or glint versions, which currently needs manual download, so you can easily replace them with your custom model.
I have added support for CPU-only inference to master branch, you can now run REST API without using any GPU with deploy_cpu.sh script.
Also added corresponding Dockerfile_cpu which builds 1.5Gb sized image, instead of 8.3Gb TensorRT version.
Hi SthPhoenix,
Thank you for the update. I have managed to create a custom wrapper for ArcFace to hold a pretrained model beyond the ones currently supported.
Regards,
Gokul