Is CUDA necessary to run CPU container?

[Felix]

I am met with this error when I am running deploy_cpu.sh from the master branch.
Is it not possible to run the CPU container without having CUDA?

Preparing models...
Traceback (most recent call last):
  File "prepare_models.py", line 5, in <module>
    from modules.model_zoo.getter import prepare_backend
  File "/app/modules/model_zoo/getter.py", line 20, in <module>
    from .exec_backends import triton_backend as triton_backend
  File "/app/modules/model_zoo/exec_backends/triton_backend.py", line 13, in <module>
    import tritonclient.utils.cuda_shared_memory as cudashm
  File "/usr/local/lib/python3.8/site-packages/tritonclient/utils/cuda_shared_memory/__init__.py", line 49, in <module>
    _ccudashm = cdll.LoadLibrary(_ccudashm_path)
  File "/usr/local/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcudart.so.11.0: cannot open shared object file: No such file or directory
Starting InsightFace-REST using 1 workers.
Traceback (most recent call last):
  File "/usr/local/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/main.py", line 425, in main
    run(app, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/main.py", line 447, in run
    server.run()
  File "/usr/local/lib/python3.8/site-packages/uvicorn/server.py", line 68, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/uvicorn/server.py", line 76, in serve
    config.load()
  File "/usr/local/lib/python3.8/site-packages/uvicorn/config.py", line 448, in load
    self.loaded_app = import_from_string(self.app)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/./app.py", line 19, in <module>
    from modules.processing import Processing
  File "/app/./modules/processing.py", line 14, in <module>
    from .face_model import FaceAnalysis, Face
  File "/app/./modules/face_model.py", line 11, in <module>
    from modules.model_zoo.getter import get_model
  File "/app/./modules/model_zoo/getter.py", line 20, in <module>
    from .exec_backends import triton_backend as triton_backend
  File "/app/./modules/model_zoo/exec_backends/triton_backend.py", line 13, in <module>
    import tritonclient.utils.cuda_shared_memory as cudashm
  File "/usr/local/lib/python3.8/site-packages/tritonclient/utils/cuda_shared_memory/__init__.py", line 49, in <module>
    _ccudashm = cdll.LoadLibrary(_ccudashm_path)
  File "/usr/local/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcudart.so.11.0: cannot open shared object file: No such file or directory

[SthPhoenix]

CUDA isn't required for CPU version.
I'll fix it shortly, thanks!

[Felix]

Cool, thank you 👍🏼
In that case, I just noticed that the pip package httpx is also missing.
You might wan't to add this to the Dockerfile as well.

[SthPhoenix]

Cool, thank you 👍🏼 In that case, I just noticed that the pip package httpx is also missing. You might wan't to add this to the Dockerfile as well.

Thanks! I haven't checked CPU version for a while.

[SthPhoenix]

Should be working now

[Felix]

Should be working now
Works like a charm 🥳
Thank you so much!