TRT Server Not Starting

[Andrea Rosasco]

Hi, I'm trying to start the server running

JINA_LOG_LEVEL=DEBUG python -m clip_server tensorrt-flow.yml

and I'm getting the following output

⠋ Waiting clip_r... ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 0:05:00
DEBUG  gateway/rep-0@26 completed warmup task in 300.07745909690857s. 
⠧ Waiting clip_r... ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 0:15:26

As you can see it's been waiting for clip_r for 15 minutes. I'm guessing it is compiling the trt engines for clip but with no debug prints it is hard to tell what is going on.

Could anyone confirm that the command works and that it eventually run the server? Thanks in advance

[Andrea Rosasco]

Running nvidia-smi the vRAM looks empty so I don't think it is actually building the engines.