When i use plan in Triton Inference Server get a error
17702513221 opened this issue · comments
Can you help me how to transfet model to Triton Inference Server
Can you tell me when have Triton Inference Server examples
Do you have error on triton side during loading, or in api during inference?
If it's on triton side, I have noticed that SCRFD models won't work for some reason, haven't much time to check it yet.
Triton backend is least tested in my api and is included for testing purposes right now.
during loading ,I have a error use InsightFace-REST's plan's model.
I try to use trtexec change onnx to trt,I have a error
1 is my yolov4 tensorrt's model and insightFace-REST's plan's model,The type is difference.
2 is when i try to use it in triton's error
3 is my try to use trtexec change onnx to tensorrt
4 is my change code
my sdk is tensorrt 21.04 triton is 21.06
Yes, that's exactly the same problem I encountered with SCRFD in Triton server.
InstanceNormalizationPlugin fails to load in Triton.
In python version of TensorRT this can be fixed by calling trt.init_libnvinfer_plugins(None, "") before loading models, but I wasn't able to find corresponding parameter in Triton server.
BTW, TRT and Triton versions should match to guarantee compatibility.
As for your screenshot 4, actual input name is input.1
so you should provide shapes like input.1:1x3x640x640
./trtexec --onnx=/workspace/face/models/onnx/scrfd_10g_gnkps/scrfd_10g_gnkps.onnx --minShapes=input.1:1x3x640x640 --optShapes=input.1:1x3x640x640 --maxShapes=input.1:1x3x640x640 --explicitBatch --saveEngine=/workspace/face/models/onnx/scrfd_10g_gnkps/scrfd_10g_gnkps.trt --fp16
I success,next i want to try detect