Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
JackIRose opened this issue · comments
Checklist
- I have searched related issues but cannot get the expected help.
- 2. I have read the FAQ documentation but cannot get the expected help.
- 3. The bug has not been fixed in the latest version.
Describe the bug
使用python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-64x64-800x800.py rtmdet_tiny_8xb32-300e_coco.py rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth det.jpg --work-dir mmdeploy/RTMdet_t_trt --device cuda --dump-info后,能够成功生成engine文件并输出测试图片。
但是当我使用C++加载engine文件进行推理时会报一下错误:
[03/05/2024-13:55:15] [TRT] [E] 1: [pluginV2Runner.cpp::nvinfer1::rt::load::293] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[03/05/2024-13:55:15] [TRT] [E] 4: [runtime.cpp::nvinfer1::Runtime::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
Reproduction
int main(int argc, char** argv)
{
// create a model using the API directly and serialize it to a stream
char* trtModelStream{ nullptr };
size_t size{ 0 };
std::cout << "assssssssss" << std::endl;
std::ifstream file("E:/工厂机器视觉-行为动作识别/mmdeploy-main/RTM-ins/end2end.engine", std::ios::binary);
if (file.good()) {
file.seekg(0, file.end);
size = file.tellg();
file.seekg(0, file.beg);
trtModelStream = new char[size];
assert(trtModelStream);
file.read(trtModelStream, size);
file.close();
}
Logger m_logger;
IRuntime* runtime = createInferRuntime(m_logger);
assert(runtime != nullptr);
std::cout << "1111111111111111111" << std::endl;
// 初始化插件库
bool success = initLibNvInferPlugins(&m_logger, "");
std::cout << "success: " << success << std::endl;
ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream, size, nullptr);
assert(engine != nullptr);
std::cout << "2222222222222222222" << std::endl;
IExecutionContext* context = engine->createExecutionContext();
assert(context != nullptr);
std::cout << "3333333333333333333" << std::endl;
// generate input data
float data[BATCH_SIZE * 3 * IN_H * IN_W];
for (int i = 0; i < BATCH_SIZE * 3 * IN_H * IN_W; i++)
data[i] = 1;
//测试doInferece函数100次,计算平均时间
auto start = std::chrono::high_resolution_clock::now();
// Run inference
float prob[BATCH_SIZE * 3 * IN_H * IN_W / 4];
for (int i = 0; i < 100; i++) {
doInference(*context, data, prob, BATCH_SIZE);
}
std::cout<<prob[0]<<std::endl;
auto end = std::chrono::high_resolution_clock::now();
std::cout << "RTMpose处理一帧图片所需时间" << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count()/100 << "ms" << std::endl;
// Destroy the engine
context->destroy();
engine->destroy();
runtime->destroy();
return 0;
}
Environment
03/05 15:58:48 - mmengine - INFO -
03/05 15:58:48 - mmengine - INFO - **********Environmental information**********
03/05 15:59:02 - mmengine - INFO - sys.platform: win32
03/05 15:59:02 - mmengine - INFO - Python: 3.8.18 (default, Sep 11 2023, 13:39:12) [MSC v.1916 64 bit (AMD64)]
03/05 15:59:02 - mmengine - INFO - CUDA available: True
03/05 15:59:02 - mmengine - INFO - MUSA available: False
03/05 15:59:02 - mmengine - INFO - numpy_random_seed: 2147483648
03/05 15:59:02 - mmengine - INFO - GPU 0: Quadro P2200
03/05 15:59:02 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5
03/05 15:59:02 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.5, V11.5.50
03/05 15:59:02 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30154 版
03/05 15:59:02 - mmengine - INFO - GCC: n/a
03/05 15:59:02 - mmengine - INFO - PyTorch: 1.12.0
03/05 15:59:02 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
- C++ Version: 199711
- MSVC 192829337
- Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 2019
- LAPACK is enabled (usually provided by MKL)
- CPU capability usage: AVX2
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.3.2 (built against CUDA 11.5)
- Magma 2.5.4
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,
03/05 15:59:02 - mmengine - INFO - TorchVision: 0.13.0
03/05 15:59:02 - mmengine - INFO - OpenCV: 4.9.0
03/05 15:59:02 - mmengine - INFO - MMEngine: 0.10.3
03/05 15:59:02 - mmengine - INFO - MMCV: 2.1.0
03/05 15:59:02 - mmengine - INFO - MMCV Compiler: MSVC 192930148
03/05 15:59:02 - mmengine - INFO - MMCV CUDA Compiler: 11.3
03/05 15:59:02 - mmengine - INFO - MMDeploy: 1.3.1+unknown
03/05 15:59:02 - mmengine - INFO -
03/05 15:59:02 - mmengine - INFO - **********Backend information**********
03/05 15:59:03 - mmengine - INFO - tensorrt: 8.4.1.5
03/05 15:59:03 - mmengine - INFO - ONNXRuntime: None
03/05 15:59:03 - mmengine - INFO - ONNXRuntime-gpu: 1.17.1
03/05 15:59:03 - mmengine - INFO - ONNXRuntime custom ops: Available
03/05 15:59:03 - mmengine - INFO - pplnn: None
03/05 15:59:03 - mmengine - INFO - ncnn: None
03/05 15:59:03 - mmengine - INFO - snpe: None
03/05 15:59:03 - mmengine - INFO - openvino: None
03/05 15:59:03 - mmengine - INFO - torchscript: 1.12.0
03/05 15:59:03 - mmengine - INFO - torchscript custom ops: NotAvailable
03/05 15:59:03 - mmengine - INFO - rknn-toolkit: None
03/05 15:59:03 - mmengine - INFO - rknn-toolkit2: None
03/05 15:59:03 - mmengine - INFO - ascend: None
03/05 15:59:04 - mmengine - INFO - coreml: None
03/05 15:59:04 - mmengine - INFO - tvm: None
03/05 15:59:04 - mmengine - INFO - vacc: None
03/05 15:59:04 - mmengine - INFO -
03/05 15:59:04 - mmengine - INFO - **********Codebase information**********
03/05 15:59:04 - mmengine - INFO - mmdet: 3.3.0
03/05 15:59:04 - mmengine - INFO - mmseg: None
03/05 15:59:04 - mmengine - INFO - mmpretrain: None
03/05 15:59:04 - mmengine - INFO - mmocr: None
03/05 15:59:04 - mmengine - INFO - mmagic: None
03/05 15:59:04 - mmengine - INFO - mmdet3d: None
03/05 15:59:04 - mmengine - INFO - mmpose: 1.3.1
03/05 15:59:04 - mmengine - INFO - mmrotate: None
03/05 15:59:04 - mmengine - INFO - mmaction: None
03/05 15:59:04 - mmengine - INFO - mmrazor: None
03/05 15:59:04 - mmengine - INFO - mmyolo: None
Error traceback
04/19/2024-17:31:40] [E] [TRT] 1: [pluginV2Runner.cpp::nvinfer1::rt::load::293] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[04/19/2024-17:31:40] [E] [TRT] 4: [runtime.cpp::nvinfer1::Runtime::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.