trtexec throws error related to constant_folding(trt v8601)
thunder95 opened this issue · comments
FAILED TensorRT.trtexec [TensorRT v8601]
[optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/model/layers.31/self_attn/Slice_6_output_0[Constant].../lm_head/MatMul]}.)
Tried to convert a llm to trt model, used pytorch 2.2.0 cpu, with the following parameters:
org_dummy_input = make_train_dummy_input(128)
output = torch.onnx.export(model,
org_dummy_input,
"baichuan2.onnx",
verbose=False,
do_constant_folding=False,
opset_version=14,
input_names=['input_ids', 'attention_mask'],
output_names=['lm_head'],
dynamic_axes={"input_ids": {0: "batch_size", 1: "sequence_length"},
"attention_mask": {0: "batch_size", 1: "sequence_length"},
"lm_head": {0: "batch_size"}})
This model is from https://github.com/baichuan-inc/Baichuan2.
And this error is quite similar to this issue: NVIDIA/TensorRT#2124.
Please help me, thanks!