用rknpu转换识别模型出错: onnx.onnx_cpp2py_export.checker.ValidationError: Field 'shape' of type is required but missing.

[Joey]

环境

• 【FastDeploy版本】： develop分支，4月27日的2c5fd91a7f321f483140f1a1a7ff899de6f8845e
• 【编译命令】x86标准编译
• 【系统平台】: Linux x64(Ubuntu 18.04)
• 【硬件】： cpu
• 【编译语言】： C++

问题日志及出现问题的操作流程

按照这个文档操作的https://github.com/PaddlePaddle/FastDeploy/blob/develop/examples/vision/ocr/PP-OCR/rockchip/cpp/README.md。
使用上面文档里面链接下载的PP-OCRv3文字识别模型，可以顺利转onnx，转rknn，使用我自己训练过的识别模型可以转onnx，但转rknn就报错退出了：
python ../export.py --config_path ppocrv3_rec.yaml --target_platform rk3588 {'mean': [[127.5, 127.5, 127.5]], 'std': [[127.5, 127.5, 127.5]], 'model_path': './ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx', 'outputs_nodes': None, 'do_quantization': False, 'dataset': None, 'output_folder': './ch_PP-OCRv3_rec_infer'} W __init__: rknn-toolkit2 version: 1.5.0+1fa95b5c E load_onnx: Catch exception when loading onnx model: /home/joey/samplecodes/2023/FastDeploy/examples/vision/ocr/PP-OCR/rockchip/rknpu2_tools/config/ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx! E load_onnx: Traceback (most recent call last): E load_onnx: File "rknn/api/rknn_base.py", line 1382, in rknn.api.rknn_base.RKNNBase.load_onnx E load_onnx: File "rknn/api/rknn_base.py", line 658, in rknn.api.rknn_base.RKNNBase._create_ir_and_inputs_meta E load_onnx: File "rknn/api/ir_graph.py", line 58, in rknn.api.ir_graph.IRGraph.__init__ E load_onnx: File "rknn/api/ir_graph.py", line 503, in rknn.api.ir_graph.IRGraph.rebuild E load_onnx: File "/home/joey/anaconda3/envs/rknn/lib/python3.6/site-packages/onnx/checker.py", line 106, in check_model E load_onnx: C.check_model(protobuf_string) E load_onnx: onnx.onnx_cpp2py_export.checker.ValidationError: Field 'shape' of type is required but missing. W If you can't handle this error, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (Pwd: rknn) Path: RK_NPU_SDK / RK_NPU_SDK_1.X.0 / develop / If the error still exists in the latest version, please collect the corresponding error logs and the model, convert script, and input data that can reproduce the problem, and then submit an issue on: https://redmine.rock-chips.com (Please consult our sales or FAE for the redmine account) Traceback (most recent call last): File "../export.py", line 52, in <module> assert ret == 0, "Load model failed!" AssertionError: Load model failed!

训练代码是：PaddleOCR-release-2.6，训练命令是：

python3 -m paddle.distributed.launch --gpus '0'  tools/train.py -c configs/rec/PP-OCRv3/ch_PP-OCRv3_rec_distillation.yml -o Global.pretrained_model=./pretrain_models/ch_PP-OCRv3_rec_train/best_accuracy

导出pdmodel命令是：

python3 tools/export_model.py -c  configs/rec/PP-OCRv3/ch_PP-OCRv3_rec_distillation.yml -o Global.pretrained_model=./output/rec_ppocr_v3_distillation/best_accuracy  Global.save_inference_dir=./inference/ch_PP-OCRv3_rec_rubber/

训练配置文件：
`Global:
debug: false
use_gpu: true
epoch_num: 900
log_smooth_window: 20
print_batch_step: 10
save_model_dir: ./output/rec_ppocr_v3_distillation
save_epoch_step: 90
eval_batch_step: [0, 2000]
cal_metric_during_train: true
pretrained_model:
checkpoints:
save_inference_dir:
use_visualdl: false
infer_img: doc/imgs_words/ch/word_1.jpg
character_dict_path: ppocr/utils/ppocr_keys_v1.txt
max_text_length: &max_text_length 40
infer_mode: false
use_space_char: true
distributed: true
save_res_path: ./output/rec/predicts_ppocrv3_distillation.txt
use_amp: False

scale_loss: 1024.0

#use_dynamic_loss_scaling: True

Optimizer:
name: Adam
beta1: 0.9
beta2: 0.999
lr:
name: Piecewise
decay_epochs : [700]
#values : [0.0005, 0.00005]
values: [0.00005, 0.00001]
warmup_epoch: 5
regularizer:
name: L2
factor: 3.0e-05

Architecture:
model_type: &model_type "rec"
name: DistillationModel
algorithm: Distillation
Models:
Teacher:
pretrained:
freeze_params: false
return_all_feats: true
model_type: *model_type
algorithm: SVTR
Transform:
Backbone:
name: MobileNetV1Enhance
scale: 0.5
last_conv_stride: [1, 2]
last_pool_type: avg
Head:
name: MultiHead
head_list:
- CTCHead:
Neck:
name: svtr
dims: 64
depth: 2
hidden_dims: 120
use_guide: True
Head:
fc_decay: 0.00001
- SARHead:
enc_dim: 512
max_text_length: *max_text_length
Student:
pretrained:
freeze_params: false
return_all_feats: true
model_type: *model_type
algorithm: SVTR
Transform:
Backbone:
name: MobileNetV1Enhance
scale: 0.5
last_conv_stride: [1, 2]
last_pool_type: avg
Head:
name: MultiHead
head_list:
- CTCHead:
Neck:
name: svtr
dims: 64
depth: 2
hidden_dims: 120
use_guide: True
Head:
fc_decay: 0.00001
- SARHead:
enc_dim: 512
max_text_length: *max_text_length
Loss:
name: CombinedLoss
loss_config_list:

• DistillationDMLLoss:
  weight: 1.0
  act: "softmax"
  use_log: true
  model_name_pairs:
  • ["Student", "Teacher"]
    key: head_out
    multi_head: True
    dis_head: ctc
    name: dml_ctc
• DistillationDMLLoss:
  weight: 0.5
  act: "softmax"
  use_log: true
  model_name_pairs:
  • ["Student", "Teacher"]
    key: head_out
    multi_head: True
    dis_head: sar
    name: dml_sar
• DistillationDistanceLoss:
  weight: 1.0
  mode: "l2"
  model_name_pairs:
  • ["Student", "Teacher"]
    key: backbone_out
• DistillationCTCLoss:
  weight: 1.0
  model_name_list: ["Student", "Teacher"]
  key: head_out
  multi_head: True
• DistillationSARLoss:
  weight: 1.0
  model_name_list: ["Student", "Teacher"]
  key: head_out
  multi_head: True

PostProcess:
name: DistillationCTCLabelDecode
model_name: ["Student", "Teacher"]
key: head_out
multi_head: True

Metric:
name: DistillationMetric
base_metric_name: RecMetric
main_indicator: acc
key: "Student"
ignore_space: False

Train:
dataset:
name: SimpleDataSet
data_dir: /dataset/redecode_txt_recognition/
ext_op_transform_idx: 1
label_file_list:
- /dataset/redecode_txt_recognition/train_label.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- RecConAug:
prob: 0.5
ext_data_num: 2
image_shape: [48, 320, 3]
max_text_length: *max_text_length
- RecAug:
- MultiLabelEncode:
- RecResizeImg:
image_shape: [3, 48, 320]
- KeepKeys:
keep_keys:
- image
- label_ctc
- label_sar
- length
- valid_ratio
loader:
shuffle: true
batch_size_per_card: 32
drop_last: true
num_workers: 4
Eval:
dataset:
name: SimpleDataSet
data_dir: /dataset/redecode_txt_recognition/
label_file_list:
- /dataset/redecode_txt_recognition/valid_label.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- MultiLabelEncode:
- RecResizeImg:
image_shape: [3, 48, 320]
- KeepKeys:
- image
- label_ctc
- label_sar
- length
- valid_ratio
loader:
shuffle: false
drop_last: false
batch_size_per_card: 4
num_workers: 4

`

[Joey]

我比较了转出来的onnx是不一样的，应该pdmodel就有点不一样，是rknn的识别模型需要改架构吗？