Error from dynamic graph to static graph, KeyError: 'StructuredToParameterName@@'
kritohyh opened this issue · comments
---------------------------------------------------------------------------KeyError Traceback (most recent call last) in
6 # !pdtools param to_dynamic -s output/darknet53/pretrain/DarkNet53/ -d DarkNet53
7 from paddletools.checkpoints import dynamic2static
----> 8 dynamic2static(param_file="output/darknet53/best_model/model", filename="111")
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddletools/checkpoints.py in dynamic2static(param_file, filename)
99 assert len(os.listdir(filename)) == 0, "dir {} should be empty!".format(filename)
100 logger.info("start to read dynamic params...")
--> 101 static_dict = _read_dynamic_params(param_file)
102 logger.info("found {} parameters. start to save to {}...".format(len(static_dict), filename))
103 for name, data in static_dict.items():
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddletools/checkpoints.py in _read_dynamic_params(param_file)
48 with fluid.dygraph.guard(place):
49 model_state_dict, _ = fluid.load_dygraph(param_file, keep_name_table=True)
---> 50 name_table = model_state_dict.pop("StructuredToParameterName@@")
51 for name, data in model_state_dict.items():
52 state_dict[name_table[name]] = data
KeyError: 'StructuredToParameterName@@'
I used the . pdparams parameter file saved during the paddlex model training process
Well, although the model from paddlex also uses .pdparams as the suffix of the model parameters, it's a static graph, not a dynamic graph.
At present, paddletools doesn't support the conversion of such model parameters. If you really need this feature, I will think about it recently.
Well, although the model from paddlex also uses
.pdparamsas the suffix of the model parameters, it's a static graph, not a dynamic graph.
At present, paddletools doesn't support the conversion of such model parameters. If you really need this feature, I will think about it recently.
Thank you very much for your response!
The source of this requirement is that I want to read the parameters saved by the image classifier of PaddleX, such as DarkNet53, into the PaddleX target detection model like yolov3, but the existing Paddlex does not support it.
On the other hand, the parameter 'pretrain_weights' in the Paddlex function model.train provides the ability to selectively read parameters that fit the model structure when loading the official pre-trained static graph parameters, so I want to convert the parameters.
Doesn't thepretrain_weights parameter provided by PaddleX meet your needs? Have you tried directly using the model path, saved by the image classifier of PaddleX, as the value of pretrain_weights? According to their doc, I think that the pretrain_weights can be successfully loaded, if the parameter names of DarkNet53 in image classification and that in yolov3 are the same.
By the way, if you have any problems with PaddleX, you can create an issue on PaddleX.
Doesn't the
pretrain_weightsparameter provided by PaddleX meet your needs? Have you tried directly using the model path, saved by the image classifier of PaddleX, as the value ofpretrain_weights? According to their doc, I think that the pretrain_weights can be successfully loaded, if the parameter names of DarkNet53 in image classification and that in yolov3 are the same.By the way, if you have any problems with PaddleX, you can create an issue on PaddleX.
Unfortunately, the current Paddlex API can only read the entire set of model parameters by thepretrain_weights parameter, and if the YOLO detection layer does not read the parameters, it will report an error：
Oh, I see. I have modified some code in PaddleTools just now, and it can convert your model now. You can pull the latest code and use python3 setup.py install --user --force to re-install your paddletools. Now, you can use pdparams2static function to convert your model.
from paddletools.checkpoints import pdparams2static
pdparams2static(param_file="output/darknet53/best_model/model", filename="static_model_dir")After the conversion is complete, you can use the path/to/static_model_dir as the value of pretrain_weights for yolov3 detection model.
My God, thank you very much for your generosity! I was so excited and couldn't wait to try it. But in baidu AIstudio nootbook environment, I can't import function pdparams2static . However,static2dynamic is ok.
Is it caused by using the command: pip Install PaddleTools-mast.zip ?
I have reinstalled PaddleTools according to the way you said. Now I successfully converted the parameters trained under the Paddlex classification model into the official standard static graph parameters, and read the parameters to the Detection Model successfully. Thank you very much , it's a powerful and convenient tool!!!