Giters

microsoft / nn-Meter

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how to predict model generated from keras model.save()?

manideep-bandaru opened this issue · comments

commented

Hi,

I have generated a keras model and saved it using keras model.save() function which results in saved_model folder consisting of assests,variable,keras_metadata.pb and saved_model.pb file.

When running the nn-meter predictor command:

nn-meter predict --predictor cortexA76cpu_tflite21 --predictor-version 1.0 --tensorflow saved_model.pb

resulted in the following error:

(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/dwconv-bn-relu.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/add.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/bnrelu.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/relu.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/global-avgpool.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/bn.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/maxpool.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/hswish.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/fc.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/conv-bn-relu.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/split.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/se.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/avgpool.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/concat.pkl
(nn-Meter) load predictor /home/.nn_meter/data/predictor/cortexA76cpu_tflite21/channelshuffle.pkl
(nn-Meter) Start latency prediction ...
Traceback (most recent call last):
File "/home/Desktop/Projects/nn-Meter/py3.9_env/bin/nn-meter", line 33, in
sys.exit(load_entry_point('nn-meter', 'console_scripts', 'nn-meter')())
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/utils/nn_meter_cli/interface.py", line 266, in nn_meter_cli
args.func(args)
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/utils/nn_meter_cli/predictor.py", line 56, in apply_latency_predictor_cli
latency = predictor.predict(model, model_type) # in unit of ms
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/predictor/nn_meter_predictor.py", line 106, in predict
graph = model_file_to_graph(model, model_type, input_shape, apply_nni=apply_nni)
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/ir_converter/utils.py", line 42, in model_file_to_graph
converter = FrozenPbConverter(filename)
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/ir_converter/frozenpb_converter/frozenpb_converter.py", line 15, in init
parser = FrozenPbParser(file_name)
File "/home/Desktop/Projects/nn-Meter/py3.9_env/nn-Meter/nn_meter/ir_converter/frozenpb_converter/frozenpb_parser.py", line 16, in init
graph.ParseFromString(f.read())
google.protobuf.message.DecodeError: Error parsing message with type 'tensorflow.GraphDef'

How can i predict the model generated from saved_model.pb?

I saw in the examples provided there is single .pb file which can be predicted. So do i need to convert the saved_model folder into single .pb file? If so, how to convert saved_model to single .pb file?

I have attached the generated saved_model below for your reference.

Please respond as soon as possible.

Thanks.

mobilenetv1_0.zip

commented

Hello, sorry for the late reply.
Our .pb converter currently only works with tensorflow 1.x, and these type of pb files have been deprecated in tf2. If you want to use it in tensorflow 2.x, you may encounter bugs like the one you mentioned. We suggest using an ONNX model for latency prediction.
Sorry for the inconvenience.

Jiahang

commented

Hello, sorry for the late reply. Our .pb converter currently only works with tensorflow 1.x, and these type of pb files have been deprecated in tf2. If you want to use it in tensorflow 2.x, you may encounter bugs like the one you mentioned. We suggest using an ONNX model for latency prediction. Sorry for the inconvenience.

Jiahang

Thank you for the quick response , we have already moved on by doing what you asked to do.