Integration of Tensorflow Lite and CMSIS-NN

[Hemanth Sabbella]

We are stuck at a problem that requires the weights and biases to be represented in Qm.n formats from a trained TensorFlow lite model. The trained CNN model on Tensorflow lite or Brevitas provides int8 weights, biases, and output activations. The model also specifies the scaling value and zero point, which it uses to convert the float32 weights to int8 weights during the computation internally. A screenshot of the trained weights from one convolution layer is attached [here.]

CMSIS-NN requires the int8 weights to be represented in Qm.n format. It is essential to know the fixed-point representation(Qm.n) since the CMSIS-NN API calls require us to left-shift the biases and right-shift the output activations for correct computation. This task is performed by the CMSIS-API when we provide the exact left shift and right shift values in the header file. A screenshot of the header file and the API call is attached.

Header File snapshot :

API Call Snapshot :

The problem is we do not know how to convert the scaling factor and zero-point values to a Qm.n format. We wondered if there is a way to know the Qm.n formats of the weights, biases, and output activations from Tensorflow lite. CMSIS-NN help guide provided by ARM mentions specific scripts that directly do these conversions like code_gen.py, but unfortunately, these help scripts are no longer available on the Github page (https://github.com/ARM-software/ML-examples/blob/master/cmsisnn-cifar10/code_gen.py). It would be great if you can help.