google / gemmlowp

Low-precision matrix multiplication

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to calculate non-linear function after 8bit quantization ?

youck9224 opened this issue · comments

For 8bit quantization, zero point and scale are applied.
But when in non-linear function layer,
I want to know if I can process the input data without converting it to a real number.
Or is there any way to calibrate?
please answer about my question.

non-linear function : tanh / sigmoid / softmax / exp(x)

gemmlowp provides a fixed-point computation library for this purpose.
It contains implementations of exp, tanh and logistic functions.
See:
https://github.com/google/gemmlowp/blob/master/fixedpoint/fixedpoint.h

For Softmax, an implementation is found in TensorFlow Lite here, built on top of the aforementioned gemmlowp fixedpoint library:
https://github.com/tensorflow/tensorflow/blob/305fec9fddc3bdb5bb574a134b955bf4b07fd795/tensorflow/lite/kernels/internal/reference/softmax.h#L60