Add two feature maps
Aspirinkb opened this issue · comments
Adding two feature maps in ResNet is a common operation, but how to add outputs of two layers with different scale
s and zero_point
s?
Let's say r3 = r1*r2
, and
r1 = s1*(q1-z1)
,
r2 = s2*(q2-z2)
r3 = s3 * (q3-z3)
.
So how to get q3
?
Obviously,
q3 = t1*(q1-z1) + t2*(q2-z2) + z3
, where
t1 = s1/s3
t2 = s2/s3
.
How to get the result of t1*(q1-z1)
?
This is outside of the scope of gemmlowp, which is only a matrix multiplication library.
Here is how addition of quantized arrays is implemented in TFLite - reference (slow portable) code here:
https://github.com/tensorflow/tensorflow/blob/458e0e300bbb8e9c9824a960fef13877983369f4/tensorflow/lite/kernels/internal/reference/reference_ops.h#L566-L597
more optimized NEON code in ../optimized/optimized_ops.h.