Get wrong answer when using function in file `marian/src/tensors/cpu/fbgemm/packed_gemm.cpp`

[umiswing]

Hello! I'm trying to refer to marian to optimize the matrix multiplication on cpu in my own nmt system.

I modified the function in marian/src/tensors/cpu/fbgemm/packed_gemm.cpp to fit my project‘s interface. But most of the code has not changed.

According to my own tests, it works well when the values in source matrix are all positive or negative. However, when the source matrix has both positive and negative values, the output matrix is quite different from the correct result.

I want to know does the same problem appear in your test? Or it is just caused by the bug in my project?

I will be very grateful to your help.

[Young Jin Kim]

Hi. In marian, we have both positive and negative values in both A and B matrices without a problem.

[Kenneth Heafield]

Though you do need to shift one because the underlying operation is unsigned * signed.

[umiswing]

Though you do need to shift one because the underlying operation is unsigned * signed.

Hi! Do you mean perform a shift operation on the source matrix to convert all the values to positive values? I try this and use the fbgemmPacked8Pack() and fbgemmPackedGemm() in marian to perform the matrix multiplication. Then I perform another shift operation to shift the output matrix back. But the result is still quite different from the correct result.

I am sure I shifted the matrix in a correctly, because I tested it using my own MatMul code and it works well.

By the way, I just found that my machine does not support avx512, it only supports avx2. Does this lead to the problem appear in my project?

Thanks to your reply and I'm very grateful to your help.

[umiswing]

I haven't figure out how marian prepack the weight matrix. But I finally fix the bug in my project by replacing the repackedBN processing in the function fbgemmPacked8Gemm() with what marian do in function fbgemmPacked8Pack(). Thank you for your help.