softmax-output-imagenet-test and softmax-output-smoketest test failed
Danliran opened this issue · comments
Hi NNPACK team,
platform: aarch64 hardware
os: ubuntu 18.04 arm64
softmax-output-imagenet-test and softmax-output-smoketest test case failed.
It seems the result of nnp_softmax_output__reference and nnp_softmax_output is not the same.
[==========] Running 12 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 6 tests from OUT_OF_PLACE
[ RUN ] OUT_OF_PLACE.batch1
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.357721 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch1 (1 ms)
[ RUN ] OUT_OF_PLACE.batch2
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.368963 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch2 (0 ms)
[ RUN ] OUT_OF_PLACE.batch16
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.374419 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch16 (4 ms)
[ RUN ] OUT_OF_PLACE.batch64
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.391025 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch64 (17 ms)
[ RUN ] OUT_OF_PLACE.batch128
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.38515 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch128 (78 ms)
[ RUN ] OUT_OF_PLACE.batch256
/home/taishan/NNPACK/test/testers/softmax.h:128: Failure
Expected: (maxError) < (errorLimit()), actual: 0.381074 vs 1e-05
[ FAILED ] OUT_OF_PLACE.batch256 (102 ms)
[----------] 6 tests from OUT_OF_PLACE (204 ms total)
[----------] 6 tests from IN_PLACE
[ RUN ] IN_PLACE.batch1
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.368789 vs 1e-05
[ FAILED ] IN_PLACE.batch1 (0 ms)
[ RUN ] IN_PLACE.batch2
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.368311 vs 1e-05
[ FAILED ] IN_PLACE.batch2 (0 ms)
[ RUN ] IN_PLACE.batch16
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.374365 vs 1e-05
[ FAILED ] IN_PLACE.batch16 (5 ms)
[ RUN ] IN_PLACE.batch64
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.381611 vs 1e-05
[ FAILED ] IN_PLACE.batch64 (16 ms)
[ RUN ] IN_PLACE.batch128
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.386007 vs 1e-05
[ FAILED ] IN_PLACE.batch128 (93 ms)
[ RUN ] IN_PLACE.batch256
/home/taishan/NNPACK/test/testers/softmax.h:157: Failure
Expected: (maxError) < (errorLimit()), actual: 0.385632 vs 1e-05
[ FAILED ] IN_PLACE.batch256 (98 ms)
[----------] 6 tests from IN_PLACE (212 ms total)
[----------] Global test environment tear-down
[==========] 12 tests from 2 test cases ran. (416 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 12 tests, listed below:
[ FAILED ] OUT_OF_PLACE.batch1
[ FAILED ] OUT_OF_PLACE.batch2
[ FAILED ] OUT_OF_PLACE.batch16
[ FAILED ] OUT_OF_PLACE.batch64
[ FAILED ] OUT_OF_PLACE.batch128
[ FAILED ] OUT_OF_PLACE.batch256
[ FAILED ] IN_PLACE.batch1
[ FAILED ] IN_PLACE.batch2
[ FAILED ] IN_PLACE.batch16
[ FAILED ] IN_PLACE.batch64
[ FAILED ] IN_PLACE.batch128
[ FAILED ] IN_PLACE.batch256
12 FAILED TESTS
If we compile softmax funtion with "-O2/3" option, the test case will fail. I think the root reason is out of order execution, but I don`t know which step should add memory barrier or data and instruction sync function.
It is likely because of -ffast-math: https://github.com/Maratyszcza/NNPACK/blob/master/CMakeLists.txt#L467
yes, you are right. I find it. Can I submit a PR to fix this issue on ARM platform? @Maratyszcza
http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
-ffast-math
Sets the options -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only, -fno-rounding-math, -fno-signaling-nans, -fcx-limited-range and -fexcess-precision=fast.
This option causes the preprocessor macro __FAST_MATH__ to be defined.
This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications.
FIX this issue. PR: #185 @Maratyszcza