CMSIS-NN - Overwrite `output` Between Consecutive Unit Tests

[Fabian Peddinghaus]

The CMSIS-NN unit tests often consecutively execute functions in the same unit test case. They make use of the same input values and parameters and are expected to produce the same output. Taking the test_arm_convolve_s8.c as an example. Here, we first test arm_convolve_s8() and then arm_convolve_wrapper_s8 in every test case; using the same input and output variables.

While tinkering around, I came across the (admittedly) rare case that a modified (broken) version of arm_convolve_wrapper_s8() would not modify the output array at all. It would simply do some calculations, but never write its result back to the output array. However, due to the fact, that the output array does not get reset between consecutive calls, the unit tests would still run through without indicating an error (they still had the correct values from the previous function invocation). This can of course potentially apply to all unit test cases where more than one function is called and expected to produce the same output.

A simple and straightforward fix to this issue would be to overwrite the output between function invocations. This can be done by adding a single line act[i] = 0; in the validate() function here (for validate_s16() analogously).

To nitpick some more: In theory, the same can be said for the intermediate buffer ctx.buf. It, or rather its underlying memory, should (ideally) also not be left untouched. While it does get freeed and malloced between consecutive calls, the chances of malloc returning the same memory (buffer) are quite high. Thus, in the (admittedly very rare) case that only a single im2col iteration would be needed to compute the output tensor, a similarly implemented followup function could have a broken im2col implementation and "cheat" its way through the unit tests by simply relying on the successful computation of the im2col buffer from the previous function.

What both these cases come down to is that one should ideally reset all output (and ideally also intermediate) buffers between consecutive unit tests to either zero (or even better random) values. This ensures that functions do not "cheat" and that they have to compute all needed values themselves.

[felix-johnny]

@fabianpedd Thanks for the issue. I can see what you are referring to. Clearing the output buffers between two consecutive calls, yes, it should be cleared for better robustness.
As for im2col a clarification in arm_nnfunctions.h that the caller is responsible for the integrity of it is something that is coming up. Making an example of that in the unit tests is a good place to show that. I'll split this issue into two for better visibility.