coremark was impacted by some extra print

Question

coremark was impacted by some extra print

duchao713 opened this issue 3 years ago · comments

I added some print in the "main" function (after the "iterate" function and the total_time calculation, for sure), then I found that the coremark changed.

Is that reasonable ?
I thought it should not impact the coremark since I did not change the iterate process at all.

Thanks.

Peter Torelli · Answer 1 · Fri Apr 01 2022 23:21:34 GMT+0800 (China Standard Time)

Your inclination is correct, adding code outside the timing loop should not impact the score. If you could share your modifications it would help debug.

Chao Du · Answer 2 · Wed Apr 06 2022 17:54:12 GMT+0800 (China Standard Time)

For example, I made a test on my machine (X86, Linux), with the latest coremark source code on 'main' branch.

I typed 'make' directly.

before any modifications, 'run1.log' shows:

2K performance run parameters for coremark. CoreMark Size : 666 Total ticks : 13759 Total time (secs): 13.759000 Iterations/Sec : 29071.880224 Iterations : 400000 Compiler version : GCC7.5.0 Compiler flags : -O2 -DPERFORMANCE_RUN=1 -lrt Memory location : Please put data memory location here (e.g. code in flash, data on heap etc) seedcrc : 0xe9f5 [0]crclist : 0xe714 [0]crcmatrix : 0x1fd7 [0]crcstate : 0x8e3a [0]crcfinal : 0x65c5 Correct operation validated. See README.md for run and reporting rules. CoreMark 1.0 : 29071.880224 / GCC7.5.0 -O2 -DPERFORMANCE_RUN=1 -lrt / Heap

then I deleted some prints (line 359-378, core_main.c)，then 'run1.log' shows:
2K performance run parameters for coremark. CoreMark Size : 666 Iterations : 400000 Compiler version : GCC7.5.0 Compiler flags : -O2 -DPERFORMANCE_RUN=1 -lrt Memory location : Please put data memory location here (e.g. code in flash, data on heap etc) seedcrc : 0xe9f5 [0]crclist : 0xe714 [0]crcmatrix : 0x1fd7 [0]crcstate : 0x8e3a [0]crcfinal : 0x65c5 Correct operation validated. See README.md for run and reporting rules. CoreMark 1.0 : 29014.942695 / GCC7.5.0 -O2 -DPERFORMANCE_RUN=1 -lrt / Heap

The total_time was already got before the print, so I thought it should not change the score.

Chao Du · Answer 3 · Wed Apr 06 2022 18:00:24 GMT+0800 (China Standard Time)

It is only the fluctuation which is acceptable ?

Peter Torelli · Answer 4 · Wed Apr 06 2022 23:16:50 GMT+0800 (China Standard Time)

This is normal run-to-run variation when running on an OS. Typically we expect to see a few percent variation, you are seeing 1/10th that, about ~0.2%. If this was running baremetal on a system with no IO, no asynchronous tasks, and no interrupts, you would see the same score to multiple decimal points.