Extremely large loopMul iteration set leads to very long test run

[Holger Drewes]

We are just looking into analyzing some opcode behavior and performance in our VM and I've along tried to re-activate some previously slowly running tests we had excluded.

In the vmPerformance directory there are both these tests loopExp and loopMul.

While loopExp is running reasonably ok for us in under a second, loopMul takes for ages (48 Minutes !).

I now had a closer look and saw that the number of iterations is just dramatically higher on loopMul. While loopExp is executing the highly used opcodes in the range of 1000s times (e.g. PUSH opcode 2145 called), loopMul is doing a completely other level with e.g. ~ 130 Mio (!) executions of the PUSH opcodes and other opcodes on the same level.

Is this extremely high rate intended? Or is this an error/mistake in the filler code? Wouldn't something like at least 100x less executions (1 Mio or so) also be enough?

Thanks a lot! 🙏

[winsvega]

this is a legacy test that we have since a very long time. it is now converted into state test.
some tests here are not intended to be run by slow vm. this is basically a stress test so you can come up with optimisations. evmone can run ALL state tests on a scale of slow VPS as fast as 1500 seconds this is absolute record.
the reasonable time is 2500 seconds on slow VPS. http://retesteth.ethdevops.io/. anything more that 3000 seconds is an issue of t8n tool optimisation.

the test itself does not test anything as far as I know it just runs many instructions in the loop to see how fast you can process it. so it can be disabled.

[Holger Drewes]

Thanks, I know this is a stress test. There is just no gain in executing the same thing 130 Mio times. So 2500 seconds means > 40 minutes for a single test. This is just a waste of ressources and time, since 1 Mio executions or so would achieve the same thing.

Is it possible to just quickly adopt these numbers? Or is this too much buried in the legacy test filler? 🤔

[winsvega]

I see no problem, you can just make a copy of this test that is reasonable for your evm.

Because geth, nimbus, evmone
Run this in seconds