Performance degradation in the current master

Question

Performance degradation in the current master

ujh opened this issue 9 years ago · comments

While working on #177 I wanted to have a performance baseline of the current master to see how it would compare to the engine with my code added. To do that I started running bin/benchmark on 6d2caa7 and discovered that the percentag of wins on 9x9 dropped to 65.5% ± 3.4. That's 10% less than 0.2.3! This is of course unacceptable and I also have the feeling that this can't just be due to changes in the Rust compiler. To verify this I'll run the following benchmarks:

6d2caa7 on 13x13
The compiled binary for 0.2.3 on 9x9 & 13x13 (I might just have made an error when writing down the numbers in the CHANGELOG)

If these results stay the same I'll investigate the differences between 0.2.3 and the current master.

Urban Hafner · Answer 1 · Mon Sep 14 2015 16:20:28 GMT+0800 (China Standard Time)

The 13x13 tests however are looking good: 10% ± 2.1. That's significantly better than 0.2.3 with 6% ± 1.7.

Urban Hafner · Answer 2 · Mon Sep 14 2015 16:25:52 GMT+0800 (China Standard Time)

Maybe it's good enough to just release 0.2.4 so that I have a proper baseline for #177? What do you think @iopq?

Urban Hafner · Answer 3 · Mon Sep 14 2015 23:17:54 GMT+0800 (China Standard Time)

I went ahead a create 0.2.4.