benchmarks affect each other?
leeoniya opened this issue · comments
hey @dfilatov
i was running the SSR benchmarks and noticed a significant difference when running each independently vs all at once. i assume there's some mem leaks or poor GC interactions which affect the results between bench runs.
this is what i'm seeing: https://github.com/leeoniya/domvm/tree/3.x-dev/demos/bench/ssr
Hi, @leeoniya
Actually, when I was implementing it, I had only noticed that Vue affected all other results after it. So I had to put it last.
Currently I've tried to run them separately and haven't noticed any significant difference on my macbook.
k, another point to consider is running them on a desktop/macbook vs a linux server (where SSR will almost certainly be running). i could not get vidom metrics to be faster than inferno on a 4096 Linode instance in node v8.1.3 or on my win10x64 desktop.
just thought i'd point this out. you may also be interested in the recently started https://github.com/ivijs/ssr-benchmark which includes vidom's benchmarks.
original thread here: localvoid/ivi#1
on linode domvm does about 20x React's perf (not 14x as mentioned in the thread, which was my desktop metrics)
Yes, the difference between vidom and inferno is tiny on nodejs 8.x, about 10% on my macbook, unlike nodejs 6.x where it can reach 40-50%.