Benchmark ORM on Truffleruby vs CRuby
AlexeyMatskevich opened this issue · comments
I was inspired by the article and decided to reproduce the tests for ActiveRecord and ORM in the spirit of Shopify benchmark.
In the results of the article from 2022, the results were as follows
I've created a repository where I defined Dockerfile for ruby and for truffleruby.
And create test inspired by a version from Shopify benchmark but replaced sqlite with postgres.
To customize truffleruby, I have been studying your documentation and followed the next steps:
- Use Oracle GraalVM instead of Community Edition version
- Use the Latest Release 24.0.1
- Use the Correct Runtime Configuration - JVM configuration (install JVM version)
- Disabling the Global C-Extension Lock
- Run with Enough Warmup
I end up with the following results:
For MRI 3.3.1
user system total real
ActiveRecord: 8.095543 0.786087 8.881630 ( 10.745872)
For Truffleruby 24.0.1-JVM (I've tried running benchmark 10 times to test within a single run to check for warmup)
user system total real
ActiveRecord 0: 23.492176 1.226223 24.718399 ( 20.238396)
ActiveRecord 1: 19.394902 1.204641 20.599543 ( 20.041559)
ActiveRecord 2: 19.457829 1.269488 20.727317 ( 19.964144)
ActiveRecord 3: 19.282167 1.163987 20.446154 ( 20.192037)
ActiveRecord 4: 18.421440 1.026265 19.447705 ( 19.845829)
ActiveRecord 5: 18.538499 1.024583 19.563082 ( 19.866452)
ActiveRecord 6: 18.313226 1.094071 19.407297 ( 19.839851)
ActiveRecord 7: 18.565056 1.097710 19.662766 ( 19.882381)
ActiveRecord 8: 18.382542 1.079965 19.462507 ( 19.820898)
ActiveRecord 9: 18.484171 1.083560 19.567731 ( 19.841563)
My system:
NixOS 24.05
Docker version 24.0.9, build v24.0.9
AMD 5950x
128 GB RAM
SSD - intel 905p 1.5 TB
I believe I am doing something wrong that is causing me to get these results, can I get some help with this issue?
Let's discuss at #3583