Evaluate using Profile-Guided Optimization (PGO) and LLVM BOLT
zamazan4ik opened this issue · comments
Hi! A few days ago I read the article about PostgresML migration from Python to Rust. I was especially interested in your performance results since I have an additional idea of how it's possible to potentially improve the PostgresML performance.
Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects - the results are available here. According to the tests, PGO helps with achieving better performance in many software applications like databases, compilers, network applications, etc. I think trying to optimize PostgresML with PGO can be a good idea.
I can suggest the following action points:
- Perform PGO benchmarks on PostgresML. If it shows improvements - add a note to the documentation about possible improvements in PostgresML's performance with PGO. PGO documentation examples in different projects you can find here.
- Providing an easier way (e.g. via a build option) to build PostgresML with PGO can be helpful for the end-users and maintainers since they will be able to optimize PostgresML according to their own workloads.
- Optimize pre-built PostgresML binaries with PGO.
Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.
For the Rust projects, I recommend starting with cargo-pgo. More details about PGO support in Rust can be found in the official docs.
Here are some examples of how PGO optimization is integrated in other projects:
- Rustc: a CI script for the multi-stage build
- GCC:
- Clang: Docs
- Python:
- Go: Bash script
- V8: Bazel flag
- ChakraCore: Scripts
- Chromium: Script
- Firefox: Docs
- Thunderbird has PGO support too
- PHP - Makefile command and old Centminmod scripts
- MySQL: CMake script
- YugabyteDB: GitHub commit
- FoundationDB: Script
- Zstd: Makefile
- Foot: Scripts
- Windows Terminal: GitHub PR
- Pydantic-core: GitHub PR
- file.d: GitHub PR
- OceanBase: CMake flag
- NodeJS: Configure script