bgwriter takes 100% cpu in s3 mode
pashkinelfe opened this issue · comments
I created 10K partitions and did inserts in the table by pgbench in 50 connections.
After ~100s inserts in s3 mode bgwriter process took 100% CPU
In s3-disabled mode bgwriter was ~1% CPU
TPS (s3) was ~32000, TPS (without s3) - ~30000, so bgwriter didn't slowed down concurrent inserts. But maybe performance could be even better if bgwriter tasks could be somehow optimized?
orioledb.s3_num_workers = 20
Flamegraph of bgwriter during concurrent inserts. 60 sec@ 1000 probes per second
(original non-recompressed svg:
bgwriter-concurrent-inserts.svg.zip
Many similar pg_check_dir calls during checkpointing delay many-tables cluster shutdown in both s3 and non-s3 mode (#328)
Please, recheck on main (234115c)