Allow more than 32 generalization jobs
pnorman opened this issue · comments
osm2pgsql-gen currently is limited to 32 threads.
osm2pgsql/src/gen/osm2pgsql-gen.cpp
Lines 754 to 757 in 374eca4
This is limiting me to using less than 50% of my CPU.
Is there any hard limit elsewhere in the code that prevents the upper limit from being removed?
I took that limit from the --number-processes
parameter. But that is a bit different than the number of jobs here. So I don't have anything against making the limit larger. I think we should have some limit, to save the user from themselves doing something ridiculous here. And the question is at which point more threads will not make this faster.
Tile based CPU bound work like this will generally parallelize until the CPU is saturated, with as many threads as there are hardware threads. The OSMF render servers have 112 threads, and they're moderate as servers go.