Build tiles on multiple threads
Koxx3 opened this issue · comments
Hello,
I am trying to build the valhalla tiles using multiple threads.
It's running on docker on debian Vmware virtual machine (32 threads / 256Gb RAM).
I've updated the docker-compose.yml with concurrency / server_threads parameter ...
version: '3.0'
services:
valhalla:
image: ghcr.io/gis-ops/docker-valhalla/valhalla:latest
container_name: valhalla_latest
ports:
- 10002:8002
#build:
# context: .
# args:
# - VALHALLA_UID=1000
# - VALHALLA_GID=1000
volumes:
- ./custom_files/:/custom_files
# - ./gtfs_feeds:/gtfs_feeds # only enable with build_transit=True
environment:
# Auto-download PBFs from Geofabrik
#- tile_urls=https://download.geofabrik.de/europe/andorra-latest.osm.pbf https://download.geofabrik.de/europe/albania-latest.osm>
- tile_urls=http://download.geofabrik.de/europe/france-latest.osm.pbf
- server_threads=32 # determines how many threads will be used to run the valhalla server
- serve_tiles=True # If True, starts the service. If false, stops after building the graph.
- use_tiles_ignore_pbf=True # load existing valhalla_tiles.tar directly
- tileset_name=valhalla_tiles # name of the resulting graph on disk
- build_elevation=False # build elevation with "True" or "Force": will download only the elevation for areas covered by the gra>
- build_admins=False # build admins db with "True" or "Force"
- build_time_zones=False # build timezone db with "True" or "Force"
- build_transit=False # build transit, needs existing GTFS directories mapped to /gtfs_feeds
- build_tar=True # build an indexed tar file from the tile_dir for faster graph loading times
- force_rebuild=False # forces a rebuild of the routing tiles with "True"
# - path_extension=graphs # this path will be internally appended to /custom_files; no leading or trailing path separator!
# adapt to the expected build time
# healthcheck:
# test: curl --fail -s http://localhost:8002/status || exit 1
# interval: 1s
# retries: 10
# start_period: 2s
# timeout: 1s
mem_limit: 128g
cpus: 30
But it's still working on a single thread. why ?
Not all stages of the build process are multithreaded. Notably the OSM parsing stage (first one). Most things after that are.
ok, thanks for the answer !