Parallel job rendering
hholzgra opened this issue · comments
Right now rendering only really works in a single threaded way, even though the underlying render library can render multiple maps at the same time.
Different ideas to run this as a system service:
- re-visit / re-enable the original multi threaded implementation
- run multiple service instances that simply process requests by different criteria, each of them still being single threaded; e.g having separate services for single page, multi page, and API requests)
- similar to the previous item, but have a "regular" and a "slow" queue, with jobs being moved to "slow" queue after some time
Additional TODOs:
- how to display multiple queue stats in map list, status overview ...
Current plan:
- add "queue" field to rendering job model
- display queue name in addition to position in list and detail view
- modify to_render() method to accept optional queue name
- add command line argument parsing to daemon.py to allow to specify queue name when calling
- add queue name to queue position indicator in list view
- file multi page layout requests to separate "multipage" queue
- make render status check multi-queue aware
- have new requests automatically filed into "default", "api" or "multipage" queue
- show queue name in map details
- allow to filter list by queue name
- have default systemd service render everything regardless of queue name?