Scale to avoid "Waiting in line. X ahead of us."
nicolas-raoul opened this issue · comments
I often use the tool, but most of the time I get:
Waiting in line. 2 ahead of us.
... or similar.
When that happens, the wait last for a very long time, so I always give up and search manually.
That's a shame, it is a wonderful tool when it works. It is essential if Commons really wants to become the place to go when looking for a quality image.
I know it is easier said than done, but how about making fastcci more scalable?
- Run several requests in parallel threads?
- Install the whole thing on several servers and do some load balancing?
- Set up a time limit? (for instance only return the results found within 1 minute)
Yes, I often run into the same issue.
It is my understanding that FastCCI already is load-balanced across 3 workers − see https://tools.wmflabs.org/openstack-browser/project/fastcci for relevant links.
I have redeployed fastcci onto new VMs with more ram and I have taken measures to avoid server crashes when an invalid database is generated due to interrupted MySQL connections.