Unawaited promises behavior
Marsup opened this issue · comments
Hi,
I am seeing a strange behavior that may be coming from a misunderstanding of how piscina or worker threads behave.
In a scenario where the worker triggers some async task that is not awaited, it seems that this task is "suspended", and will be resumed the next time the worker wakes up.
I have put up a reproduction here. Notice how the "Resolved" logs appear at the beginning of the next run of the worker and not 500ms after it's finished. This is a trivial case with timers, but it happens for us with http (axios) calls.
What am I missing here?
Hey!
This delay is expected, and it is due to how Piscina is designed. Here it is important to highlight how the tasks are processed.
On Piscina, each script you define as Worker (after worker thread creation and so on), is instantiated as a Worker class internally. This allows Piscina to handle communication between the workers and the parent thread to orchestrate result communication and task scheduling.
For a Worker to consider a task done, the worker will execute the exposed function by the script in either sync or async way by calling the function entry point and waiting for a result or by waiting for the Promise to resolve/reject.
After the worker finishes, it will send back the result to the parent thread (main thread) and pause the worker execution, including the event-loop until a new task is scheduled (if there is something enqueued for stdout or stderr Piscina will wait for them to be flushed as well before halting the worker).
The trick is after the completion of the task. As if you spawn async tasks that open out-of-scope-handles outside of the scope of the Worker entry point, Piscina will not be able to detect them and will stop the execution of the worker until the Parent thread indicates that there is a new task ready for being picked (reference#Worker.ts).
As Piscina is not able to detect you have an open handle, as the console.log has not been buffered on stdout yet because the handle stills open, Piscina will hold the execution for worker until further notice (reference#index.ts).
Here the obvious recommendation will be to not do that and better to handle everything within the scope of the entrypoint of your worker as most likely your app can enter into an undesired state.
Hey,
Sorry for the late reply. Thanks a lot for this very detailed answer, it's all clear now.
Shouldn't it be part of the docs? I think it deserves a bit more exposure.
No worries! Happy to be of help 🙂
Yeah, I think it would be super helpful to have it in the docs 👌