Dispatcher stays running after daemons(n = 0L) when not all tasks are dispatched yet
wlandau opened this issue · comments
When there is a backlog of at least 1 task waiting to be dispatched to a worker, the dispatcher in mirai c79a7e0 no longer exits when I call daemons(n = 0L). Using Mac OS.
library(mirai)
library(ps)
url <- "ws://127.0.0.1:5700"
daemons(n = 1L, url = url)
#> [1] 1
dispatcher <- ps_handle(nextget("pid"))
tasks <- replicate(2, mirai(Sys.sleep(100)))
launch_local(url = url)
Sys.sleep(2)
daemons(n = 0L)
#> [1] 0
Sys.sleep(10)
ps_is_running(dispatcher)
#> [1] TRUE
Thanks! Talk of Chesterton's fences...
Thanks so much. And I see mirai 0.11.3 is on CRAN! Very exciting.
Is it feasible to have the daemons exit as well on daemons(n = 0L), even if there is a long task still running? The reason I ask is wlandau/crew#141. A targets pipeline runs inside a central background process, and interrupting that process skips the on.exit() callback which would ordinarily run controller$terminate() to kill the crew workers manually.
Sorry this came up just after a package release!
Exciting indeed! All the best with the crew / targets releases!
I've read that issue and it isn't clear to me that the daemon processes should terminate immediately as asserted. They terminate when their tasks finish, as they have realised, which is in fact as intended and documented in mirai.
The reason it is intended is that it is common for a 'mirai' to run a task such as writing to a file or database rather than returning a value. Think of model checkpointing as an example. You'd want that process to finish even if the main process crashes in the meantime.
If this is something you still want to take up, please open a new issue so it can be tracked.