Is there a programmatic way to kill all workers of a queue?

[Geert]

During normal operations I'm using idleTimeout > 0 to allow workers to be kept around, allowing for some persistent caching in between invocations.

This has worked well for me except I'm now occasionally getting a OOM in a workerthread. Ideally I would get the source of the OOM, but the more pragmatic way seems to once in a while 'just' kill the workerthreads, requiring Piscina to create a new workerthread instance.

Is there a way to give this command to a Piscina instance? Alternatively, if that's not doable, I could periodically call destroy() and create a new Piscina instance I guess

[Carlos Fuentes]

👋

Currently, there's no way to achieve that in Piscina through its API.
Depending on how your workload looks like, you can periodically call Piscina#close to flush all the pending tasks and later destroy the pool.

Although destroyed, Piscina does not hold this state, so you can perfectly close/destroy the pool and start adding tasks right after without a problem.

[Carlos Fuentes]

e.g.

const { join } = require('node:path')

const Piscina = require('.')

const pool = new Piscina({
  maxThreads: 2,
  minThreads: 2,
  filename: join(__dirname, 'worker.js')
})

;(async () => {
  for (let i = 0; i < 10; i++) {
    await pool.run(`console.log(${i})`)
  }

  await pool.close()
  // or alternatively await pool.destroy()

  for (let i = 0; i < 10; i++) {
    pool.run(`console.log(${i})`)
  }
})()

[Geert]

So just to confirm: pool.close() should be enough to close all worker threads completely (thus freeing up all memory from worker threads)? Then afterwards I can just do pool.run() to add workers again?

Then I would not need pool.destroy().

Or asked differently, is there any reason to use pool.destroy() at all in the above scenario?

[Carlos Fuentes]

Use pool.destroy if you want to fully tear down the threads independently of their state (will discard any actual work and mark them as aborted).

Use pool.close if you seek for a graceful shutdown 👍