mcollina / fastq

Fast, in memory work queue

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Waiting for the queue to be fully processed

robogeek opened this issue · comments

Your documentation didn't help me understand how to use the drain function to wait until the queue is fully processed.

What I came up with is:

const queue = fastq.promise(renderDocumentInQueue,
                            config.concurrency);

const waitFor = [];
for (let entry of filez) {
    waitFor.push(queue.push(entry));
}

await Promise.all(waitFor);

This is using the Promise version of fastq. In that version, the push method returns a Promise. Therefore pushing the Promise's into an array lets us use Promise.all to know when all tasks have been completed.

Does this look correct?

An alternative I've coded but haven't tested is:

// await new Promise((resolve, reject) => {
//    queue.drain = function() {
//        resolve();
//    }
// });

It seems from your test scripts that one simply assigns a function to the drain field, and it'll automatically be called. Therefore one way to wait for this function to be called is this Promise here.

Is either preferable over the other?

This is using the Promise version of fastq. In that version, the push method returns a Promise. Therefore pushing the Promise's into an array lets us use Promise.all to know when all tasks have been completed.

Does this look correct?

Yes