`removeOnComplete` not working with only "age" key

[jvcmanke]

Description

I am currently running some tests to check if this library fits my use-case well, and I really like it so far.
However I am having trouble setting up the removeOnComplete feature with the KeepJobs option of age, it seems to not remove ever.

I have tested with the removeOnComplete option set to true and it is removed after completion as expected.

Minimal, Working Test code to reproduce the issue.

I am running two separate processes one as a producer and one as the consumer:

// producer.js
const Queue = require("bull");

const queue = new Queue("myqueue", "redis://....");

async function createJob() {
  const job = await queue.add(
    { message: "Hello" },
    { removeOnComplete: { age: 5 } }
  );
  console.log("Created!");

  const jobId = job.id;
  let count = 0;
  const intervalId = setInterval(async () => {
    const fetchedJob = await queue.getJob(jobId);
    if (fetchedJob) {
      console.log(
        count++,
        fetchedJob.id,
        fetchedJob.returnvalue,
        await fetchedJob.isCompleted()
      );
    } else {
      console.log(count++, "didn't find");
      clearInterval(intervalId);
    }
  }, 1000);
}

createJob();

// consumer.js
const Queue = require("bull");

const queue = new Queue("myqueue", "redis://....");

function delay(ms) {
  return new Promise((resolve) => {
    setTimeout(() => resolve(), ms);
  });
}

console.log("Running...");
queue.process(async (job) => {
  console.log(job.data);

  await delay(3000);

  return { message: "World!" };
});

You can run them on a console with node consumer.js and node producer.js

The idea here is:

• In the producer:

  • We add a message to the queue (with removeOnComplete = { age: 5 } );
  • Call a function every 1 second fetching the job via its ID;
  • Log some information (Job ID, Response, isCompleted);

• In the consumer:

  • We consume the message from the same queue;
  • Wait for 3 seconds;
  • Respond with a response object;

What I expected to see:

1. Send de message;
2. Find it with status not completed for around 3 seconds;
3. Find it with status completed and response for around 5 seconds;
4. Not find it anymore;

On the producer's console should result:

0 8 null false
1 8 null false
2 8 null false
3 8 { message: 'World!' } true
4 8 { message: 'World!' } true
5 8 { message: 'World!' } true
6 8 { message: 'World!' } true
7 8 { message: 'World!' } true
8 didn't find

However it keeps "finding" the Job indefinitelly.

0 8 null false
1 8 null false
2 8 null false
3 8 { message: 'World!' } true
4 8 { message: 'World!' } true
5 8 { message: 'World!' } true
6 8 { message: 'World!' } true
7 8 { message: 'World!' } true
8 8 { message: 'World!' } true
9 8 { message: 'World!' } true
10 8 { message: 'World!' } true
11 8 { message: 'World!' } true
12 8 { message: 'World!' } true
...

Bull version

v4.11.3

Additional information

[Juraj Zovinec]

Just noting this feature is not working with removeOnFail in version Bull version 4.8.4.

[Manuel Astudillo]

We have tests for this feature here:

bull/test/test_queue.js

Line 734 in 4d197e8

 it('should keep of jobs newer than specified after completed with removeOnComplete', async () => {