Error thrown back to scheduler is undefined.

[vtamosaitis]

Description

I am working with something similar to below code. I am not able to correctly handle the error properly because the error thrown back to the scheduler is undefined. I've checked that the correct error is logged by the 'failed' event handler. I've also checked that the error received by the processor is correct. Below code does not actually give the error I am having, but it is unfortunately the best I can provide because actual code is calling a google API (through google-api-nodejs-client), so actual code contains secrets, etc. I went through a good amount of effort to try to reproduce without the API call, but was having no luck.

I am primarily hoping that the below snippet gives a good enough overview so that someone might have some idea why an error which is received by the 'failed' event listener and the processor is not being received by the scheduler or from anywhere lower on the call stack, yet the errors are still being caught in a try-catch.

import Queue from 'bull';
import { calculateTotal } from './calculateTotalService';

export class ServiceQueueProcessor {
    private queue: Queue.Queue;

    constructor() {

        this.queue = new Queue('service_queue', {
            redis: {
                host: 'localhost',
                port: 6379
            },
            limiter: {
                max: 2, // Process 2 job at a time
                duration: 150, // 5 second delay between batches
            },
            defaultJobOptions: {
                removeOnComplete: 0,
                removeOnFail: 50,
            },
        });
        
        this.queue.process('*', async job => {
            const serviceName = job.name;
            const args = job.data;
            switch (job.name) {
                case 'api_call':
                    // const res = await googleService.someApiCall(args[0], args[1]);
                    // api call throws custom error like throw {some: 'test', error: 'test'} 
                default:
                    throw new Error('Invalid job name', job.name);
            }
        });
        
        this.queue.on('failed', (job, err) => {
            console.error('Job ${job.id} failed', { err, job: job.toJSON() });
        });
    }
    
    async scheduleAPICall(a: number, b: number): Promise<number | void> {
        const job = await this.queue.add('api_call', [a, b]);
        try {
            console.log('scheduler entered');
            const result = await job.finished();
            return result;
        } catch(error){
            console.error('scheduler error', { error, job: job.toJSON() });
            throw error;
        }
    }
}

Bull version

^3.11.0

Additional information

Bull is being used as a rate limiter and is replacing another library which no longer works as of Redis v7.

[Manuel Astudillo]

Why don't you just check what error you are getting here:

case 'api_call':
                    // const res = await googleService.someApiCall(args[0], args[1]);
                    // api call throws custom error like throw {some: 'test', error: 'test'} 

and translate it or something to a proper Error object and then throw that instead?

Btw, if you are starting with Bull it is recommended to use BullMQ instead: https://github.com/taskforcesh/bullmq

[vtamosaitis]

Why don't you just check what error you are getting here:

case 'api_call':
                    // const res = await googleService.someApiCall(args[0], args[1]);
                    // api call throws custom error like throw {some: 'test', error: 'test'} 

and translate it or something to a proper Error object and then throw that instead?

Returned the custom error and rethrew it in the scheduler instead. For some reason it works this way when throwing does not work.

Btw, if you are starting with Bull it is recommended to use BullMQ instead: https://github.com/taskforcesh/bullmq

I had done some playing around with bullmq, but it would require upgrading dependencies, which also meant upgrading environment. Not something I wanted to take up at the moment, but I have a solution ready for upgrading to bullmq when that time comes.