Problem with cron job implementation and execution
AnonC0DER opened this issue · comments
Description
I am encountering an issue while using the node-cron library for scheduling cron jobs. I'm unsure whether the problem lies in my implementation or with the library itself, but it seems likely that the issue is with my implementation as I couldn't find any relevant issue.
I attempted to implement a feature to store jobs in my SQLite database so that they are not lost when the application restarts. However, I am facing conflicts in the execution time of these jobs. For instance, if I create a job with a 1-minute cron expression and run it on initialization, and then create 3 or 4 more jobs with different time intervals, they do not execute as expected. The job that should run every 20 minutes gets executed sooner than it should.
I have made efforts to debug my code, and what I have observed is that the jobs are stored correctly, and their cron expressions are accurate. However, they do not execute at the right time. Even when I use the nextDate function to determine the next execution times, it shows incorrect times despite having the correct cron expression.
Code Snippet:
Here is the relevant part of my implementation:
declare global {
var jobs: Array<JobType>
}
globalThis.jobs = []
// Loop to create jobs (inside a public method)
for (let reminder of reminders) {
globalThis.jobs.push({
job: new CronJob(
reminder.job.cron_expression,
async () => this.runReminder(reminder.id),
null,
true
),
jobId: reminder.jobId
})
}
// Private method to run the reminder
private async runReminder(reminder_id: number) {
// Retrieving reminder details from the database
let reminder = await this.prisma_client.reminder.findFirst({
where: { id: reminder_id },
include: { job: true }
})
if (!reminder) return
if (reminder.job.is_enabled) {
Constructing the reminder message
let base_url = HAMCHAT_URL.split("@")[1]
let assigner_msg_url = null
let assignee_msg = `Reminder: ${reminder.description}`
console.log(assignee_msg)
// Updating the last run timestamp in the database
await this.prisma_client.job.update({
where: { id: reminder.jobId },
data: { last_run: new Date() }
})
}
}
// Method to stop jobs (runs within an interval every 6 seconds)
public async stopJobs() {
for (let job of globalThis.jobs) {
let job_obj = await this.prisma_client.job.findFirst({
where: { id: job.jobId }
})
if (!job_obj?.is_enabled) {
job.job.stop()
const index = globalThis.jobs.indexOf(job, 0)
if (index > -1) {
globalThis.jobs.splice(index, 1)
}
}
}
}Expected Behavior:
The cron jobs should execute according to their specified cron expressions and intervals.
Actual Behavior:
The cron jobs are not executing at the expected times. The execution time some jobs conflicts with the specified cron expressions.
Additional Information:
- Node-cron library version: 3.0.2
- Database: SQLite
Screenshots
No response
Additional information
No response
Hello @AnonC0DER 👋
First of all, the code you show is indeed the API of the cron library (this repository), but version 3.0.2 is the latest version of the node-cron library (not this repository), while our latest version is 2.3.1.
I'm still trying to figure out where your issue could come from, maybe it has something to do with your stopJobs function? I'm also not sure whether the onTick function can be async, so I need to investigate this as well.
In the meantime, could you try to provide more debug information, like logs showing how the jobs execute at the wrong time? It would be helpful to issue debug logs (with the current time, job id & cron expression) when:
- a job is instantiated inside your public method
runReminderruns for a job- a job is stopped inside
stopJobs
Hello @sheerlox, thank you for your response.
I apologize for the confusion. Initially, I thought the issue might be with the library, so I attempted to use node-cron. However, I ended up confusing them due to the similarity in names. The current version of the library is 2.2.0.
Regarding the stopJobs method, it won't be able to stop anything if the is_enabled field in the database is set to true. Since all my jobs are currently enabled, I don't think the issue lies with this method.
Logs
First of all, here is what the stored data looks like in the database:
Job model
id | cron_expression | is_enabled | last_run | creation_date
7 | */1 * * * * | 1 | 1690100220347 | 1690099544193
6 | */10 * * * * | 1 | 1690101000395 | 1690099536397
Reminder model
id | assigner| assignee| jobId| description
7 | hesam | hesam | 7 | test 1m
6 | hesam | hesam | 6 | test 10m
This is what I have inside the jobs global array:
{
job: <ref *1> CJ {
context: [Circular *1],
_callbacks: [ [AsyncFunction (anonymous)] ],
onComplete: null,
cronTime: {
source: '*/10 * * * *',
second: [Object],
minute: [Object],
hour: [Object],
dayOfMonth: [Object],
month: [Object],
dayOfWeek: [Object]
},
unrefTimeout: undefined,
running: true,
_timeout: Timeout {
_idleTimeout: 341100,
_idlePrev: [TimersList],
_idleNext: [TimersList],
_idleStart: 10196,
_onTimeout: [Function: callbackWrapper],
_timerArgs: undefined,
_repeat: null,
_destroyed: false,
[Symbol(refed)]: true,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 937,
[Symbol(triggerId)]: 0
}
},
jobId: 6
}
---
{
job: <ref *1> CJ {
context: [Circular *1],
_callbacks: [ [AsyncFunction (anonymous)] ],
onComplete: null,
cronTime: {
source: '*/1 * * * *',
second: [Object],
minute: [Object],
hour: [Object],
dayOfMonth: [Object],
month: [Object],
dayOfWeek: [Object]
},
unrefTimeout: undefined,
running: true,
_timeout: Timeout {
_idleTimeout: 41096,
_idlePrev: [TimersList],
_idleNext: [TimersList],
_idleStart: 10200,
_onTimeout: [Function: callbackWrapper],
_timerArgs: undefined,
_repeat: null,
_destroyed: false,
[Symbol(refed)]: true,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 938,
[Symbol(triggerId)]: 0
}
},
jobId: 7
}
As you can see, the cron expressions are set correctly.
This is what exactly runReminder is doing :
-----------------------------
Reminder: test 6m
last_run: null
creation_time: Sun Jul 23 2023 12:14:29 GMT+0330 (Iran Standard Time)
now: Sun Jul 23 2023 12:18:00 GMT+0330 (Iran Standard Time)
-----------------------------
Reminder: test 1m
last_run: Sun Jul 23 2023 12:17:00 GMT+0330 (Iran Standard Time)
creation_time: Sun Jul 23 2023 11:35:44 GMT+0330 (Iran Standard Time)
now: Sun Jul 23 2023 12:18:00 GMT+0330 (Iran Standard Time)
-----------------------------
Reminder: test 3m
last_run: Sun Jul 23 2023 12:15:00 GMT+0330 (Iran Standard Time)
creation_time: Sun Jul 23 2023 12:14:24 GMT+0330 (Iran Standard Time)
now: Sun Jul 23 2023 12:18:00 GMT+0330 (Iran Standard Time)
-----------------------------
Note that this problem is something I can't easily reproduce. Sometimes, only the first reminder execute at the wrong time, while all subsequent intervals are as expected. I truly have no idea what is causing this issue.
This issue #680 might be related to mine, too.
Maybe something goes wrong when we try to store things.
@AnonC0DER from the logs you sent, the execution times seem to be working as expected.
For example, the job supposed to run every 6 minutes ran at 12:18, because it runs every 6 minutes from minute 0.
Are you trying to make it run every 6 minutes from the creation date?
Yeah, you are correct. The misunderstanding might be because of too many jobs at the same time.
But it would be great if I could run the job every 6 minutes from the creation date, can I manage that using this library?
[EDIT] tl;dr: it doesn't seem to be possible using the cron syntax, thus neither with this library.
It might be possible, but because (the original) cron wasn't designed to cover this use case, it will require a bit of work to determine the correct cron expression.
Let's take your 6m interval (*/6 * * * *) as an example:
Reminder: test 6m
last_run: null
creation_time: Sun Jul 23 2023 12:14:29 GMT+0330 (Iran Standard Time)
now: Sun Jul 23 2023 12:18:00 GMT+0330 (Iran Standard Time)
You could offset your expression so it starts at the creation minute: 14-59/6 * * * * (see this SO answer).
But by doing this, it will never run between minute 0 to 14, so you would need to find the minimal offset required to be aligned. You can do this like so:
minutesOffset = creationMinute - minutesInterval * Math.floor(creationMinute/minutesInterval)
In our example: 14 - 6 * Math.floor(14/6) = 2
Which would then give the following cron expression: 2-59/6 * * * *
But then I tested this approach with a 35-minute interval starting at minute 59, and it doesn't work as expected:
59 - 35 * Math.floor(59/35) = 24 -> 24-59/35 * * * *
From the Crontab.guru test:
next at 2023-07-23 11:59:00
then at 2023-07-23 12:24:00
then at 2023-07-23 12:59:00
then at 2023-07-23 13:24:00
Since minutes do not roll over in cron, from minute 24 to 59, there are 35 minutes as expected, but then from minute 59 to 24 only 25 minutes.
So I guess achieving the expected result with this library (or any library that only provides the cron syntax) isn't possible, and you probably need to find another solution/library that better fits your needs.
Maybe a simple setInterval would do the job here? What do you think would best suit you?
Please let me know, I'm interested in this specific use case and potential solutions 😉
Hey @AnonC0DER, did you manage to find an appropriate solution to your issue? 😄
Hello again, @sheerlox. I apologize for the delay in getting back to you.
I sincerely appreciate your assistance.
Although I haven't implemented it yet, I believe I have a good understanding of how to proceed.
I'm considering adding a field to my database that specifies the desired duration for the job to run.
By utilizing the creation_date and last_run fields, I can effectively manage the task at hand.
All I need to do is calculate the difference between those fields and compare the result with my interval field.
With this approach, I am confident that it will work as intended.
Thanks for your reply @AnonC0DER 🙏
I'm not sure I understand how that solves the issue, are we still talking about executing a job every x minutes from the creation date?
In any case, I don't want to overwhelm you, so if you'd agree to share more details once you've tried implementing it, I'd be more than happy to read your take 😄
I'll close this issue in the meantime since we agreed it isn't a bug from the library, but please feel free to update us here with your findings!
Also, if you'd like more help with your implementation, please join our Discord server so we can chat more openly ✨