Excessive CPU/RAM usage
IscoV opened this issue · comments
Hello, here is my problem.
I am new to using node-cron, I scheduled the task correctly and checked that it works correctly. The problem is in the excessive use of CPU/RAM that node-cron generates and it collapses the VPS server where it is stored.
Attached image
Although the CPU usage looks low, it increases over time.
All this when the script is started and kept active with pm2
If anyone knows anything about this or how to mediate excessive resource usage, I would be grateful.
I'm seeing similar behavior using node-cron 3.0.1. The worker script runs out of memory, crashes, and restarts about twice an hour.
Here's the snippet of code I'm using to set up my cron jobs:
cron.schedule('0 9 * * 1', async () => {
console.log('Running weekly report job');
await weeklyReportJob.run();
}, {
timezone: process.env.APP_TIMEZONE
});
// monthly on the first
cron.schedule('0 13 1 * *', async () => {
console.log('Running billing report job');
await billingReportJob.run();
}, {
timezone: process.env.APP_TIMEZONE
});
Other info:
Node cron: 3.0.1
Node version: 16
Typescript version: 4.6.4
Note: production version is compiled with tsc to run in pure JS
I think maybe some memory leak in 3.0.1
down-grade to 3.0.0 (no change other code) then ..solved....
We also faced problems in 3.0.1 regarding increased memory consumption. About 150MB more memory was used by our services. Downgrading to version 3.0.0 again solved the issue.
We had the same problem, ram usage increased by about 10mb per minute, crashing our express.js server after 6 hrs. This was replicated locally and occurred even with an empty service running. Downgrading to 3.0.0 solved it.
I faced a similar issue. And noticed that once I removed a options.timezone specification in my job the RAM usage was stable.
Here's a screenshot of my metrics.
I did notice that the timematcher implementation was updated between the versions to get rid of the moment dependency.
I might need to spend some more time investigating the cause. @supercoffee Would you be able to give it a try without the timezone set?
Other info:
Node cron: 3.0.1
Node version: 16
Typescript version: 4.7.4 (tranpiled to JS)
@gokulchandra In my testing the memory leak only happens when a timezone is set. Without a timezone, there is no memory leak (at least for as long I monitored it).
@gokulchandra Good observation about the time matcher. I looked through the diffs myself a while back and noticed that too but I didn't think much of it. After searching for "intl datetimeformat memory leak", I found this old Github issue in Node itself about that specific function causing a memory leak. It seems relevant, but the issue is now closed and the fix was rolled out in v12.20.0 according to this comment.
@merencia could you please have a look?
@merencia If you are really considering dropping moment.js as a dependency, as even lighter (2Kb) alternative with timezone support I would recommend DayJS.
I suggest it in case at some point you plan to change it.
I consider that moment.js does not consume, it absorbs resources.
Also note that in version 3.0.0, although the cron constructor allows to set the name property, this is not reflected anywhere even when viewing the task.
@supercoffee thanks for pointing that out, in my case removing the timezone solved my R14 heroku alerts
@merencia or @t-bonatti - could you please have a look or hand over the maintenance of the package?
removing the timezone option is fixing it, thanks
Any fix for this other than removing timezone?
Does anyone know which Timezone is used if no timezone option is provided?
Does anyone know which Timezone is used if no timezone option is provided?
I would assume it uses UTC by default
It's a shame that a lib that is used by more than 60,000 other packages has such a critical problem and the author doesn't even bother to merge already created fix...
Is there any possibility this will be fixed? This caused a slow leak while I was out of town and I got a massive bill because of it.
@Tanner-Scadden use this patch for now
diff --git a/src/time-matcher.js b/src/time-matcher.js
index 52a0b8ea5db9914d7e7b7ef74f8da9c8fcdbc181..6f24160a528c799a43fea4f415237b6597918723 100644
--- a/src/time-matcher.js
+++ b/src/time-matcher.js
@@ -15,6 +15,18 @@ class TimeMatcher{
this.pattern = convertExpression(pattern);
this.timezone = timezone;
this.expressions = this.pattern.split(' ');
+ this.dtf = this.timezone
+ ? new Intl.DateTimeFormat('en-US', {
+ year: 'numeric',
+ month: '2-digit',
+ day: '2-digit',
+ hour: '2-digit',
+ minute: '2-digit',
+ second: '2-digit',
+ hourCycle: 'h23',
+ fractionalSecondDigits: 3,
+ timeZone: this.timezone
+ }) : null;
}
match(date){
@@ -31,20 +43,8 @@ class TimeMatcher{
}
apply(date){
- if(this.timezone){
- const dtf = new Intl.DateTimeFormat('en-US', {
- year: 'numeric',
- month: '2-digit',
- day: '2-digit',
- hour: '2-digit',
- minute: '2-digit',
- second: '2-digit',
- hourCycle: 'h23',
- fractionalSecondDigits: 3,
- timeZone: this.timezone
- });
-
- return new Date(dtf.format(date));
+ if(this.dtf){
+ return new Date(this.dtf.format(date));
}
return date;
same problem with node-corn 3.0.2
CONER https://www.npmjs.com/package/croner seems to be much better alternative