Not suitable as a drop in replacement of setTimeout
gblazex opened this issue · comments
Hi! Looks like a promising library to do submillisecond timing, but it seems inaccurate currently.
Here is a little test to demonstrate it:
It simply fills up a queue with numbers every millisecond and starts removing an element after 1000 milliseconds in the queue.
This means that the queue grows for 1000 milliseconds initially, then the removing kicks in, which keeps the element count at 1000 (one remove, one insert, one remove, one insert...).
var queue = [];
var counter = 0;
var delay = 1;
function addToQueueRecursive() {
// just add a unique number to the queue
var number = counter++;
queue.push(number);
// and keep doing it asyncly
setTimeout(addToQueueRecursive, delay);
// elements only stay in queue for a while
setTimeout(function () {
queue.splice(queue.indexOf(number), 1);
}, delay * 1000);
}
// start filling up the queue
addToQueueRecursive();
// simple logging to see the state of queue
setInterval(function () {
console.log(queue.length);
}, 1000);
This will show the correct 1000, 1000, 1000, ... log message over and over again.
If I replace setTimeout with this library however the number is kind of random (somewhere between 880-988).
This is what I added at the top to replace the native setTimeout:
var NanoTimer = require('nanotimer');
var setTimeout = function(fn, delayInMs) {
(new NanoTimer()).setTimeout(fn, null, delayInMs + 'm');
};
Thoughts?
And here is a sample output:
980
919
923
916
912
907
899
894
888
887
881
884
Thanks for the bug report. I do believe I have an idea of why this is occurring. In order to attain higher accuracy, this timer tries to use the internal setTimeout/setInterval functions (which do not poll) when a delay or interval greater than 25 ms is specified to aim for just before the desired execution time, and then switch over to fast polling. That way the timer doesn't constantly hog the CPU yet still retains good accuracy.
Now this is very dependent on your own CPU, OS, how many other things are running and so on, but what's happening is that your chain of setTimeout calls, which specify a delay of 1 ms, do not use that deferred case however your setInterval call does. So your setTimeouts are running along happily until 25 ms before setInterval should fire, the internal setInterval call returns and begins polling. At that point, I think the CPU is taken over and the remaining setTimeout calls do not get a chance to run. So you would expect then that there should be approximately 25 missing values in that case if all of the remaining setTimeout calls do not get a chance to run.
Indeed, that is almost the case on my system.
Slightly more than 975 which means a few of those remaining setTimeouts do get through.
Currently, I am far too busy to work up a solution for this but I just wanted to let you know I believe this is what is happening. Perhaps after I graduate college in ~2 weeks I will have some more time too look into it.
I used your nanotimer.setTimeout and the native setInterval.
It seems that setInterval is running fine (little delay in the beginning), but nanotimer doesn't produce a constant 1000 long queue.
elapsed: 1002ms
queue length: 981
elapsed: 1007ms
queue length: 941
elapsed: 1001ms
queue length: 911
elapsed: 1001ms
queue length: 881
elapsed: 1000ms
queue length: 890
elapsed: 1000ms
queue length: 889
elapsed: 1000ms
queue length: 888
elapsed: 1000ms
queue length: 883
elapsed: 1000ms
queue length: 888
elapsed: 1000ms
queue length: 882
elapsed: 1000ms
queue length: 888
elapsed: 1000ms
queue length: 884
elapsed: 1000ms
queue length: 895And the modified logging code:
// simple logging to see the state of queue
var last = Date.now()
setInterval(function () {
var now = Date.now()
console.log('elapsed: ' + (now-last) + 'ms');
console.log('queue length: ' + queue.length);
last = now;
}, 1000);
Also it seems to me that the timer becomes less and less accurate as the time goes on.
Instead of the desired 1,000,000 nanoseconds for a 1 ms delay this is what I see:
avg diff: 1024162 (976 samples)
avg diff: 1054310 (1902 samples)
avg diff: 1067710 (2814 samples)
avg diff: 1080811 (3703 samples)
avg diff: 1091285 (4583 samples)
avg diff: 1097565 (5467 samples)
avg diff: 1101790 (6353 samples)
avg diff: 1105817 (7232 samples)
avg diff: 1108950 (8112 samples)
avg diff: 1111693 (8991 samples)
avg diff: 1114225 (9866 samples)
avg diff: 1116352 (10742 samples)
avg diff: 1117358 (11626 samples)
avg diff: 1117787 (12515 samples)
avg diff: 1118405 (13401 samples)
avg diff: 1119557 (14279 samples)
avg diff: 1120607 (15156 samples)
avg diff: 1121744 (16031 samples)
avg diff: 1122790 (16906 samples)
avg diff: 1123676 (17781 samples)
avg diff: 1124520 (18656 samples)
avg diff: 1125407 (19528 samples)
avg diff: 1126175 (20402 samples)
avg diff: 1126851 (21276 samples)
avg diff: 1127530 (22150 samples)
avg diff: 1128213 (23022 samples)
avg diff: 1128738 (23896 samples)
avg diff: 1129496 (24764 samples)
avg diff: 1130031 (25635 samples)
avg diff: 1130674 (26504 samples)Ah okay I think I made that change and then forgot and came back to the code and thought you were using the timer's setInterval. I'll try to look into this in the near future.
Out of curiosity though, why are you using a chain of setTimeouts to add new values? This would cause the error to accumulate. Why not use a timer.setInterval to do the same thing? I can understand if you're just copying an implementation that uses the internal functions. With nanotimer though, the setInterval would be far better for this.
You've probably noticed that sometimes the first few calls have a pretty large error. In other words they might run to 1.2 - 1.5 ms instead of 1 ms. Then the system catches up and it becomes a bit more accurate. Even still, an error of 1.001 ms would come to a full millisecond error in this situation by the time you start removing things.
With setInterval, none of that would happen. The first interval might fire at 1.2 - 1.5 as before, but the second will know it needs to only wait 0.5 - 0.8 ms for the next.
Here is what I mean. You might need to play around with the variables if you want them encapsulated and such.
var NanoTimer = require('nanotimer');
var timer1 = new NanoTimer();
var timer2 = new NanoTimer();
var number;
var queue = [];
var counter = 0;
var delay = 1;
function addToQueue(){
number = counter++;
queue.push(number);
}
function splitQueue(){
timer2.setInterval(function(){
queue.splice(queue.indexOf(number), 1);
console.log(queue.length);
}, '', '1m');
}
timer1.setInterval(addToQueue, '', '1m');
timer2.setTimeout(splitQueue, '', '1s');
The only reason why this requires 2 timer objects is because currently each timer can only have a single setTimeout and setInterval running. This could probably be changed relatively easily though.
And here are the results from this snippet:
I think this library is supposed to be a more precise setTimeout / setInterval
Because with the native API things like setTimeout(fn, 0.1) is not possible currently.
I'd be using it for simulations where native setTimeout would limit speed of the simulation. It isn't always 1ms in real-life, but first I have to see that the code works in the simplest case (1ms, 1ms, 1ms, 1ms, ...), then I add random intervals and different distributions.
But currently it doesn't work in the simplest case. :(
Ah so you want varying timeout delays. Like I said, unfortunately I can't spend a lot of time debugging this issue currently. I am not surprised this bug exists either, because I never did any real testing with nanotimer and recursive uses when I first created it. Will try to find a fix soon.
After looking at this more, I am certain this is not a "bug". Since you're doing this recursively, you end up with 1,000 different setTimeout calls queue-ing up that are all delayed by 1 second that splice the array. Nanotimer uses the setImmediate call to process more quickly and be more accurate in the first place, but I think this issue proves that 1000 simultaneous setImmediate assignments cannot be handled. Further backing that up is the fact that your CPU does not behave the way mine does. It seems quite dependent on how fast your own computer is and how much of this it can handle. You mentioned the error on yours gets progressively worse, but on mine it stays quite constant.
I do think there is a solution to this however, and it still involves setInterval like I mentioned above. The fix would be to implement a function called changeInterval so you can apply your distributions or change it however you want in the code. This is certainly the way to go, since relying on setInterval results in just one process that has to be worried about with setImmediate rather than 1,000, or n.
After revisiting this, I get much worse results with the builtin setTimeout and setInterval functions (varying between 750 and 850), yet with nanotimer I'm getting consistently between 990 and 1000. So I don't think this is really an issue with nanotimer to say the least.