Implement dynamic rate limiting algorithm for IRC twitter output

[Marak]

We need to implement a way for kohai to be a better irc citizen. Currently kohai is spitting out every tweet he picks up immediately. Sometimes this is valuable, other times it's disruptive.

There is a two part solution for this:

1. Kohai needs to be able to operate in multiple "volumes". The default volumes setting will be 10. This means at 10, kohai will spit out tweets as fast as he gets them in. A volume of 0 means that kohai is muted. Volume values will need to be set per room.
2. Any volume value above 0 will be plugged into a rate limiting algorithm. I don't have the exact math yet, but the basic idea is that kohai will determine how many tweets he can pump per 30 seconds as to how noisy the room is. The idea is that kohai will not dominate the conversation if other people are talking. If no one is talking, kohai's volume could increase to a level of 10.

[Sami Samhuri]

Here's a basic rate limiting queue I just came up with. I haven't run the code so for all I know it doesn't even parse, but someone might be able to run with it. I added a volume knob which has a one second latency. Making it instantaneous is easy, just call limit() after adjusting volume (mute & unmute could be changed to call volume() instead of setting volume directly).

https://gist.github.com/990471

Not sure this is even close to what you want but should be good for at least brainstorming if nothing else.

[Marak]

@samsonjs - Thank you!

It's going to take me a non-trivial amount of time to review this. I'll definitely test it out when I can. Hopefully @AvianFlu or others can check it out in the mean-time.

[Sami Samhuri]

No problem. The hard part is adjusting the volume anyway ;-) Hit me with any questions you might have.

[AvianFlu]

I'm going to make a silent bot to sit in #Node.js for a while and keep track of the number of IRC messages sent per minute over the observed time period. That data should be helpful in setting the specific volume levels later.

[AvianFlu]

databot is up and running in #Node.js. His nick is WallyDater. He is silent, and the saved data will be posted in a gist later.

[AvianFlu]

This is the first hour of data. https://gist.github.com/992044

[AvianFlu]

Some busier chat https://gist.github.com/992209

[Marak]

Okay, now that we understand that data, we should build the first piece of the puzzle in kohai. We need to implement the ability for kohai to store the current "chat_rate" of an IRC room.

The "chat rate" will be determined by the number of messages (n) sent in the past (x) seconds. The rate will update every 1 second. The "chat_rate" is a moving average. Here is some pseudo code, I haven't tried running it at all. It should calculate the chat_rate as a moving average.

      // pseudo code
      var chat_rate      = 0,
          message_count  = 0,
          timespan       = 30,
          rolling_values = [];

      client.on('message', function(){
        message_count++;
      });

      setInterval(function(){

        if (rolling_values.length >= timespan) {
          rolling_values.shift();
        }

        rolling_values.push(message_count);
        message_count = 0;

        var sum = 0, 
            cnt = 0;

        rolling_values.forEach(function(v) {
          sum += v;
          cnt ++;
        });

        chat_rate = sum;

      }, 1000);

[AvianFlu]

I've got this algorithm implemented and working - I'm moving on to the volume part next.