ashtuchkin / node-millenium

Node.js 1 million HTTP Comet connections test

Node.js test for 1M HTTP Comet connections

This is the server part of my test of 1M concurrent connections on Node.js server. Client part is ec2-fleet and uses Amazon Web Services.

If you can read Russian, see habrahabr article for more details.

See some graphs below.

How it looks like

Reproducing the test

Prerequisites:

1. Dedicated server on your favorite hosting with >=16Gb RAM and >=8 cores, external IP.
2. AWS account with 1 dollar. We will use 40 micro instances ($0.02/hr) for 1 hour.

Steps:

1. Prepare server.
2. Install fresh Ubuntu 12.04.
3. Increase limit on open file descriptors: write * - nofile 1048576 to /etc/security/limits.conf. Reboot.
4. git clone git://github.com/ashtuchkin/node-millenium.git and run node server.js. Leave this terminal open.
5. Set up AWS and ec2-fleet.
6. Open EC2 control panel of your AWS account https://console.aws.amazon.com/ec2/home
7. Choose 3 regions where you will launch client instances in. In each of them, go to Security Groups, choose 'default', go to tab 'Inbound' at the bottom, add custom TCP rule: port 8889, source 0.0.0.0/0. Dont forget to 'Apply Rule Changes'.
8. On your laptop, git clone git://github.com/ashtuchkin/ec2-fleet.git. Edit file aws-config.json to add your AWS security keys and regions you've chosen.
9. Start test.
10. In separate terminal, issue ./aws.js status. This will give you an overview of all instances in all regions. Leave this open too.
11. Start 40 instances in AWS: ./aws.js start 40. Wait ~2 minutes while they are starting.
12. Target them to your server: ./aws.js set ip <ip of your server>.
13. Gradually increase (in steps of ~2500-5000) the number of connections each instance makes to the server ./aws.js set n <number of connections> until you reach maximum of 25000 connections.
14. Go have a beer. You have a server with 1 million connections (40*25k).
15. Gradually decrease the number of connections (same steps).
16. Terminate all aws instances that we started (if you had other instances running, they are not touched): ./aws.js stop all
17. Get log from server to make pretty graphs.

Test: node server.js

Dotted line - connection count, with maximum at 1 million.

Test: node --nouse-idle-notification server.js

Test: node --nouse-idle-notification server.js (2 times)

Test: node --nouse-idle-notification --expose-gc server.js

(Garbage collector is manually called every minute)

License: MIT