Consider decoupling reading from socket and the parsing, logging of HTTP, and sending responses
bmenrigh opened this issue · comments
Right now each worker does a read on the socket to get the HTTP request, parses the request, and then builds a response to send back.
Building the response is very CPU intensive so we don't want too many workers going in parallel or everything slows down. Testing shows that maximum performance is achieved roughly when the number of workers matches the number of physical CPU cores.
However, reading on the socket can tie a worker up doing nothing for however long the client-read-timeout is (by default 300 ms). This means with just a small number of bogus requests at a time, all the workers can be tied up for 300 ms. It's easy to starve legitimate requests by tying up workers in this way.
One possible solution is to have a very large pool of workers that just read from the sockets and then a smaller set of workers that parse the requests and respond. The easiest thing to do here is to take out the reading code and put it in its own worker pool and then feed the read results to the existing (now modified) workers.
So instead of ACCEPT -> WORKER we'd have ACCEPT -> READ -> WORKER where there are many more routines implementing READ. The more routines that are dedicated to just reads, the harder it is to tie up all of the reading routines waiting for a timeout.
This was done in commit e7f3a8b
Now connections are processed in two stages. First a "readworker" calls Read() on the socket and then whatever is read is handed off to a normal worker. This allows there to be far more readworkers than normal workers, all reading on sockets and possibly waiting for a read timeout. In testing, this makes Slowloris style attacks ineffective.