Connection pooling
RajivKurian opened this issue · comments
All the examples on the net suggest using one connection and to depend on callbacks. Since node.js is single threaded is there any use of a connection pool given node's single threaded model? If the answer is yes, is there a way to use connection pool using node_redis?
Hey R,
node uses a connection pool under the hood (at least for HTTP, and I would imagine other protocols as well) — thus I think node_redis uses one as well.
Cheers,
D
I myself only see a single TCP connection being made from our backend server to redis over 6379 (as observed via tcptrack).
Could somebody clarify this? Does it make sense to maintain a single connection between client and server, or should node_redis be creating many connections (the way that the mongodb native node driver does it) to handle heavy loads?
Each client instance uses a single TCP connection. Because node_redis pipelines all commands, this ends up increasing throughput for most workloads by reducing the number of kernel/user traversals in both client and server. This is something that happens automatically, and most of the time, you shouldn't need to worry about it.
However, there are a few times when opening multiple connections is necessary. If you are doing subscribe or blocking pop, then this ties up the connection on the server side unti the command completes. If you are doing watch, then the watch commands will be shared between all activity of the server. For these cases, you need to set up a new connection.
I've toyed with automatically setting up new connections for these cases, and it ends up adding a lot of complication, especially with testing.
Matt, should we understand from your last comment that pooling is not going to be a part of node_redis?
I understand this is not useful in most cases, but as you said, there are few ones where it may actually make sense.
+1, a connection pool is useful for high throughput applications.
The most useful case of pooling I can think of is more of a contextual one -- where it transparently handles subscribing connections vs standard read/write connections, and possibly a separate connection for WATCH transactions. Other than that I don't objectively know where the benefit of pooling would surpass that of pipelining on a high throughput application.