Giters

redis / rueidis

A fast Golang Redis client that supports Client Side Caching, Auto Pipelining, Generics OM, RedisJSON, RedisBloom, RediSearch, etc.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Maximum pool size?

knadh opened this issue · comments

commented

Hi @rueian. Just starting to experiment with this lib.

How are maximum number of connection per client to a node determined and managed? BlockingPoolSize indicates that it's for specific commands like BPLOP. Does it actually dictate the total number of connections that a client will make to a node? The equivalent in go-redis would be PoolSize and MaxActiveConns.

commented

Hi @knadh,

BlockingPoolSize is used by blocking commands like BPLOP and is also used by client.Dedicate() and client.Dedicated(). They all share the same pool to a node. The maximum size of the pool is 1000 by default. If you don't use the above functions then no connection will be in the pool.

All other commands such as GET and SET share another pool. This pool, when rueidis connects to a cluster, uses 1 connection to a node by default. When rueidis connects to a standalone redis, it uses at most 4 connections by default depending on your CPU cores. This behavior can be changed by setting the PipelineMultiplex option.

commented

Thank you @rueian. Trying to understand the conceptual difference between how rueidis handles connection pooling compared to libs.

redigo and goredis both have maxIdle / maxActive constraints to control the pool size. Based on the amount of activity (and latency of responses) over a connection, new connections are spawned and messages spread across the pool until it hits max conns. It's easy to observe the jump in pool size on systems with heavy traffic. This is similar to DB drives, HTTP clients etc. Similarly, when there are many app instances that connect to one Redis instance, these params can be used to predictably control the max connections Redis will receive.

In that context:

All other commands such as GET and SET share another pool. This pool, when rueidis connects to a cluster, uses 1 connection to a node by default. When rueidis connects to a standalone redis, it uses at most 4 connections by default depending on your CPU cores. This behavior can be changed by setting the PipelineMultiplex option.

If say PipelineMultiplex = 100.

  • Does it keep 100 connections open to every node in a cluster (or a standalone instance?)
  • How does it interact with keep-alive? Do these connections die off when hitting the keep-alive limit and immediately get re-connected? Or is the re-connection based on activity?
  • What is the biggest conceptual difference between PipelineMultiplex and the min/max idle/active concept in other libs?

Apologies for firing off too many questions. Trying to understand the subtleties as there are no explicit control params. We're evaluating the lib for workloads in a critical financial environment. Thank you!

commented

Hi @knadh,

With the latest v1.0.26, the maximum PipelineMultiplex is 8 which results in 256 connections to a node because 1 << 8 == 256. I don't recommend setting it to larger than 2 which is more than 4 connections to a node. The Redis documentation also suggests that "Using more than 8 threads is unlikely to help much".

Say if PipelineMultiplex = 1 then

  1. rueidis keeps 2 connections open to every node in a cluster, because 1<<1 == 2. If you have 3 nodes then there will be 6 connections opened by rueidis.
  2. These connections will die only in two cases: 1) there is an IO error, including a TCP keep-alive error and 2) client.Close() is called. In the former case, the dead connection will be reconnected based on activity.
  3. The min/max idle/active concept is much more suitable for blocking use cases. For pipelining use cases, we tend to share connections and keep them open.
commented

That explains it. Thank you!