django / channels_redis

Connecting to redis is no longer done by calling await aioredis.create_pool(...).
link

Plus 1, any plans for upgrading airedis?

This has been discussed in #268 and mentioned here.

Upgrading to 2.0 is the plan, just no one has done the work yet.

This library is now part of the mainstream Redis library as of 4.2.0-rc1 redis/redis-py#1899

Oh, that is good!

Very happy to take a PR implementing this swap over.

I dug into this a bit, could you explain what the reasoning is for this?

channels_redis/channels_redis/core.py

Lines 73 to 79 in f5eef16

    
           async def create_conn(self, loop): 
        
               # One connection per pool since we are emulating a single connection 
        
               kwargs = {"minsize": 1, "maxsize": 1, **self.host} 
        
               if not (sys.version_info >= (3, 8, 0) and AIOREDIS_VERSION >= (1, 3, 1)): 
        
                   kwargs["loop"] = loop 
        
               if self.master_name is None: 
        
                   return await aioredis.create_redis_pool(**kwargs)

It's creating a single connection pool then treating and referring to it as a connection. Some of the paradigms the library uses like checking if the pool is closed don't map in redis-py. You can check if a connection is closed and you can release all the connections in a pool, but you don't check if a pool is closed.

It was added for the sentinel support. 🤔 @qeternity — Can you comment on the reasoning there? Thanks.

Hi @ipmb - The API for pools and connections is very similar (at least in aioredis 1.3.x). We can't use a raw redis connection with sentinel because a sentinel connection is actually two redis connections: one pubsub connection to a sentinel, and then a connection to the actual redis instance. You need to listen to the pubsub connection for changes from the sentinel(s). This logic was all orchestrated automatically by the aioredis sentinel pool. However, because non-pubsub layer does not use the library pooling, and marshals connections manually, we don't want to risk letting the sentinel connections in the library pool grow, because we will end up having many pools. All of this was done to make sentinel drop-in compatible with the non-pubsub layer.

I have not looked at the new redis-py migration, but in the 1.3.x branch, you absolutely could check if a pool is closed: https://github.com/aio-libs/aioredis-py/blob/8a207609b7f8a33e74c7c8130d97186e78cc0052/aioredis/sentinel/pool.py#L160

Yeah, it's not available here https://github.com/redis/redis-py/blob/c5d19b8571d2b15a29637f56a51b0da560072945/redis/asyncio/connection.py#L1379-L1538

I'm guessing we can iterate over the connections ourselves and verify they have all been closed.

Yep, there were a lot of breaking changes in 2.x which is why we never migrated.

That said, the fix here should be to use pools instead. This library has basically reimplemented its own pool manager.

This was done in #296 — will be in 4.0. 4.0.0b2 is available for testing now.

	async def create_conn(self, loop):
	# One connection per pool since we are emulating a single connection
	kwargs = {"minsize": 1, "maxsize": 1, **self.host}
	if not (sys.version_info >= (3, 8, 0) and AIOREDIS_VERSION >= (1, 3, 1)):
	kwargs["loop"] = loop
	if self.master_name is None:
	return await aioredis.create_redis_pool(**kwargs)

migrate aioredis to v2.0