redis / ioredis

🚀 A robust, performance-focused, and full-featured Redis client for Node.js.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

BZPOPMIN will hang for ever when a short disconnection happens and the port is different from 6379

manast opened this issue · comments

This one was pretty weird but quite serious. If you issue a BZPOPMIN command, and there is a small disconnection while the command is blocking, then it will stay blocking forever, effectively hanging the app that uses the command.

Now, one of the insane things about this issue is that to reproduce it you must use a port different than the standard 6379, and also use docker network disconnect, it is not enough to just stop the docker container running Redis:

Try this code:
bug.mjs:

import Redis from "ioredis";

const connection = new Redis({
    host: "localhost",
    port: 6380,
});

connection.on("connect", () => console.log("Redis connected!"));
connection.on("ready", () => console.log("Redis ready!"));
connection.on("error", (err) => console.error("Redis error:", err));
connection.on("end", () => console.log("Redis connection ended"));
connection.on("reconnecting", () => console.log("Redis reconnecting..."));

async function test() {
    console.log("Gonna issue a BZPOPMIN command...")
    const result = await connection.bzpopmin("key", 0)
    console.log("Issued BZPOPMIN command!", result)
}

test().catch(console.error);

docker-compose.yml:

version: "3.7"
services:
  redis:
    image: redis:alpine
    ports:
      - "6380:6379"

Start the docker container with:

docker-compose up

Run the test with:

node bug.mjs

Results in:

Gonna issue a BZPOPMIN command...
Redis connected!
Redis ready!

Now in a different terminal, and depending on where you run the docker (I had my docker compose on directory ioredis-econnreset:

docker network disconnect ioredis-econnreset_default ioredis-econnreset-redis-1
sleep 3
docker network connect ioredis-econnreset_default ioredis-econnreset-redis-1

The Redis network was disconnected and connected again after 3 seconds. Open a Redis cli:

redis-cli -p 6380
> 127.0.0.1:6380> zadd key 10 test
(integer) 1
127.0.0.1:6380>

The terminal with the node app will stay the same, with no error or nothing. If you restart the app and without disconnecting and reconnecting you try again to ZADD to the key you will get instead:

Gonna issue a BZPOPMIN command...
Redis connected!
Redis ready!
Issued BZPOPMIN command! [ 'key', 'test', '10' ]

Let me know if you need more information.

Hi @manast,

I can not reproduce this with ioredis 5.4.1, node 20.14.0, the latest redis alpine version (7.2.5) and docker 26.1.3 (colima on macos).

The terminal with the node app will stay the same, with no error or nothing.

I do get connection errors and reconnecting output as expected when disconnecting
and after a few seconds reconnecting the docker network like you described ( I also tried it with less time between the disconnect and connect docker commands (1 second) and more than 3 seconds >=10 seconds but the result is the same).

I always get the Issued BZPOPMIN command! [ 'key', 'test', '10' ]
output in the apps terminal as expected even without restarting the app.

image