sub.fetch when connection to server lost : UnexpectedEOF not properly raised

[romainb-met]

What version were you using?

nats-py==2.2.0
nats-server: nats:2.9.14-alpine (running in a k8s pod)
CLI : v0.0.35
Python 3.8

What environment was the server running in?

Kubernetes pod.

Is this defect reproducible?

Given one has a NATS server available and can connect to it yes by using the following script :

import nats
from nats.errors import TimeoutError as NATSTimeoutError
from nats.errors import UnexpectedEOF
import sys

import asyncio

NATS_URL = ""
NATS_PORT = ""
NATS_USER= "user"
NATS_PASSWORD = "password"
STREAM = "stream"
CONSUMER = "consumer"
SUBJECT = "subject"
BATCH_SIZE = 5
NATS_JS_TIMEOUT = 5
NATS_CONNECTION_TIMEOUT = 30

async def handle_nats_error(err:Exception):

    if isinstance(err, UnexpectedEOF):
        raise UnexpectedEOF("NATS connection closed unexpectedly.")
    elif isinstance(err, ConnectionRefusedError):
        raise ConnectionRefusedError("NATS connection to server refused.")

async def create_nats_connection() -> nats.aio.client.Client:
    """
    Create and return a NATS client connection to the specified NATS server.

    Returns
    -------
    nats.aio.client.Client
        A NATS client connection object that can be used to interact with the NATS server.
    """
    # Connect to NATS
    nats_url = f"nats://{NATS_URL}:{NATS_PORT}"
    return await nats.connect(
        servers=nats_url,
        user=NATS_USER,
        password=NATS_PASSWORD,
        connect_timeout=NATS_CONNECTION_TIMEOUT,
        error_cb=handle_nats_error,
    )


async def create_js_context() -> nats.js.JetStreamContext:
    """
    Create and return a NATS client connection to the specified NATS server, and a JetSTream context

    Returns
    -------
    nats.js.JetStreamContext
        A NATS JetSTream context object that can be used to interact with the NATS server JetStream
    """
    # Connect to NATS
    nc = await create_nats_connection()
    js = nc.jetstream()
    return nc, js


async def main():

    while True:
        try:
            print("Connecting to NATS")
            nc, js = await create_js_context()
            print("Pull subscribing to NATS")
            psub = await js.pull_subscribe(
                subject=SUBJECT, durable=CONSUMER, stream=STREAM
            )
            print("Fetching messages")
            messages = await psub.fetch(batch=BATCH_SIZE, timeout=NATS_JS_TIMEOUT)
            print(messages)
            print("Closing NATS connection")
            nc.close()
        except NATSTimeoutError:
            print("NATS connection timed out.")
            await asyncio.sleep(5)
            continue # continue to catch new messages
        except UnexpectedEOF:
            print("NATS connection closed unexpectedly => exiting.")
            sys.exit(1)
        except ConnectionRefusedError:
            raise ConnectionRefusedError("NATS connection to server refused.")
        except Exception as e:
            print(f"Error while fetching messages from NATS: {e}")
        await nc.close() if "nc" in locals() else None

if __name__ == "__main__":
    asyncio.run(main())

Given the capability you are leveraging, describe your expectation?

In case the connection of the server is lost during fetch (nats.errors.UnexpectedEOF) I would like for the error to be properly raised so that I can catch it with an Except.

Given the expectation, what is the defect you are observing?

Here my attempt was to catch the UnexpectedEOF generated when I cut the port forward to the server.
The UnexpectedEOF is indeed created, but not raised properly. It should be handled per handle_nats_error and then catch per the except. But here it seems that default error callback is called first somehow. See logs below :

Connecting to NATS
Pull subscribing to NATS
Fetching messages
NATS connection timed out.
Connecting to NATS
Pull subscribing to NATS
Fetching messages
nats: encountered error
Traceback (most recent call last):
  File "/home/user/myrepo/venv/lib/python3.8/site-packages/nats/aio/client.py", line 2030, in _read_loop
    await self._error_cb(err)
  File "/home/user/myrepo/scripts/test_nats_fetch.py", line 22, in handle_nats_error
    raise UnexpectedEOF("NATS connection closed unexpectedly.")
nats.errors.UnexpectedEOF: nats: unexpected EOF
nats: encountered error
Traceback (most recent call last):
  File "/home/user/myrepo/venv/lib/python3.8/site-packages/nats/aio/client.py", line 2030, in _read_loop
    await self._error_cb(err)
  File "/home/user/myrepo/scripts/test_nats_fetch.py", line 22, in handle_nats_error
    raise UnexpectedEOF("NATS connection closed unexpectedly.")
nats.errors.UnexpectedEOF: nats: unexpected EOF
NATS connection timed out.

My expectation is no traceback and no TimeoutError instead of UnexpectedEOF :

Connecting to NATS
Pull subscribing to NATS
Fetching messages
NATS connection timed out.
Connecting to NATS
Pull subscribing to NATS
Fetching messages
NATS connection closed unexpectedly => exiting.

Any ideas on how to reach this behavior ? Or if maybe newer version of the client may fix this ?