terricain / aioboto3

Wrapper to use boto3 resources with the aiobotocore async backend

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

'Unclosed connection' warning when using S3 client

maretodoric opened this issue · comments

  • Async AWS SDK for Python version: aioboto3==12.4.0
  • Python version: 3.10.12
  • Operating System: Ubuntu 22.04.2 LTS (Jammy Jellyfish)

Description

I'm receiving warning when using S3 client:

Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

I have following function that is supposed to return information about object stored in S3:

async def get_s3_object(bucket: str, filepath: str):
    cfg = load_settings()
    session  = aioboto3.Session(aws_access_key_id=cfg.aws.key, aws_secret_access_key=cfg.aws.secret_key, region_name=cfg.aws.region)

    async with session.client('s3') as s3:
        try:
            s3_ob = await s3.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return s3_ob

Another function calls get_s3_object and everything works fine, but i get warning mentioned above.

I'm using the client as suggested, by creating a session then using the .client or .resource as context manager, but it appears as if it's not closing the S3 connection

Can you replicate this using aiobotocore alone? I have a suspicion its something to do with how the connection pool will keep connections open to speed up subsequent requests, whilst it's annoying I dont have much time to work on fixing it as it shouldn't affect operation.