Retry on `ResponseStreamingError` when getting s3 objects.

[Adrian Caruana]

Describe the feature

Include ResponseStreamingError in the list of "standard" errors/exceptions that are retried when getting s3 objects.

Use Case

I'm getting ResponseStreamingError sometimes when doing .get_object with an s3 client. Even when my client is configured with retries, the client does not retry when this exception is raised.

The following code reproduces the error:

import http.server
import logging
import socketserver
import threading
from http import HTTPStatus
from io import BytesIO

import boto3
import botocore.client
from boto3.s3.transfer import TransferConfig


def repro(endpoint: str) -> None:
    session = boto3.Session(aws_access_key_id='', aws_secret_access_key='')
    client = session.client(
        "s3",
        endpoint_url=endpoint,
        config=botocore.client.Config(
            max_pool_connections=1, retries=dict(mode='standard', max_attempts=3)
        ),
    )

    client.get_object(Bucket='bucket', Key='bad')["Body"].read()


class Handler(http.server.BaseHTTPRequestHandler):
    def do_HEAD(self):
        self.send_response(HTTPStatus.OK)
        self.send_header('Content-Length', '10000')
        self.end_headers()

    def do_GET(self):
        self.send_response(HTTPStatus.OK)
        self.send_header('Content-Length', '10000')
        self.end_headers()
        # content length and body do not agree - simulating a connection drop
        self.wfile.write(b'\x00' * 1000)


if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)

    with socketserver.TCPServer(("localhost", 0), Handler) as httpd:
        threading.Thread(target=httpd.serve_forever, daemon=True).start()
        repro(f'http://localhost:{httpd.server_address[1]}')

Proposed Solution

No response

Other Information

The following issue might be related: boto/botocore#3132

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

1.34.51

Environment details (OS name and version, etc.)

Ubuntu, python 3.11.8, urllib3 1.26.18

[Tim Finnigan]

Hi @adriancaruana thanks for reaching out. In the issue you linked (boto/botocore#3132) it looks like this was fixed via boto/s3transfer#301 for urllib3 2+. Could you try updating your versions of Boto3/urllib3? The latest Boto3 version is 1.34.99 per the CHANGELOG. Or if you could at least try testing in 1.34.63 / urllib3 2.2.1+.