Incompatibility with boto3 S3 put_object

[Dave Randall]

I'm trying to use httpretty to simulate attempting to put an object in an S3 bucket without having appropriate permissions to do so.

Here is a truncated version of what I'm doing:

import boto3
import httpretty

httpretty.enable()
httpretty.register_uri(
    httpretty.PUT,
    "https://foo-bucket.s3.amazonaws.com/foo-object",
    body="""<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
    <RequestId>foo</RequestId>
    <HostId>foo</HostId>
</Error>""",
    status=403
)

session = boto3.Session(aws_access_key_id="foo", aws_secret_access_key="foo")
s3_client = session.client('s3')
s3_client.put_object(Bucket="foo-bucket", Key="foo-object", Body=b"foo")

Expected result
The code should raise a botocore.exceptions.ClientError exception.

Actual result
The code raises ValueError: flush of closed file. Full traceback:

Traceback (most recent call last):
  File "/home/dave/.cache/pypoetry/virtualenvs/bucket-dir-yXDMZxoY-py3.7/lib/python3.7/site-packages/botocore/httpsession.py", line 323, in send
    chunked=self._chunked(request.headers),
  File "/home/dave/.cache/pypoetry/virtualenvs/bucket-dir-yXDMZxoY-py3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/home/dave/.cache/pypoetry/virtualenvs/bucket-dir-yXDMZxoY-py3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/dave/.cache/pypoetry/virtualenvs/bucket-dir-yXDMZxoY-py3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/home/dave/git/helpers/pyenv/versions/3.7.4/lib/python3.7/http/client.py", line 1352, in getresponse
    response.close()
  File "/home/dave/git/helpers/pyenv/versions/3.7.4/lib/python3.7/http/client.py", line 415, in close
    super().close() # set "closed" flag
  File "/home/dave/git/helpers/pyenv/versions/3.7.4/lib/python3.7/http/client.py", line 428, in flush
    self.fp.flush()
ValueError: flush of closed file

My assumption is that botocore is attempting to flush the socket, but httpretty has already closed it.

Note: It's possible to workaround this problem by registering a 100 response before the 403 response, but this does not accurateively represent the actual behaviour of interacting the S3 API.

[Gabriel Falcão]

@daveygit2050 thanks for reporting this and already providing such neat snippet, I'll copy it into a functional test and try to fix the issue soon.

[Gabriel Falcão]

Fixed in the release 1.1.1

[Dave Randall]

Thanks @gabrielfalcao!