s3 client put_object fails due to x-amz-content-sha256

[Oleg Korsak]

• Async AWS SDK for Python version: 10.1.0
• Python version: 3.11
• Operating System: Ubuntu 22.04.1 LTS

Description

Upload dictionary, converted to json to s3 bucket.

What I Did

import asyncio
import json

import aioboto3
from botocore.config import Config


async def test():
    body = json.dumps({'aaa': 'bbb'})
    key = f'test.json'
    session = aioboto3.Session(
        region_name='eu-north-1',
        aws_access_key_id='xxx',
        aws_secret_access_key='xxx',
    )
    client = session.client(
        's3',
        region_name='eu-north-1',
        aws_access_key_id='xxx',
        aws_secret_access_key='xxx',
        config=Config(signature_version='v4'),
    )

    async with client as s3:
        await s3.put_object(Body=body.encode(), Bucket='bucket-name', Key=key)


if __name__ == '__main__':
    asyncio.run(test())

Fails with:

/home/test_s3.py 
/home/venv/lib/python3.11/site-packages/botocore/utils.py:1720: FutureWarning: The S3RegionRedirector class has been deprecated for a new internal replacement. A future version of botocore may remove this class.
  warnings.warn(
Traceback (most recent call last):
  File "/home/test_s3.py", line 30, in <module>
    asyncio.run(test())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/test_s3.py", line 26, in test
    await s3.put_object(Body=body.encode(), Bucket='bucket-name', Key=key)
  File "/home/venv/lib/python3.11/site-packages/aiobotocore/client.py", line 358, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidRequest) when calling the PutObject operation: Missing required header for this request: x-amz-content-sha256

Removing config= leads to:

botocore.exceptions.ClientError: An error occurred (MalformedXML) when calling the PutObject operation: The XML you provided was not well-formed or did not validate against our published schema

[Oleg Korsak]

it seems that malformed XML issue was due to broken dependencies in venv. I had awscli installed which messed with different versions. But still... missing header case is still here.

[Terri Cain]

I wouldn't recommend specifying the signature manually, in this case you've specified the wrong one. Try using s3v4 instead.