Signed urls generated by aiobotocore.signers.AioRequestSigner rejected by AWS

[Morgan Hoban]

Describe the bug
Expected: Signed URL used as a password for AWS elasticache, in the same manner as botocore's request signer, with aiobotocore successfully authenticates.

Observed: Using aiobotocore's signer returns AuthenticationError: invalid username-password pair or user is disabled.

The following code using a botocore signer authenticates and runs successfully:

import asyncio
from boto3.session import Session
import botocore
from botocore.signers import RequestSigner
from redis.asyncio import StrictRedis

region = {fill-in-region-here}
service = "elasticache"
sts_token_expires_in = 900
redis_cluster_name = {fill-in-cluster-name-here}
redis_endpoint = {fill-in-redis-endpoint-here}
redis_user = {fill-in-redis-user-here}

session = Session()

client = session.client(service, region_name=region)
service_id = client.meta.service_model.service_id

params = {
    "method": "GET",
    "url": f"http://{redis_cluster_name}",
    "body": {
        "Action": "connect",
        "User": redis_user
    },
    "headers": {},
    "context": {}
}

signer = RequestSigner(
    #service_id,
    botocore.model.ServiceId("ElastiCache"),
    region,
    service,
    "v4",
    session.get_credentials(),
    session.events
)

signed_url = signer.generate_presigned_url(
    params,
    region_name=region,
    expires_in=sts_token_expires_in,
    operation_name=service
)

redis_credential = signed_url.replace("http://", "")

redis = StrictRedis(host=redis_endpoint, port=6379, ssl=True, username=redis_user, password=redis_credential)

await redis.ping()

On the same host using the same IAM role, the following code fails to authenticate:

import asyncio
import aiobotocore
import botocore
from redis.asyncio import StrictRedis
from aiobotocore.session import AioSession
from aiobotocore.signers import AioRequestSigner

service = "elasticache"
redis_cluster_name = {insert-cluster-name-here}
redis_endpoint = {insert-endpoint-here}
redis_user = {insert-redis-user-here}
region = {insert-region-here}

params = {
    "method": "GET",
    "url": f"http://{redis_cluster_name}",
    "body": {
        "Action": "connect",
        "User": redis_user
    },
    "headers": {},
    "context": {}
}

async def get_pre_signed_url():
    emitter = aiobotocore.hooks.AioHierarchicalEmitter()
    session = aiobotocore.session.AioSession(event_hooks=emitter)
    credentials = await session.get_credentials()
    region = region
    endpoint = {}
    signer = AioRequestSigner(
        service_id=botocore.model.ServiceId("ElastiCache"),
        credentials=credentials,
        region_name=region,
        signing_name="elasticache",
        signature_version="v4",
        event_emitter=emitter
    )
    url = await signer.generate_presigned_url(
        params,
        expires_in=3600,
        operation_name=service
    )
    return url

async def ping_redis_cache():
    signed_url = await get_pre_signed_url()
    redis_credential = signed_url.replace("http://", "")
    # Connect to the Redis cache using the pre-signed URL as the password
    redis = StrictRedis(host=redis_endpoint, port=6379, ssl=True, username=redis_user, password=redis_credential)
    response = await redis.ping()

loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(ping_redis_cache())

Checklist

• I have reproduced in environment where pip check passes without errors
• I have provided pip freeze results
• I have provided sample code or detailed way to reproduce
• I have tried the same code in botocore to ensure this is an aiobotocore specific issue
• I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
• I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection

pip freeze results

Environment:

• Python Version: python 3.10
• OS name and version: Debian Bullseye

Additional context
Add any other context about the problem here.

[Jakob Keller]

Hello and thanks for reporting!

Your example is fairly complex and I am struggling to reproduce the issue on my end. In general, resigned URLs are supported by aiobotocore and I just confirmed this locally for the S3 service.

I noticed two deviations in your example that could be to blame:

• you're not passing the region parameter to await signer.generate_presigned_url()
• you're passing different values for expires_in to signer.generate_presigned_url()

Could you please try to align those and try again?

[Morgan Hoban]

Aligned the values, turned out the issue was with the expires_in value. Any value greater than 900 causes both methods to fail, and value 900 or less and they both succeed.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html#PresignedUrl-Expiration says that

A presigned URL remains valid for the period of time specified when the URL is generated. If you create a presigned URL with the Amazon S3 console, the expiration time can be set between 1 minute and 12 hours. If you use the AWS CLI or AWS SDKs, the expiration time can be set as high as 7 days.

If you created a presigned URL by using a temporary token, then the URL expires when the token expires, even if you created the URL with a later expiration time. For more information about how the credentials you use affect the expiration time, see [Who can create a presigned URL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html#who-presigned-url).

yet it is looking like the observed behavior is that the presigned URL is being rejected if its expiry is longer than the temporary token used to generate it... literally not what the docs say.

But thats an AWS issue not a aiobotocore one. Thanks @jakob-keller for the help!