ImdsCredentialsProvider does not follow redirects

[Arne Georg Gisnås Gleditsch]

Describe the bug

When running under k8s and using kiam to proxy IAM metadata requests, requests towards /latest/meta-data/iam/security-credentials result in HTTP 301 responses with redirects to /latest/meta-data/iam/security-credentials/ (trailing slash). This is different from the behavior when accessing EC2 metadata directly, where both variants result in HTTP 200. Arguably this should be addressed kiam side, but I don't think any development is done in that project at this point. It seems the Rust SDK has chosen to address this by simply always appending the final slash: awslabs/aws-sdk-rust#560

Expected behavior

Instantiating ImdsCredentialsProvider() should be able to discover the IAM/kiam provided role associated with the running context under k8s/kiam.

Current behavior

When no profileOverride is supplied to ImdsCredentialsProvider, it fails with

Suppressed: aws.smithy.kotlin.runtime.auth.awscredentials.CredentialsProviderException: failed to load instance profile
	at aws.sdk.kotlin.runtime.auth.credentials.ImdsCredentialsProvider.resolve(ImdsCredentialsProvider.kt:83)
	at aws.sdk.kotlin.runtime.auth.credentials.ImdsCredentialsProvider$resolve$1.invokeSuspend(ImdsCredentialsProvider.kt)
	...
Caused by: aws.sdk.kotlin.runtime.config.imds.EC2MetadataError: error retrieving instance metadata
	at aws.sdk.kotlin.runtime.config.imds.ImdsClient$get$op$1$1.deserialize(ImdsClient.kt:117)

Steps to Reproduce

Instantiating ImdsCredentialsProvider under k8s/kiam.

Possible Solution

Append terminating slash to URL used for profile discovery in loadProfile.

Context

No response

AWS Kotlin SDK version used

1.1.1, relevant code appears unchanged up until 1.2.6.

Platform (JVM/JS/Native)

JVM

Operating System and version

Linux 6.1.85