getObjectAttributes fails with "We encountered an internal error. Please try again." if key contains chinese characters

[Anders Carlsen]

Describe the bug

In my scenario I upload a file using S3 Client putObject method, key string contains chinese characters.
For example a file named 0103雲端備份訂價表.xlsx.

The upload works fine.

I then want to get attributes of the uploaded file to verify different things using getObjectAttributes - I use the same key as for upload. This fails with the following error.

aws.sdk.kotlin.services.s3.model.S3Exception: We encountered an internal error. Please try again.
at aws.sdk.kotlin.services.s3.serde.GetObjectAttributesOperationDeserializerKt.throwGetObjectAttributesError(GetObjectAttributesOperationDeserializer.kt:81)
at aws.sdk.kotlin.services.s3.serde.GetObjectAttributesOperationDeserializerKt.access$throwGetObjectAttributesError(GetObjectAttributesOperationDeserializer.kt:1)
at aws.sdk.kotlin.services.s3.serde.GetObjectAttributesOperationDeserializerKt$throwGetObjectAttributesError$1.invokeSuspend(Unknown Source:13)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source:1)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source:1)

Expected behavior

I expect getObjectAttributes function call to not fail.

Current behavior

Calling getObjectAttributes works for other keys, but fails consistently for keys with chinese characters (and also possibly other special characters). It works with Java SDK (getObjectMetadata).

Steps to Reproduce

Upload a file to a bucket:

val request = PutObjectRequest {
    bucket = "somebucket"
    key = "0103雲端備份訂價表.xlsx"
    body = file.asByteStream()
}
s3Client.putObject(request)

Try to get object attributes:

val request = GetObjectAttributesRequest {
    bucket = "somebucket"
    key = "0103雲端備份訂價表.xlsx"
    objectAttributes = listOf(
       ObjectAttributes.ObjectSize,
       ObjectAttributes.Checksum,
    )
}
s3Client.getObjectAttributes(request)

Possible Solution

None

Context

I want to get metadata for an object with chinese characters in key.

AWS Kotlin SDK version used

1.0.8

Platform (JVM/JS/Native)

JVM

Operating System and version

Android 14

[Aaron Todd]

Thanks for the bug report, I'm able to recreate this. Interestingly I'm seeing the same thing with the CLI and AWS SDK for Java v2:

CLI

>  aws --version
aws-cli/2.14.5 Python/3.11.6 Darwin/22.6.0 source/x86_64 prompt/off


> aws s3api get-object-attributes --bucket <my-bucket> --key 0103雲端備份訂價表.xlsx --object-attributes ObjectSize
An error occurred (InternalError) when calling the GetObjectAttributes operation (reached max retries: 2): We encountered an internal error. Please try **again.**

---

AWS SDSK for Java v2

    val javaV2Resp = S3Client.create().getObjectAttributes { builder ->
        builder.bucket(bucketName)
            .key(objKey)
            .objectAttributes(software.amazon.awssdk.services.s3.model.ObjectAttributes.OBJECT_SIZE)
            .build()
    }
    println(javaV2Resp)

Exception in thread "main" software.amazon.awssdk.services.s3.model.S3Exception: We encountered an internal error. Please try again. (Service: S3, Status Code: 500, ...)

---

Other things I noticed:

• One request on v0.32.0-beta made it through fine and then failed on subsequent requests
• AWS CLI, AWS SDK for Java v2, and AWS SDK for Kotlin use the same encoding for the object key (0103%E9%9B%B2%E7%AB%AF%E5%82%99%E4%BB%BD%E8%A8%82%E5%83%B9%E8%A1%A8.xlsx) in the URI
• AWS CLI, AWS SDK for Java v2, and AWS SDK for Kotlin are all able to read back the content via GetObject

Given this is happening in multiple SDKs and the CLI I'll be opening a ticket with S3

[Aaron Todd]

internal: V1140391686

[Anders Carlsen]

Just FYI I should have been more clear, I meant it worked in Java SDK for Android - the SDK for Android doesn't seem to have the getObjectAttributes method at all.

[Aaron Todd]

This appears to be confirmed a service side issue but I have no ETA for a fix at the moment.

[Anders Carlsen]

Any workarounds? With the Android Java SDK there is the option to retrieve metadata using s3client.getObjectMetadata().

Is there a similar command in the Kotlin SDK? I can't seem to find it..

[Aaron Todd]

Yeah under the hood that method is just calling the HeadObject operation. I'm not sure why it's renamed in the Android SDK.

val metadata = s3.headObject { 
    bucket = myBucket
    key = myKey
}

println(metadata)

See the Kotlin API ref docs for headObject

[Anders Carlsen]

Thanks, that works fine. :-)