Giters

boto / botocore

The low-level, core functionality of boto3 and the AWS CLI.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Client.head_object raises ClientError instead of NoSuchKey

vlaci opened this issue · comments

commented

Describe the bug

I am not sure if the following issue is an error in documentation or implementation.

The AWS SDK documentation of HeadObject1 states that it signals the NoSuchKey error with an HTTP 404 status code. botocore's documentation2 also mentions S3.Client.exceptions.NoSuchKey exception. This exception however is not raised when using head_object on a non-existing object, instead a generic ClientError is received.

According to my limited understanding of botocore it always parses specific errors from the response body. As HeadObject response does not contain a body, it falls back to use the HTTP status code to construct the exception object.

Expected Behavior

Preferably, Head* and Get* calls should return the same specific (e.g. NoSuchKey) exception when called with the same arguments. I understand, that the exception details won't be present for Head* variants.

Alternatively, the documentation of Head* methods should be changed, mentioning that only the generic ClientError can be raised in this case.

Current Behavior

Client.head_* calls raise generic ClientError instead of endpoint-specific ones based on status code.

Reproduction Steps

# botocore-nosuchkey.py
import botocore.session
import moto


mock = moto.mock_aws()
mock.start()

session = botocore.session.get_session()
client = session.create_client("s3")
client.create_bucket(Bucket="testbucket")

try:
    client.get_object(Bucket="testbucket", Key="nonexistent")
except client.exceptions.ClientError as exc:
    print(repr(exc))

try:
    client.head_object(Bucket="testbucket", Key="nonexistent")
except client.exceptions.ClientError as exc:
    print(repr(exc))
$ python3 botocore-nosuchkey.py
NoSuchKey('An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.')
ClientError('An error occurred (404) when calling the HeadObject operation: Not Found')

Possible Solution

The service description contains the mapping between status code and error code3, so botocore could be able to figure out the specific exception in this case as well.

Additional Information/Context

No response

SDK version used

1.34.89

Environment details (OS name and version, etc.)

NixOS 24.05.20240509.6c8fca0 (Uakari) with Python 3.11.9

Footnotes

  1. https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_Errors ↩

  2. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/head_object.html ↩

  3. https://github.com/boto/botocore/blob/8b29f58b6f09935e5ec83f4adade36e81643d32f/botocore/data/s3/2006-03-01/service-2.json#L772C19-L772C28 and https://github.com/boto/botocore/blob/8b29f58b6f09935e5ec83f4adade36e81643d32f/botocore/data/s3/2006-03-01/service-2.json#L7476C1-L7476C38 ↩

commented

Thanks for reaching out - this has come up a few times before, I'll share the recent comment that I made here on a related issue to provide more context:

...Related issues have come up here a few times before, for example boto/boto3#2499. Both the HeadObject and HeadBucket APIs return 404 errors if the object or bucket does not exist. And as noted in the documentation:

A HEAD request has the same options as a GET operation on an object. The response is identical to the GET response except that there is no response body. Because of this, if the HEAD request generates an error, it returns a generic code, such as 400 Bad Request, 403 Forbidden, 404 Not Found, 405 Method Not Allowed, 412 Precondition Failed, or 304 Not Modified. It's not possible to retrieve the exact exception of these error codes.

So this is the behavior of the underlying S3 API and therefore not something that will be addressed in Boto3. As mentioned in previous issues it would be a breaking change to update the model. But please use the Provide feedback at the bottom of API documentation pages to send any additional feedback directly to the S3 team.

commented

I've been digging a bit further and finally figured out why botocore is unable to handle these cases for HEAD request.

I was initially under the impression that based on the current schema, it is possible for botocore to "recover" the error code from the HTTP status code given that all HTTP status codes can be mapped unambiguously to error codes (per endpoint). Unfortunately, this is not the case. Although there are only 3(!) HEAD requests in the whole API and 2 of them are HeadObject and HeadBucket which has a single error code specified, but Media Store's DescribeObject has 2 error cases1 which result in HTTP 404 status codes (in addition to a bunch of common 400 errors), so my assumption is proven wrong.

Footnotes

  1. https://docs.aws.amazon.com/mediastore/latest/apireference/API_objstore_DescribeObject.html#API_objstore_DescribeObject_Errors ↩