sqs-to-dc should support `--no-sign-request` like s3-to-dc for s3 file load
pindge opened this issue · comments
Pin Jin commented
Encountered an error when trying to setup a test case for handle_bucket_notification_message
where s3 object
cannot load without a proper aws access secret key and id
, s3-to-dc
has --no-sign-request
as an option, this option should be adapted in sqs-to-dc
as well
try:
s3 = boto3.resource("s3")
obj = s3.Object(bucket_name, key).get(ResponseCacheControl="no-cache")
data = safe_load(obj["Body"].read())
uri = f"s3://{bucket_name}/{key}"
except Exception as e:
> raise IndexingException(
f"Exception thrown when trying to load s3 object: {e}"
)
E odc.apps.dc_tools.utils.IndexingException: Exception thrown when trying to load s3 object: An error occurred (InvalidAccessKeyId) when calling the GetObject operation: The AWS Access Key Id you provided does not exist in our records.