KeyError: 'error' when creating client/getting object
smandava98 opened this issue · comments
Describe the bug
packages/botocore/exceptions.py", line 41, in init
msg = self.fmt.format(**kwargs)
KeyError: 'error'
this happens sporadically. Below is my code:
def get_boto3_client():
if not hasattr(thread_local_storage, 's3_client'):
Create a new boto3 session for this thread if it doesn't already exist
session = boto3.session.Session(
aws_access_key_id=os.getenv('CLOUDFLARE_ACCESS_KEY_ID'),
aws_secret_access_key=os.getenv('CLOUDFLARE_SECRET_ACCESS_KEY'),
region_name=os.getenv('CLOUDFLARE_REGION_NAME')
)
Create and store the s3 client in the thread's local storage
thread_local_storage.s3_client = session.client(
's3',
endpoint_url=os.getenv('CLOUDFLARE_ENDPOINT_URL')
)
return thread_local_storage.s3_client
def getitem(self, index):
client = get_boto3_client()
logging.info("Loaded boto3 client..")
file_name = self.data[index].strip()
response = client.get_object(Bucket='training-data', Key=file_name)
file_content = response['Body'].read()
buffer = io.BytesIO(file_content)
sample = torch.load(buffer)
I am streaming data to train an ML model via boto3 from cloudflare. I do not know how I get this error:
KeyError: 'error'
Can someone please help? I don't get the endpoint_resolver key errors as I do believe my client creation is handled in a thread safe manner but that error key error I have no idea...
Expected Behavior
No key error
Current Behavior
Key error
Reproduction Steps
See above code
Possible Solution
No response
Additional Information/Context
No response
SDK version used
1.34.1
Environment details (OS name and version, etc.)
MacOS Sonoma
Hi @smandava98, thanks for reaching out. Closing this in favor of boto/boto3#4034.
This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.