AttributeError: __aenter__ on 2.5.3
rdbisme opened this issue · comments
Describe the bug
2.5.3 version breaks some code we have with AttributeError: __aenter__. Reverting to 2.5.2 makes the error to vanish. The interesting bit is that we can reproduce this only on docker. Running the same code locally doesn't break.
Checklist
- I have reproduced in environment where
pip checkpasses without errors - I have provided
pip freezeresults - I have provided sample code or detailed way to reproduce
- I have tried the same code in botocore to ensure this is an aiobotocore specific issue
- I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
- I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection
pip freeze results
aiobotocore==2.5.3
aiohttp==3.8.5
aioitertools==0.11.0
aiosignal==1.3.1
async-timeout==4.0.2
attrs==23.1.0
behave==1.2.6
behave-pandas==0.5.0
botocore==1.31.17
Bottleneck==1.3.7
cachetools==5.3.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
codetiming==1.4.0
contexttimer==0.3.3
cryptography==38.0.4
decorator==5.1.1
deprecation==2.1.0
docstring-parser==0.15
exchange-calendars==4.2.8
frozenlist==1.4.0
fsspec==2023.6.0
idna==3.4
janus==1.0.0
jmespath==1.0.1
json-logging==1.2.11
korean-lunar-calendar==0.3.1
methodtools==0.4.7
mmh3==3.1.0
msgpack==1.0.5
multidict==6.0.4
mypy-extensions==1.0.0
networkx==2.8.8
numpy==1.25.2
packaging==23.1
pandas==2.0.3
parse==1.19.1
parse-type==0.6.2
pause==0.3
plotly==5.15.0
polars==0.16.18
psutil==5.9.5
py==1.11.0
pyarrow==12.0.1
pycparser==2.21
pydantic==1.10.12
pyluach==2.2.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0.1
requests==2.31.0
retry==0.9.2
s3fs==2023.6.0
six==1.16.0
tabulate==0.9.0
tenacity==8.2.2
threadpoolctl==3.2.0
toml==0.10.2
toolz==0.12.0
tqdm==4.65.0
typed-argument-parser==1.8.1
typing-inspect==0.9.0
typing_extensions==4.7.1
tzdata==2023.3
universal-pathlib==0.1.0
urllib3==1.26.16
uvloop==0.17.0
websockets==11.0.3
wirerope==0.4.7
wrapt==1.15.0
yarl==1.9.2
Environment:
- Python Version: 3.10
- OS name and version:
Linux b6fd2ef3b383 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Thu Jan 12 16:32:13 UTC 2023 x86_64 GNU/Linux(docker)
Additional context
Traceback
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/xxx/common.py", line 216, in load_json
with file as f:
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/core.py", line 102, in __enter__
f = self.fs.open(self.path, mode=mode)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1241, in open
f = self._open(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 659, in _open
return S3File(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 2066, in __init__
super().__init__(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1597, in __init__
self.size = self.details["size"]
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1610, in details
self._details = self.fs.info(self.path)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 121, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 106, in sync
raise return_result
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 61, in _runner
result[0] = await coro
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 1271, in _info
out = await self._call_s3(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 348, in _call_s3
return await _error_wrapper(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 140, in _error_wrapper
raise err
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 113, in _error_wrapper
return await func(*args, **kwargs)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/client.py", line 366, in _make_api_call
http, parsed_response = await self._make_request(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/client.py", line 391, in _make_request
return await self._endpoint.make_request(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/endpoint.py", line 100, in _send_request
while await self._needs_retry(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/endpoint.py", line 262, in _needs_retry
responses = await self._event_emitter.emit(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/hooks.py", line 66, in _emit
response = await resolve_awaitable(handler(**kwargs))
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/_helpers.py", line 15, in resolve_awaitable
return await obj
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/utils.py", line 414, in redirect_from_error
new_region = await self.get_bucket_region(bucket, response)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/utils.py", line 483, in get_bucket_region
async with self._client as client:
AttributeError: __aenter__
self._client is a weakproxy.
-> async with self._client as client:
(Pdb) p self._client
<weakproxy at 0x7084ddca9530 to S3 at 0x7084de125690>
(Pdb) dir(self._client)
['_PY_TO_OP_NAME', '__aenter__', '__aexit__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_async_getattr', '_cache', '_client_config', '_convert_to_request_dict', '_emit_api_params', '_endpoint', '_exceptions', '_exceptions_factory', '_get_waiter_config', '_load_exceptions', '_loader', '_make_api_call', '_make_request', '_register_handlers', '_request_signer', '_resolve_endpoint_ruleset', '_response_parser', '_ruleset_resolver', '_serializer', '_service_model', '_user_agent_creator', 'abort_multipart_upload', 'can_paginate', 'close', 'complete_multipart_upload', 'copy_object', 'create_bucket', 'create_multipart_upload', 'delete_bucket', 'delete_bucket_analytics_configuration', 'delete_bucket_cors', 'delete_bucket_encryption', 'delete_bucket_intelligent_tiering_configuration', 'delete_bucket_inventory_configuration', 'delete_bucket_lifecycle', 'delete_bucket_metrics_configuration', 'delete_bucket_ownership_controls', 'delete_bucket_policy', 'delete_bucket_replication', 'delete_bucket_tagging', 'delete_bucket_website', 'delete_object', 'delete_object_tagging', 'delete_objects', 'delete_public_access_block', 'exceptions', 'generate_presigned_post', 'generate_presigned_url', 'get_bucket_accelerate_configuration', 'get_bucket_acl', 'get_bucket_analytics_configuration', 'get_bucket_cors', 'get_bucket_encryption', 'get_bucket_intelligent_tiering_configuration', 'get_bucket_inventory_configuration', 'get_bucket_lifecycle', 'get_bucket_lifecycle_configuration', 'get_bucket_location', 'get_bucket_logging', 'get_bucket_metrics_configuration', 'get_bucket_notification', 'get_bucket_notification_configuration', 'get_bucket_ownership_controls', 'get_bucket_policy', 'get_bucket_policy_status', 'get_bucket_replication', 'get_bucket_request_payment', 'get_bucket_tagging', 'get_bucket_versioning', 'get_bucket_website', 'get_object', 'get_object_acl', 'get_object_attributes', 'get_object_legal_hold', 'get_object_lock_configuration', 'get_object_retention', 'get_object_tagging', 'get_object_torrent', 'get_paginator', 'get_public_access_block', 'get_waiter', 'head_bucket', 'head_object', 'list_bucket_analytics_configurations', 'list_bucket_intelligent_tiering_configurations', 'list_bucket_inventory_configurations', 'list_bucket_metrics_configurations', 'list_buckets', 'list_multipart_uploads', 'list_object_versions', 'list_objects', 'list_objects_v2', 'list_parts', 'meta', 'put_bucket_accelerate_configuration', 'put_bucket_acl', 'put_bucket_analytics_configuration', 'put_bucket_cors', 'put_bucket_encryption', 'put_bucket_intelligent_tiering_configuration', 'put_bucket_inventory_configuration', 'put_bucket_lifecycle', 'put_bucket_lifecycle_configuration', 'put_bucket_logging', 'put_bucket_metrics_configuration', 'put_bucket_notification', 'put_bucket_notification_configuration', 'put_bucket_ownership_controls', 'put_bucket_policy', 'put_bucket_replication', 'put_bucket_request_payment', 'put_bucket_tagging', 'put_bucket_versioning', 'put_bucket_website', 'put_object', 'put_object_acl', 'put_object_legal_hold', 'put_object_lock_configuration', 'put_object_retention', 'put_object_tagging', 'put_public_access_block', 'restore_object', 'select_object_content', 'upload_part', 'upload_part_copy', 'waiter_names', 'write_get_object_response']
__aenter__ is well there so I don't understand what's happening here.
I can reproduce like this:
import fsspec
_f = fsspec.open("s3://<some file on s3>")
with _f:
print("a")
Traceback (most recent call last):
File "/tmp/repro.py", line 5, in <module>
with _f:
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/core.py", line 102, in __enter__
f = self.fs.open(self.path, mode=mode)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1241, in open
f = self._open(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 659, in _open
return S3File(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 2066, in __init__
super().__init__(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1597, in __init__
self.size = self.details["size"]
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/spec.py", line 1610, in details
self._details = self.fs.info(self.path)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 121, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 106, in sync
raise return_result
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/fsspec/asyn.py", line 61, in _runner
result[0] = await coro
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 1271, in _info
out = await self._call_s3(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 348, in _call_s3
return await _error_wrapper(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 140, in _error_wrapper
raise err
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/s3fs/core.py", line 113, in _error_wrapper
return await func(*args, **kwargs)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/client.py", line 366, in _make_api_call
http, parsed_response = await self._make_request(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/client.py", line 391, in _make_request
return await self._endpoint.make_request(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/endpoint.py", line 100, in _send_request
while await self._needs_retry(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/endpoint.py", line 262, in _needs_retry
responses = await self._event_emitter.emit(
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/hooks.py", line 66, in _emit
response = await resolve_awaitable(handler(**kwargs))
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/_helpers.py", line 15, in resolve_awaitable
return await obj
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/utils.py", line 414, in redirect_from_error
new_region = await self.get_bucket_region(bucket, response)
File "/root/.local/pipx/venvs/xxx/lib/python3.10/site-packages/aiobotocore/utils.py", line 483, in get_bucket_region
async with self._client as client:
AttributeError: __aenter__. Did you mean: '_details'?
pd.read_parquet throws similar error with s3fs==2023.6.0 and aiobotocore==2.5.3. It works fine when the aiobotocore version is pinned to 2.5.2.
/venv/lib/python3.11/site-packages/pandas/io/parquet.py:509: in read_parquet
return impl.read(
/venv/lib/python3.11/site-packages/pandas/io/parquet.py:227: in read
pa_table = self.api.parquet.read_table(
/venv/lib/python3.11/site-packages/pyarrow/parquet/core.py:2939: in read_table
dataset = _ParquetDatasetV2(
/venv/lib/python3.11/site-packages/pyarrow/parquet/core.py:2465: in __init__
finfo = filesystem.get_file_info(path_or_paths)
pyarrow/_fs.pyx:571: in pyarrow._fs.FileSystem.get_file_info
???
pyarrow/error.pxi:144: in pyarrow.lib.pyarrow_internal_check_status
???
pyarrow/_fs.pyx:1490: in pyarrow._fs._cb_get_file_info
???
/venv/lib/python3.11/site-packages/pyarrow/fs.py:332: in get_file_info
info = self.fs.info(path)
/venv/lib/python3.11/site-packages/fsspec/asyn.py:121: in wrapper
return sync(self.loop, func, *args, **kwargs)
/venv/lib/python3.11/site-packages/fsspec/asyn.py:106: in sync
raise return_result
/venv/lib/python3.11/site-packages/fsspec/asyn.py:61: in _runner
result[0] = await coro
/venv/lib/python3.11/site-packages/s3fs/core.py:1271: in _info
out = await self._call_s3(
/venv/lib/python3.11/site-packages/s3fs/core.py:348: in _call_s3
return await _error_wrapper(
/venv/lib/python3.11/site-packages/s3fs/core.py:140: in _error_wrapper
raise err
/venv/lib/python3.11/site-packages/s3fs/core.py:113: in _error_wrapper
return await func(*args, **kwargs)
/venv/lib/python3.11/site-packages/aiobotocore/client.py:366: in _make_api_call
http, parsed_response = await self._make_request(
/venv/lib/python3.11/site-packages/aiobotocore/client.py:391: in _make_request
return await self._endpoint.make_request(
/venv/lib/python3.11/site-packages/aiobotocore/endpoint.py:100: in _send_request
while await self._needs_retry(
/venv/lib/python3.11/site-packages/aiobotocore/endpoint.py:262: in _needs_retry
responses = await self._event_emitter.emit(
/venv/lib/python3.11/site-packages/aiobotocore/hooks.py:66: in _emit
response = await resolve_awaitable(handler(**kwargs))
/venv/lib/python3.11/site-packages/aiobotocore/_helpers.py:15: in resolve_awaitable
return await obj
/venv/lib/python3.11/site-packages/aiobotocore/utils.py:414: in redirect_from_error
new_region = await self.get_bucket_region(bucket, response)
/venv/lib/python3.11/site-packages/aiobotocore/utils.py:483: in get_bucket_region
async with self._client as client:
E TypeError: 'weakref.ProxyType' object does not support the asynchronous context manager protocol
@shiv-planetrics does this happen on Docker or you can reproduce locally? Which version of Python (minor included?)
thanks will look asap, looks like the change to fix a bug may have introduced one instead
@rdbisme It's happening in Docker environment. I didn't test outside Docker though.
I am watching here to see if we understand the problem. Happy to make any change required in s3fs to make it work with latest aiobotocore in time for our next release. I haven't had a chance to look into it yet myself.
we can reproduce this only on docker
That's really weird! With exactly the same environment?
I am watching here to see if we understand the problem. Happy to make any change required in s3fs to make it work with latest aiobotocore in time for our next release. I haven't had a chance to look into it yet myself.
we can reproduce this only on docker
That's really weird! With exactly the same environment?
Yes. Exactly the same. I did a diff of the pip freeze. The docker is running on Amazon Linux, while my local test was on Rocky 8. Kernel version was the same (except the patch version).
repro'd, yea this has something to do with weakref, digging
ends up that change wasn't needed! reverting and adding cmt. New build should be out shortly
sorry about the noise!
Thanks for the quick fix and reactivity @thehesiod