setuptools-70.0.0 doesn't seem to work with ParallelCluster
rmarable-flaretx opened this issue · comments
AWS ParallelCluster 3.9.0 and beyond don't seem to work with the most recently released version of setuptools (70.0.0) on OSX.
❯ pcluster version
Traceback (most recent call last):
File "/opt/homebrew/bin/pcluster", line 5, in <module>
from pcluster.cli.entrypoint import main
File "/opt/homebrew/lib/python3.11/site-packages/pcluster/cli/entrypoint.py", line 32, in <module>
import pcluster.api.controllers.cluster_compute_fleet_controller # noqa: E402
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/pcluster/api/controllers/cluster_compute_fleet_controller.py", line 11, in <module>
from pcluster.api.controllers.common import configure_aws_region, convert_errors, validate_cluster
File "/opt/homebrew/lib/python3.11/site-packages/pcluster/api/controllers/common.py", line 20, in <module>
from pkg_resources import packaging
ImportError: cannot import name 'packaging' from 'pkg_resources' (/opt/homebrew/lib/python3.11/site-packages/pkg_resources/__init__.py)
Revering back to setuptools-69.5.1
seems to fix the issue.
❯ pip install setuptools==69.5.1
Collecting setuptools==69.5.1
Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB)
Using cached setuptools-69.5.1-py3-none-any.whl (894 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 70.0.0
Uninstalling setuptools-70.0.0:
Successfully uninstalled setuptools-70.0.0
Successfully installed setuptools-69.5.1
❯ pcluster version
{
"version": "3.9.0"
}
I didn't test this, but I suspect the same is true for Linux as well.
Hi @rmarable-flaretx ,
It's recommended to install pcluster in a Python virtual env, to avoid conflicts with other Python dependencies.
See instructions here: https://docs.aws.amazon.com/parallelcluster/latest/ug/install-v3-virtual-environment.html
This will solve any kind of library incompatibility issues.
I am using a virtual Python environment. The issue is that the most recent version of setuptools doesn't seem to work; downgrading to 69.5.1 did.
@rmarable-flaretx thanks for the workaround, reverting to setuptools==69.5.1
did the trick for me. Looks like setuptools==70.0.0
deprecated pkg_resources
which is causing the error. https://setuptools.pypa.io/en/latest/pkg_resources.html
I just found that we were aware of this issue, so we pinned setuptools a month ago: #6251 the fix has been included starting from ParallelCluster 3.9.1
Thank you @enrico-usai!