unable to build from source - `cannot import name 'packaging' from 'pkg_resources'`
forresti opened this issue · comments
If you do not know the root cause of the problem / bug, and wish someone to help you, please
post according to this template:
🐛 Bugs / Unexpected behaviors
I am on Linux, and I am unable to build the pytorch3d code from source.
Instructions To Reproduce the Issue:
Please include the following (depending on what the issue is):
-
Any changes you made (
git diff) or code you wrote
I did not modify the code. -
The exact command(s) you ran:
conda create -n my_env
conda activate my_env
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d
pip install -e .
- What you observed (including the full logs):
Obtaining file:///home/ubuntu/SyncMVD/pytorch3d
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/ubuntu/SyncMVD/pytorch3d/setup.py", line 17, in <module>
from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension
File "/home/ubuntu/anaconda3/envs/syncmvd2/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
from pkg_resources import packaging # type: ignore[attr-defined]
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/anaconda3/envs/syncmvd2/lib/python3.8/site-packages/pkg_resources/__init__.py)
[end of output]
The problem seems to happen when I use that older version of pytorch.
When I install pytorch with conda install pytorch::pytorch torchvision torchaudio -c pytorch, then pytorch3d builds fine. But, I am using this old PyTorch because I am trying to setup https://github.com/LIU-Yuxin/SyncMVD, which uses this old PyTorch.
Can you update setuptools in the environment?
I have the exact same issue...I have upgraded and downgraded setup tools, package manager, cuda, etc etc....
(models) D:\03 Innovation\pytorch3d>pip install -v -e .
Using pip 24.1.2 from C:\Users\perso\AppData\Roaming\Python\Python310\site-packages\pip (python 3.10)
Obtaining file:///D:/03%20Innovation/pytorch3d
Running command python setup.py egg_info
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "D:\03 Innovation\pytorch3d\setup.py", line 17, in <module>
from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension
File "C:\ProgramData\miniforge3\envs\models\lib\site-packages\torch\utils\cpp_extension.py", line 25, in <module>
from pkg_resources import packaging # type: ignore[attr-defined]
ImportError: cannot import name 'packaging' from 'pkg_resources' (C:\ProgramData\miniforge3\envs\models\lib\site-packages\pkg_resources\__init__.py)
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: 'C:\ProgramData\miniforge3\envs\models\python.exe' -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'D:\\03 Innovation\\pytorch3d\\setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base 'C:\Users\perso\AppData\Local\Temp\pip-pip-egg-info-p6rqbll5'
cwd: D:\03 Innovation\pytorch3d\\
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
@Fanie-Visagie I reduced your sample to the main output. I think we can't help - this looks like a general question of versions of python, setuptools and pytorch. It looks like you are on Windows, which may distinguish it from OP.
Hello I have the same problem trying to build a docker image to run https://github.com/NVlabs/BundleSDF.git. The docker image is this one: dockerfile.
It is failing at installing https://github.com/facebookresearch/pytorch3d.git with the same error as OP:
=> ERROR [13/18] RUN conda init bash && echo "conda activate py38" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 -- 737.3s
[13/18] RUN conda init bash && echo "conda activate py38" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 && pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable" && pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja && conda install -y -c anaconda scipy:
0.252 no change /opt/conda/condabin/conda
0.252 no change /opt/conda/bin/conda
0.252 no change /opt/conda/bin/conda-env
0.252 no change /opt/conda/bin/activate
0.252 no change /opt/conda/bin/deactivate
0.252 no change /opt/conda/etc/profile.d/conda.sh
0.252 no change /opt/conda/etc/fish/conf.d/conda.fish
0.252 no change /opt/conda/shell/condabin/Conda.psm1
0.252 no change /opt/conda/shell/condabin/conda-hook.ps1
0.252 no change /opt/conda/lib/python3.12/site-packages/xontrib/conda.xsh
0.252 no change /opt/conda/etc/profile.d/conda.csh
0.252 modified /root/.bashrc
0.252
0.252 ==> For changes to take effect, close and re-open your current shell. <==
0.252
0.531 Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113
1.573 Collecting torch==1.11.0+cu113
2.072 Downloading https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (1637.0 MB)
683.2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 GB 3.5 MB/s eta 0:00:00
685.3 Collecting torchvision==0.12.0+cu113
685.4 Downloading https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp38-cp38-linux_x86_64.whl (22.3 MB)
689.3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.3/22.3 MB 5.4 MB/s eta 0:00:00
690.0 Collecting torchaudio==0.11.0
690.0 Downloading https://download.pytorch.org/whl/cu113/torchaudio-0.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (2.9 MB)
690.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 6.0 MB/s eta 0:00:00
691.0 Collecting typing-extensions (from torch==1.11.0+cu113)
691.3 Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
692.2 Collecting numpy (from torchvision==0.12.0+cu113)
692.2 Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)
692.7 Collecting requests (from torchvision==0.12.0+cu113)
692.8 Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
693.6 Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.12.0+cu113)
693.7 Downloading pillow-10.4.0-cp38-cp38-manylinux_2_28_x86_64.whl.metadata (9.2 kB)
694.4 Collecting charset-normalizer<4,>=2 (from requests->torchvision==0.12.0+cu113)
694.5 Downloading charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
695.1 Collecting idna<4,>=2.5 (from requests->torchvision==0.12.0+cu113)
695.2 Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
696.1 Collecting urllib3<3,>=1.21.1 (from requests->torchvision==0.12.0+cu113)
696.1 Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
696.8 Collecting certifi>=2017.4.17 (from requests->torchvision==0.12.0+cu113)
696.8 Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
696.9 Downloading pillow-10.4.0-cp38-cp38-manylinux_2_28_x86_64.whl (4.5 MB)
700.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 1.3 MB/s eta 0:00:00
700.5 Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
710.2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 2.3 MB/s eta 0:00:00
710.2 Downloading requests-2.32.3-py3-none-any.whl (64 kB)
710.3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 kB 3.0 MB/s eta 0:00:00
710.3 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
710.4 Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
710.4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.0/163.0 kB 2.8 MB/s eta 0:00:00
710.5 Downloading charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
710.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.1/141.1 kB 2.6 MB/s eta 0:00:00
710.6 Downloading idna-3.7-py3-none-any.whl (66 kB)
710.6 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 2.4 MB/s eta 0:00:00
710.6 Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
710.7 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 3.3 MB/s eta 0:00:00
710.8 Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio
728.1 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
728.1 Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 idna-3.7 numpy-1.24.4 pillow-10.4.0 requests-2.32.3 torch-1.11.0+cu113 torchaudio-0.11.0+cu113 torchvision-0.12.0+cu113 typing-extensions-4.12.2 urllib3-2.2.2
729.6 Collecting git+https://github.com/facebookresearch/pytorch3d.git@stable
729.6 Cloning https://github.com/facebookresearch/pytorch3d.git (to revision stable) to /tmp/pip-req-build-ltyxrweo
729.6 Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/pytorch3d.git /tmp/pip-req-build-ltyxrweo
735.8 Running command git checkout -q 8965341
736.5 Resolved https://github.com/facebookresearch/pytorch3d.git to commit 8965341
736.5 Preparing metadata (setup.py): started
737.1 Preparing metadata (setup.py): finished with status 'error'
737.1 error: subprocess-exited-with-error
737.1
737.1 × python setup.py egg_info did not run successfully.
737.1 │ exit code: 1
737.1 ╰─> [8 lines of output]
737.1 Traceback (most recent call last):
737.1 File "", line 2, in
737.1 File "", line 34, in
737.1 File "/tmp/pip-req-build-ltyxrweo/setup.py", line 17, in
737.1 from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension
737.1 File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 23, in
737.1 from pkg_resources import packaging # type: ignore[attr-defined]
737.1 ImportError: cannot import name 'packaging' from 'pkg_resources' (/opt/conda/envs/py38/lib/python3.8/site-packages/pkg_resources/init.py)
737.1 [end of output]
737.1
737.1 note: This error originates from a subprocess, and is likely not a problem with pip.
737.1 error: metadata-generation-failed
737.1
737.1 × Encountered error while generating package metadata.
737.1 ╰─> See above for output.
737.1
737.1 note: This is an issue with the package mentioned above, not pip.
737.1 hint: See above for details.
3 warnings found (use --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 72)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 87)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 88)
dockerfile:75
74 |
75 | >>> RUN conda init bash &&
76 | >>> echo "conda activate py38" >> ~/.bashrc &&
77 | >>> conda activate py38 &&
78 | >>> pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 &&
79 | >>> pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable" &&
80 | >>> pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja &&
81 | >>> conda install -y -c anaconda scipy
82 |ERROR: failed to solve: process "/bin/bash --login -c conda init bash && echo "conda activate py38" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 && pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable\" && pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja && conda install -y -c anaconda scipy" did not complete successfully: exit code: 1
@misrolox BTW, you will need to have fvcore and iopath installed before the pip install will work, but that isn't the actual problem here. I don't know how to solve your error. Can you try with newer versions of python and pytorch?