Unpin DeepSpeed to allow 0.13.0 and greater
arnavgarg1 opened this issue · comments
Arnav Garg commented
DeepSpeed 0.13.0 was released on Jan 20th, 2024, and is to be causing our tests to fail with the following error: https://github.com/ludwig-ai/ludwig/actions/runs/7614778487/job/20737857305?pr=3901
Collecting deepspeed!=0.11.0 (from ludwig==0.9.dev0)
Downloading deepspeed-0.13.0.tar.gz (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 92.8 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [38 lines of output]
Warning: RNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
[WARNING] unable to import torch, please install it if you want to pre-compile any deepspeed ops.
Traceback (most recent call last):
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/builder.py", line 14, in <module>
from op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
ImportError: cannot import name 'TORCH_MAJOR' from 'op_builder.builder' (/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/builder.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 480, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 37, in <module>
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/__init__.py", line 48, in <module>
module = importlib.import_module(f".{module_name}", package=op_builder_dir)
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/__init__.py", line 6, in <module>
from .cpu_adam import CPUAdamBuilder
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/cpu_adam.py", line 6, in <module>
from .builder import SYCLOpBuilder
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/builder.py", line 16, in <module>
from deepspeed.ops.op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/deepspeed/__init__.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Error: Process completed with exit code 1.
Deepspeed was pinned to <0.13.0 in #3906