Earlier versions of CUDA still report an error
YH-2023 opened this issue · comments
Processing /home/hxd/work4/work/kernels/selective_scan
Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from selective-scan==0.0.2) (1.7.0)
Requirement already satisfied: packaging in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from selective-scan==0.0.2) (24.0)
Requirement already satisfied: ninja in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from selective-scan==0.0.2) (1.11.1.1)
Requirement already satisfied: einops in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from selective-scan==0.0.2) (0.7.0)
Requirement already satisfied: future in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from torch->selective-scan==0.0.2) (1.0.0)
Requirement already satisfied: typing_extensions in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from torch->selective-scan==0.0.2) (4.9.0)
Requirement already satisfied: dataclasses in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from torch->selective-scan==0.0.2) (0.6)
Requirement already satisfied: numpy in /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages (from torch->selective-scan==0.0.2) (1.24.4)
Building wheels for collected packages: selective-scan
Building wheel for selective-scan (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [132 lines of output]
torch.__version__ = 1.7.0
CUDA_HOME = /home/hxd/cuda11.0.3
running bdist_wheel
running build
running build_ext
building 'selective_scan_cuda_core' extension
creating /home/hxd/work4/work/kernels/selective_scan/build
creating /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38
creating /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc
creating /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan
creating /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus
Emitting ninja build file /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] /home/hxd/cuda11.6/bin/nvcc -I/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/TH -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/THC -I/home/hxd/cuda11.6/include -I/home/hxd/anaconda3/envs/seg/include/python3.8 -c -c /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_bwd.cu -o /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_bwd.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math --ptxas-options=-v -lineinfo -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=selective_scan_cuda_core -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_bwd.o
/home/hxd/cuda11.6/bin/nvcc -I/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/TH -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/THC -I/home/hxd/cuda11.6/include -I/home/hxd/anaconda3/envs/seg/include/python3.8 -c -c /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_bwd.cu -o /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_bwd.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math --ptxas-options=-v -lineinfo -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=selective_scan_cuda_core -D_GLIBCXX_USE_CXX11_ABI=0
In file included from /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_bwd.cu:4:
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_bwd_kernel.cuh:10:10: fatal error: ATen/cuda/Atomic.cuh: 没有那个文件或目录
10 | #include <ATen/cuda/Atomic.cuh> // For atomicAdd on complex
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_bwd.cu:4:
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_bwd_kernel.cuh:10:10: fatal error: ATen/cuda/Atomic.cuh: 没有那个文件或目录
10 | #include <ATen/cuda/Atomic.cuh> // For atomicAdd on complex
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_bwd.cu:4:
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_bwd_kernel.cuh:10:10: fatal error: ATen/cuda/Atomic.cuh: 没有那个文件或目录
10 | #include <ATen/cuda/Atomic.cuh> // For atomicAdd on complex
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[2/3] /home/hxd/cuda11.6/bin/nvcc -I/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/TH -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/THC -I/home/hxd/cuda11.6/include -I/home/hxd/anaconda3/envs/seg/include/python3.8 -c -c /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_fwd.cu -o /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_fwd.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math --ptxas-options=-v -lineinfo -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=selective_scan_cuda_core -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_fwd.o
/home/hxd/cuda11.6/bin/nvcc -I/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/TH -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/THC -I/home/hxd/cuda11.6/include -I/home/hxd/anaconda3/envs/seg/include/python3.8 -c -c /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_fwd.cu -o /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan_core_fwd.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math --ptxas-options=-v -lineinfo -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=selective_scan_cuda_core -D_GLIBCXX_USE_CXX11_ABI=0
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_fwd_kernel.cuh(176): error: identifier "C10_CUDA_KERNEL_LAUNCH_CHECK" is undefined
detected during instantiation of "void selective_scan_fwd_cuda<knrows,input_t,weight_t>(SSMParamsBase &, cudaStream_t) [with knrows=1, input_t=float, weight_t=float]"
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_fwd.cu(6): here
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_fwd_kernel.cuh(176): error: identifier "C10_CUDA_KERNEL_LAUNCH_CHECK" is undefined
detected during instantiation of "void selective_scan_fwd_cuda<knrows,input_t,weight_t>(SSMParamsBase &, cudaStream_t) [with knrows=1, input_t=float, weight_t=float]"
/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_fwd.cu(6): here
2 errors detected in the compilation of "/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan_core_fwd.cu".
[3/3] c++ -MMD -MF /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan.o.d -pthread -B /home/hxd/anaconda3/envs/seg/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/TH -I/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/THC -I/home/hxd/cuda11.6/include -I/home/hxd/anaconda3/envs/seg/include/python3.8 -c -c /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan.cpp -o /home/hxd/work4/work/kernels/selective_scan/build/temp.linux-x86_64-cpython-38/csrc/selective_scan/cus/selective_scan.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=selective_scan_cuda_core -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/ATen/Parallel.h:149,
from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
from /home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/torch/extension.h:4,
from /home/hxd/work4/work/kernels/selective_scan/csrc/selective_scan/cus/selective_scan.cpp:7:
/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/include/ATen/ParallelOpenMP.h:84: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
84 | #pragma omp parallel for if ((end - begin) >= grain_size)
|
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1516, in _run_ninja_build
subprocess.run(
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/hxd/work4/work/kernels/selective_scan/setup.py", line 140, in <module>
setup(
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
return distutils.core.setup(**attrs)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
super().run_command(command)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 364, in run
self.run_command("build")
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
super().run_command(command)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
super().run_command(command)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 88, in run
_build_ext.run(self)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 653, in build_extensions
build_ext.build_extensions(self)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
self._build_extensions_serial()
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
self.build_extension(ext)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
_build_ext.build_extension(self, ext)
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
objects = self.compiler.compile(
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 473, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1233, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/hxd/anaconda3/envs/seg/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1538, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for selective-scan
Running setup.py clean for selective-scan
Failed to build selective-scan
ERROR: Could not build wheels for selective-scan, which is required to install pyproject.toml-based projects
I did not test cuda version earlier than cuda11.3. What exactly the version of cuda you are using?
@MzeroMiko The version I'm using is cuda11.0.3，How can we solve the problem of installing selective_scan in lower versions?