No module named 'torch_int'

Question

No module named 'torch_int'

kaust2018 opened this issue a year ago · comments

Traceback (most recent call last):
File "examples/export_int8_model.py", line 10, in
from smoothquant.opt import Int8OPTForCausalLM
File "", line 259, in load_module
File "/root/anaconda3/envs/smoothquant/lib/python3.8/site-packages/smoothquant-0.0.0-py3.8.egg/smoothquant/opt.py", line 15, in
ModuleNotFoundError: No module named 'torch_int'

amazingkmy · Answer 1 · Wed May 24 2023 16:55:05 GMT+0800 (China Standard Time)

install torch-int
https://github.com/Guangxuan-Xiao/torch-int

Klaus Chen · Answer 2 · Wed May 24 2023 18:25:47 GMT+0800 (China Standard Time)

install torch-int https://github.com/Guangxuan-Xiao/torch-int

Thanks for pointing this out, but it only works in Cuda 11.3 environment? Failed to run bash build_cutlass.sh when I was using cuda12.1

amazingkmy · Answer 3 · Sat May 27 2023 22:16:38 GMT+0800 (China Standard Time)

@kaust2018
check this repo
https://github.com/NVIDIA/cutlass

I didn't know that.. maybe you can do this

hahahahaha · Answer 4 · Fri Jul 21 2023 16:06:58 GMT+0800 (China Standard Time)

@amazingkmy I found that I can't install the "torch_int" code library. Do you have a solution? If you do, I would be extremely grateful.
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/root/paddlejob/suzaolin/torch-int-main/setup.py", line 2, in
from torch.utils import cpp_extension
File "/opt/conda/envs/smoothquant/lib/python3.8/site-packages/torch/init.py", line 778, in
_C._initExtension(manager_path())
AttributeError: partially initialized module 'torch' has no attribute 'UntypedStorage' (most likely due to a circular import)

Klaus Chen · Answer 5 · Fri Jul 21 2023 19:16:30 GMT+0800 (China Standard Time)

@amazingkmy I found that I can't install the "torch_int" code library. Do you have a solution? If you do, I would be extremely grateful. × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [8 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/root/paddlejob/suzaolin/torch-int-main/setup.py", line 2, in from torch.utils import cpp_extension File "/opt/conda/envs/smoothquant/lib/python3.8/site-packages/torch/init.py", line 778, in _C._initExtension(manager_path()) AttributeError: partially initialized module 'torch' has no attribute 'UntypedStorage' (most likely due to a circular import)

Use docker to install exactly the libraries and cuda version as stated in the repo.

WelY1 · Answer 6 · Thu Aug 24 2023 17:35:36 GMT+0800 (China Standard Time)

Hi, have you successfully run smoothquant_opt_real_int8_demo.ipynb ? I encountered some mistakes with cutlass.
look at the details of error please.

/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [219,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [220,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [221,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [222,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [223,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [96,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [97,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [98,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [99,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [100,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [101,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [102,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [103,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [104,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.
/home/l50024761/llm/torch-int/submodules/cutlass/include/cutlass/arch/memory_sm75.h:208: void cutlass::arch::ldsm(cutlass::Array<unsigned int, MatrixCount, true> &, const void *) [with Layout = cutlass::layout::RowMajor; int MatrixCount = 4]: block: [0,9,0], thread: [105,0,0] Assertion `0 && __PRETTY_FUNCTION__` failed.