Error in executing deepwave in MAC

Question

Error in executing deepwave in MAC

makeabhishek opened this issue a year ago · comments

Hi Alan,
I'm trying to install deepwave on mac Apple M1 max, but getting this issue.
I installed the openmp and ninja also

brew install libomp

brew install ninja  
~ % ninja --version 
1.11.1.git.kitware.jobserver-1

Here is a list of gcc

~ % brew list gcc               
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-c++-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-g++-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-gcc-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-gcc-ar-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-gcc-nm-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-gcc-ranlib-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/aarch64-apple-darwin21-gfortran-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/c++-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/cpp-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/g++-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcc-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcc-ar-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcc-nm-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcc-ranlib-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcov-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcov-dump-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gcov-tool-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/gfortran
/opt/homebrew/Cellar/gcc/12.2.0/bin/gfortran-12
/opt/homebrew/Cellar/gcc/12.2.0/bin/lto-dump-12
/opt/homebrew/Cellar/gcc/12.2.0/include/c++/ (806 files)
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/ (607 files)
/opt/homebrew/Cellar/gcc/12.2.0/libexec/gcc/ (14 files)
/opt/homebrew/Cellar/gcc/12.2.0/share/gcc-12/ (4 files)
/opt/homebrew/Cellar/gcc/12.2.0/share/man/ (11 files)

ERROR:

---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
/opt/anaconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py in _run_ninja_build(build_directory, verbose, error_prefix)
   1894         stdout_fileno = 1
-> 1895         subprocess.run(
   1896             command,

/opt/anaconda3/lib/python3.9/subprocess.py in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    527         if check and retcode:
--> 528             raise CalledProcessError(retcode, process.args,
    529                                      output=stdout, stderr=stderr)

CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
/var/folders/2j/x6g1k4r535g3v5v0jfyf7hxm00171x/T/ipykernel_9835/3344772009.py in <module>
      1 import torch
----> 2 import deepwave
      3 import matplotlib.pyplot as plt
      4 
      5 v = 1500 * torch.ones(100, 100)

/opt/anaconda3/lib/python3.9/site-packages/deepwave/__init__.py in <module>
     36     extra_cflags = ['-march=native', '-Ofast', '-fopenmp']
     37     extra_ldflags = ['-march=native', '-Ofast', '-fopenmp']
---> 38 torch.utils.cpp_extension.load(
     39     name="deepwave",
     40     sources=sources,

/opt/anaconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py in load(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   1282         ...     verbose=True)
   1283     '''
-> 1284     return _jit_compile(
   1285         name,
   1286         [sources] if isinstance(sources, str) else sources,

/opt/anaconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py in _jit_compile(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   1507                         sources = list(hipified_sources)
   1508 
-> 1509                     _write_ninja_file_and_build_library(
   1510                         name=name,
   1511                         sources=sources,

/opt/anaconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py in _write_ninja_file_and_build_library(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_standalone)
   1622     if verbose:
   1623         print(f'Building extension module {name}...', file=sys.stderr)
-> 1624     _run_ninja_build(
   1625         build_directory,
   1626         verbose,

/opt/anaconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py in _run_ninja_build(build_directory, verbose, error_prefix)
   1909         if hasattr(error, 'output') and error.output:  # type: ignore[union-attr]
   1910             message += f": {error.output.decode(*SUBPROCESS_DECODE_ARGS)}"  # type: ignore[union-attr]
-> 1911         raise RuntimeError(message) from e
   1912 
   1913 

RuntimeError: Error building extension 'deepwave': [1/4] c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
FAILED: scalar_born.o 
c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
clang: error: unsupported option '-fopenmp'
[2/4] c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/scalar.cpp -o scalar.o 
FAILED: scalar.o 
c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/scalar.cpp -o scalar.o 
clang: error: unsupported option '-fopenmp'
[3/4] c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/elastic.cpp -o elastic.o 
FAILED: elastic.o 
c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/TH -isystem /opt/anaconda3/lib/python3.9/site-packages/torch/include/THC -isystem /opt/anaconda3/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/lib/python3.9/site-packages/deepwave/elastic.cpp -o elastic.o 
clang: error: unsupported option '-fopenmp'
ninja: build stopped: subcommand failed.

Alan Richardson · Answer 1 · Thu Apr 27 2023 02:42:03 GMT+0800 (China Standard Time)

Hi Abhishek, Thank you for including the full error message. It allows me to see that the problem is, as I immediately suspected when you said it was a Mac, OpenMP. Unfortunately the default compiler that Apple provides apparently does not include OpenMP, for some reason. I in fact successfully helped someone to install Deepwave on a Mac just a few days ago and the process unfortunately turned out to be a little bit more complicated than just installing libomp. We instead had to install a new compiler using `brew install llvm`. You will either need to add the path to the directory that includes this new version of clang++ to the front of your PATH environment variable, or set the CXX environment variable to be the path to the new clang++. The new clang++ will probably be found in `/usr/local/opt/llvm/bin`. We then also had to create a softlink because it seemed that PyTorch was looking for the OpenMP library in the wrong place. If you run the install and import of Deepwave again after installing the new compiler, and report what the new error message is, I can try to work out what softlink needs to be created.

Abhishek · Answer 2 · Sat Apr 29 2023 23:39:31 GMT+0800 (China Standard Time)

Thanks Alan,
I tried to install llvm but it gives me error. here is the detailed installation and error status

`

(base) a@pn ~ % brew install llvm
Warning: No remote 'origin' in /opt/homebrew/Library/Taps/asaini/homebrew-local-make, skipping update!
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
==> Updated Homebrew from aa36104e0 to 4.0.15 (50a699315).
Updated 2 taps (homebrew/core and homebrew/cask).
==> New Formulae
aarch64-elf-gdb                          llvm@15
ada-url                                  mdt
aichat                                   meta-package-manager
aliyunpan                                millet
amber                                    musikcube
ares                                     notify
arjun                                    ntfy
ata                                      opal
blocky                                   openssl@3.0
chatblade                                pgvector
cloudpan189-go                           poke
devcontainer                             renovate
dexter                                   resvg
elfx86exts                               sad
enchive                                  scip
ffmpeg@5                                 streamvbyte
flavours                                 thriftgo
form                                     trurl
gat                                      trust-dns
git-big-picture                          tt
go-feature-flag-relay-proxy              typst
hck                                      victoriametrics
hexer                                    vulkan-extensionlayer
hz                                       vulkan-tools
imessage-exporter                        vulkan-validationlayers
kitex                                    vvdec
ksops                                    vvenc
libansilove                              wazero
libvisual                                wxlua
libvisual-plugins                        zpaqfranz
libvisual-projectm
==> New Casks
active-trader-pro                        logitech-options
alipay-key-tool                          macwhisper
archaeology                              mullvad-browser
beardie                                  nozbe
bluos-controller                         openbb-terminal
cameracontroller                         openrocket
canon-eos-utility                        orbstack
capacities                               orcaslicer
carbide-create                           piclist
corsair-icue                             prolific-pl2303
cursor                                   qmk-toolbox
dehelper                                 reminders-menubar
displaylink                              rewind
droidcam-obs                             ricoh-theta
dymo-label                               segger-jlink
edrawmind                                silicon-labs-vcp-driver
elgato-camera-hub                        sonos
elgato-control-center                    sony-ps-remote-play
elgato-game-capture-hd                   stack
elgato-stream-deck                       steelseries-engine
elgato-video-capture                     steelseries-gg
elgato-wave-link                         synology-drive
focusrite-saffire-mixcontrol             tachidesk-sorayomi
ftdi-vcp-driver                          toshiba-color-mfp
fujitsu-scansnap-home                    tuist
garmin-express                           ubiquiti-unifi-controller
gitkraken-cli                            usmart-trade
gutenprint                               vbrokers
hummingbird                              wch-ch34x-usb-serial-driver
konica-minolta-bizhub-c750i-driver       xtool-creative-space
logi-options-plus                        yubico-authenticator
logitech-camera-settings                 yubico-yubikey-manager
logitech-g-hub                           zed

You have 4 outdated formulae installed.


The 4.0.0 release notes are available on the Homebrew Blog:
  https://brew.sh/blog/4.0.0
The 4.0.15 changelog can be found at:
  https://github.com/Homebrew/brew/releases/tag/4.0.15
==> Fetching dependencies for llvm: sqlite, python@3.11, six and z3
==> Fetching sqlite
==> Downloading https://ghcr.io/v2/homebrew/core/sqlite/manifests/3.41.2
######################################################################### 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/sqlite/blobs/sha256:9cf3cb233b6
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################### 100.0%
==> Fetching python@3.11
==> Downloading https://ghcr.io/v2/homebrew/core/python/3.11/manifests/3.11.3
######################################################################### 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/python/3.11/blobs/sha256:270d1f
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################### 100.0%
==> Fetching six
==> Downloading https://ghcr.io/v2/homebrew/core/six/manifests/1.16.0_3
######################################################################### 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/six/blobs/sha256:0dee50367c6fac
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################### 100.0%
==> Fetching z3
==> Downloading https://ghcr.io/v2/homebrew/core/z3/manifests/4.12.1-1
######################################################################### 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/z3/blobs/sha256:677562f0bb1ab8b
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################### 100.0%
==> Fetching llvm
==> Downloading https://ghcr.io/v2/homebrew/core/llvm/manifests/16.0.2
######################################################################### 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/llvm/blobs/sha256:939aa411b4126
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################### 100.0%
Error: No such file or directory @ dir_s_rmdir - /opt/homebrew/var/homebrew/locks/6c3088fc8ca82f535f9cefab66a2a33c0bb61092b337d9bd0f3bd9219f60ba30--llvm--16.0.2.arm64_monterey.bottle.tar.gz.incomplete.lock`

Alan Richardson · Answer 3 · Sat Apr 29 2023 23:48:48 GMT+0800 (China Standard Time)

I do not use MacOS, so I looked online for the solution. Apparently running `sudo chown -R $(whoami) /usr/local/var/homebrew` might solve it.

Abhishek · Answer 4 · Sun Apr 30 2023 00:08:11 GMT+0800 (China Standard Time)

Ok, thanks. I need to ask the admin to use sudo commands. I'll check that and let you know how it goes.

Thanks

Jayanth Boddu · Answer 5 · Tue May 02 2023 21:46:14 GMT+0800 (China Standard Time)

Hey Alan,
I'm facing the same problem on my Mac M1

I install llvm using Homebrew
Added the path to new clang++ to $PATH : /opt/homebrew/opt/llvm/bin

It still throws an error.

Could you tell me more about the pytorch softlink problem.

Here's my setup and error thrown

MacOS : 13.0.1
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ] on darwin
deepwave : 0.0.18

Traceback (most recent call last):
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
    subprocess.run(
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/__init__.py", line 38, in <module>
    torch.utils.cpp_extension.load(
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
    return _jit_compile(
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1509, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1624, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'deepwave': [1/4] c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/scalar.cpp -o scalar.o 
FAILED: scalar.o 
c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/scalar.cpp -o scalar.o 
clang: error: unsupported option '-fopenmp'
[2/4] c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
FAILED: scalar_born.o 
c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
clang: error: unsupported option '-fopenmp'
[3/4] c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/elastic.cpp -o elastic.o 
FAILED: elastic.o 
c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /Users/jayanthboddu/miniforge3/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /Users/jayanthboddu/miniforge3/envs/sva/lib/python3.10/site-packages/deepwave/elastic.cpp -o elastic.o 
clang: error: unsupported option '-fopenmp'
ninja: build stopped: subcommand failed.```

Thanks so much.

Alan Richardson · Answer 6 · Tue May 02 2023 22:16:52 GMT+0800 (China Standard Time)

Hello Jayanth, Thank you for trying Deepwave, and for reporting the problem you encountered. I hope to have a version of Deepwave available soon that does not require compiling C++ and CUDA code, and so eliminate installation problems like this. It looks like PyTorch might be still using the original compiler rather than the new one that you installed. Did you add the path to the new compiler at the end of your PATH environment variable or at the beginning? It needs to be at the beginning (so something like `export PATH=<path to new one>:$PATH`, depending on which shell you use) so that it will find that one before the original one. If you did put it at the beginning, then the problem might be caused by PyTorch looking for `c++` rather than `clang++`, and perhaps the new compiler doesn't softlink one to the other. You could either resolve this by creating a softlink from c++ to clang++ in the directory where your new compiler is, or you could specify the CXX environment variable to point to the new compiler (the clang++ executable itself, and not just the directory it is in, as in PATH). Let me know how it goes! I expect there will be at least one more error to overcome before it works (the one where we will need to create a softlink to the OpenMP library).

Jayanth Boddu · Answer 7 · Wed May 03 2023 00:35:42 GMT+0800 (China Standard Time)

Hey Alan

Softlinking c++ to clang++ worked for me.
I was able to run the 'Marmousi forward modelling' example.

I'm posting my entire setup in case it helps someone.
shell : zsh

arch -arm64 brew install llvm
arch -arm64 brew install libomp
arch -arm64 brew install ninja

export PATH=/opt/homebrew/opt/llvm/bin:$PATH

ln -s /opt/homebrew/opt/llvm/bin/clang++ c++

conda create --name <env name> python=3.10
conda activate <env name>
conda install numpy 
conda install pytorch::pytorch torchvision torchaudio -c pytorch
pip install deepwave

When running the forward modelling example, you could set the following environment variable if you want to use 'mps' backend , else it throws an NotImplemented error for some operations.

import os 
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"]="1"

Alan Richardson · Answer 8 · Wed May 03 2023 01:49:59 GMT+0800 (China Standard Time)

I am very glad to hear that you got it working, and grateful to you for posting the steps - that was thoughtful of you and I hope that it will help others (I will add a link to this post in the documentation to make it easier for them to find it). It looks like you encountered several other obstacles on the way. Congratulations for solving them. Unfortunately the precise steps necessary seem to vary from one system to another - you don't appear to have encountered the issue that required creating a softlink to the OpenMP library, for example - but hopefully this will at least get MacOS users most of the way to a working installation. Please let me know if you encounter any other problems or have any questions.

Jayanth Boddu · Answer 9 · Tue May 09 2023 21:09:34 GMT+0800 (China Standard Time)

Hey Alan,
I've been trying to install deepwave on a workstation running ubuntu 22.04.
It throws an error executing _run_ninja_build originating from running the command ninja -v.
My setup is as follows

Python 3.10.11 (main, Apr 20 2023, 19:02:41) [GCC 11.2.0] on linux ( 'build essential' tools have been installed )
Conda env : 23.3.1
Pytorch : 2.0.1 installed with cudatoolkit 11.8
nvcc : Cuda compilation tools, release 11.5, V11.5.119
CUDA version : 12.1 ( from nvidia-smi report )

The actual error


Traceback (most recent call last):
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
    subprocess.run(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/__init__.py", line 38, in <module>
    torch.utils.cpp_extension.load(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
    return _jit_compile(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1509, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1624, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'deepwave': [1/4] /usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/scalar.cu -o scalar.cuda.o 
FAILED: scalar.cuda.o 
/usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/scalar.cu -o scalar.cuda.o 
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
[2/4] /usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/scalar_born.cu -o scalar_born.cuda.o 
FAILED: scalar_born.cuda.o 
/usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/scalar_born.cu -o scalar_born.cuda.o 
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
[3/4] /usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/elastic.cu -o elastic.cuda.o 
FAILED: elastic.cuda.o 
/usr/bin/nvcc  -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/TH -isystem /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/include/THC -isystem /home/jayanth/.conda/envs/sva/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52 --compiler-options '-fPIC' --restrict -O3 --use_fast_math -std=c++17 -c /home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/elastic.cu -o elastic.cuda.o 
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

The output of executing ninja -v in the shell throws the following error

ninja: error: loading 'build.ninja': No such file or directory

while executing ninja --version works. Shows

1.10.2

I even tried modifying '['ninja', '-v']' to '['ninja', '--version']'.
This results in a different error when importing deepwave.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/deepwave/__init__.py", line 38, in <module>
    torch.utils.cpp_extension.load(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
    return _jit_compile(
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1535, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1934, in _import_module_from_library
    torch.ops.load_library(filepath)
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/site-packages/torch/_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "/home/jayanth/.conda/envs/sva/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/jayanth/.cache/torch_extensions/py310_cu118/deepwave/deepwave.so: cannot open shared object file: No such file or directory

Please help. Thanks in advance.

Alan Richardson · Answer 10 · Tue May 09 2023 21:35:16 GMT+0800 (China Standard Time)

Hi again Jayanth, In my experience Linux tends to be the most reliable operating system to use with Deepwave, but it seems like you found a case where there is a problem. Thank you for reporting it, and I'm sorry that it didn't work. First, although the error message is confusing, I don't think the "ninja -v" part was the problem. The error that you got when you ran that on its own was that it didn't find a build file. When installing Deepwave it will have a build file. Also the "-v" option is short for "--verbose", not "--version", which I think is the cause of the problem you encountered the second time (with your change it just ran ninja --version, printed the Ninja version and returned successfully, without building anything, and so the output of the build, which PyTorch expected, was not there). So, I think the first thing to do is to change it back to "-v". Second, it seems like the actual error that you encountered, "error: parameter packs not expanded with ‘...’", is a known issue. I'm not sure what the actual culprit is, but, from what I have read (such as NVIDIA/nccl#650 and pytorch/pytorch#71518), one solution is to downgrade to GCC 10, and the other is to make a small change to the code in std_function.h (see NVIDIA/nccl#650 (comment)). Are you comfortable with making one of those changes?

Jayanth Boddu · Answer 11 · Wed May 10 2023 14:35:25 GMT+0800 (China Standard Time)

Thanks Alan, I modified the code in std_function.h and it works now.

Alan Richardson · Answer 12 · Wed May 10 2023 18:19:49 GMT+0800 (China Standard Time)

That is good news. I am surprised that they have not fixed that, since, from those Issues that I linked to, it seems that quite a few people have been affected by it for quite a while. I continue to work on investigating how Deepwave could be rewritten to avoid requiring the user to compile the C++/CUDA code, and so avoid these problems in the future.

Abhishek · Answer 13 · Fri May 12 2023 12:53:43 GMT+0800 (China Standard Time)

I tried the mentioned steps, but getting the error.


Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.

---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1893, in _run_ninja_build(build_directory, verbose, error_prefix)
   1892     stdout_fileno = 1
-> 1893     subprocess.run(
   1894         command,
   1895         stdout=stdout_fileno if verbose else subprocess.PIPE,
   1896         stderr=subprocess.STDOUT,
   1897         cwd=build_directory,
   1898         check=True,
   1899         env=env)
   1900 except subprocess.CalledProcessError as e:
   1901     # Python 2 and 3 compatible way of getting the error object.

File /opt/anaconda3/envs/deepwave/lib/python3.10/subprocess.py:526, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    525     if check and retcode:
--> 526         raise CalledProcessError(retcode, process.args,
    527                                  output=stdout, stderr=stderr)
    528 return CompletedProcess(process.args, retcode, stdout, stderr)

CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[1], line 2
      1 import torch
----> 2 import deepwave
      3 import matplotlib.pyplot as plt
      5 v = 1500 * torch.ones(100, 100)

File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/__init__.py:38
     36     extra_cflags = ['-march=native', '-Ofast', '-fopenmp']
     37     extra_ldflags = ['-march=native', '-Ofast', '-fopenmp']
---> 38 torch.utils.cpp_extension.load(
     39     name="deepwave",
     40     sources=sources,
     41     is_python_module=False,
     42     extra_cflags=extra_cflags,
     43     extra_cuda_cflags=['--restrict', '-O3', '--use_fast_math'],
     44     extra_ldflags=extra_ldflags
     45 )

File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1284, in load(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   1192 def load(name,
   1193          sources: Union[str, List[str]],
   1194          extra_cflags=None,
   (...)
   1202          is_standalone=False,
   1203          keep_intermediates=True):
   1204     r'''
   1205     Loads a PyTorch C++ extension just-in-time (JIT).
   1206 
   (...)
   1282         ...     verbose=True)
   1283     '''
-> 1284     return _jit_compile(
   1285         name,
   1286         [sources] if isinstance(sources, str) else sources,
   1287         extra_cflags,
   1288         extra_cuda_cflags,
   1289         extra_ldflags,
   1290         extra_include_paths,
   1291         build_directory or _get_build_directory(name, verbose),
   1292         verbose,
   1293         with_cuda,
   1294         is_python_module,
   1295         is_standalone,
   1296         keep_intermediates=keep_intermediates)

File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1509, in _jit_compile(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   1505                 hipified_sources.add(hipify_result[s_abs]["hipified_path"] if s_abs in hipify_result else s_abs)
   1507             sources = list(hipified_sources)
-> 1509         _write_ninja_file_and_build_library(
   1510             name=name,
   1511             sources=sources,
   1512             extra_cflags=extra_cflags or [],
   1513             extra_cuda_cflags=extra_cuda_cflags or [],
   1514             extra_ldflags=extra_ldflags or [],
   1515             extra_include_paths=extra_include_paths or [],
   1516             build_directory=build_directory,
   1517             verbose=verbose,
   1518             with_cuda=with_cuda,
   1519             is_standalone=is_standalone)
   1520 finally:
   1521     baton.release()

File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1624, in _write_ninja_file_and_build_library(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_standalone)
   1622 if verbose:
   1623     print(f'Building extension module {name}...', file=sys.stderr)
-> 1624 _run_ninja_build(
   1625     build_directory,
   1626     verbose,
   1627     error_prefix=f"Error building extension '{name}'")

File /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1909, in _run_ninja_build(build_directory, verbose, error_prefix)
   1907 if hasattr(error, 'output') and error.output:  # type: ignore[union-attr]
   1908     message += f": {error.output.decode(*SUBPROCESS_DECODE_ARGS)}"  # type: ignore[union-attr]
-> 1909 raise RuntimeError(message) from e

RuntimeError: Error building extension 'deepwave': [1/4] c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
FAILED: scalar_born.o 
c++ -MMD -MF scalar_born.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/scalar_born.cpp -o scalar_born.o 
clang: error: unsupported option '-fopenmp'
[2/4] c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/scalar.cpp -o scalar.o 
FAILED: scalar.o 
c++ -MMD -MF scalar.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/scalar.cpp -o scalar.o 
clang: error: unsupported option '-fopenmp'
[3/4] c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/elastic.cpp -o elastic.o 
FAILED: elastic.o 
c++ -MMD -MF elastic.o.d -DTORCH_EXTENSION_NAME=deepwave -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/TH -isystem /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/torch/include/THC -isystem /opt/anaconda3/envs/deepwave/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -Ofast -fopenmp -c /opt/anaconda3/envs/deepwave/lib/python3.10/site-packages/deepwave/elastic.cpp -o elastic.o 
clang: error: unsupported option '-fopenmp'
ninja: build stopped: subcommand failed.

ninja version

(deepwave) asaini@pn2301151 ~ % ninja --version
1.11.1.git.kitware.jobserver-1

seems like common error as in Linux

Alan Richardson · Answer 14 · Fri May 12 2023 15:27:12 GMT+0800 (China Standard Time)

It appears to be still trying to use the old compiler, as the error you got is that it does not support OpenMP. I hope to have a pure Python version of Deepwave working by the end of the weekend, so I suggest that you wait and use that instead.

Alan Richardson · Answer 15 · Mon May 15 2023 03:59:37 GMT+0800 (China Standard Time)

A version of Deepwave with a pure Python scalar propagator can be installed using pip install git+https://github.com/ar4/deepwave.git@jit. I hope to add the other propagators over the next few days.

Abhishek · Answer 16 · Mon May 15 2023 07:29:36 GMT+0800 (China Standard Time)

pip install git+https://github.com/ar4/deepwave.git@jit

Thanks Alan for putting your efforts and time for the community. Really appreciate it. I successfully installed the python verison of deepwave using the command listed above. Its in one shot and tested successfully.

Alan Richardson · Answer 17 · Thu Jul 27 2023 20:00:21 GMT+0800 (China Standard Time)

I am going to close this Issue as I hope that the new release of Deepwave solves this problem by providing precompiled code, but please feel free to reopen it if there are any problems.