Failure to install on mac

Question

Failure to install on mac

srkaeppler opened this issue a month ago · comments

The documentation has gotten a lot better on installation. I decided to attempt to install on mac using brew and venv. I realize this is not supported, but I have wanted to stay away from conda because I heard that it is hard to remove from your computer, although I may go that route. I was able to get completions of:
make -j=8 and make pygimli J=8
However, when I test the code by loading the code, I get the following error:

(gimli) srkaeppler@Stephens-MBP-2 build % make check
[ 95%] Built target gimli
[ 95%] Building CXX object core/tests/unittest/CMakeFiles/gimliUnitTest.dir/unitTest.cpp.o
In file included from /Users/srkaeppler/research/data/pygimli/source/core/tests/unittest/unitTest.cpp:11:
/Users/srkaeppler/research/data/pygimli/source/core/tests/unittest/testVector.h:468:17: warning: unused variable '_AT' [-Wunused-variable]
        double *_AT = new double[k*m];
                ^
1 warning generated.
[100%] Linking CXX executable ../../../bin/gimliUnitTest
[100%] Built target gimliUnitTest
Test project /Users/srkaeppler/research/data/pygimli/source/build
    Start 1: gtest:gimli_unit_tests
1/2 Test #1: gtest:gimli_unit_tests ...........***Failed    0.47 sec
    Start 2: pgtest:pygimli_tests
2/2 Test #2: pgtest:pygimli_tests .............***Failed    0.06 sec

0% tests passed, 2 tests failed out of 2

Total Test time (real) =   0.53 sec

The following tests FAILED:
	  1 - gtest:gimli_unit_tests (Failed)
	  2 - pgtest:pygimli_tests (Failed)
Errors while running CTest
Output from these tests are in: /Users/srkaeppler/research/data/pygimli/source/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make[3]: *** [CMakeFiles/check] Error 8
make[2]: *** [CMakeFiles/check.dir/all] Error 2
make[1]: *** [CMakeFiles/check.dir/rule] Error 2
make: *** [check] Error 2

Start testing: May 29 14:57 EDT
----------------------------------------------------------
1/2 Testing: gtest:gimli_unit_tests
1/2 Test: gtest:gimli_unit_tests
Command: "/usr/bin/make" "gtest"
Directory: /Users/srkaeppler/research/data/pygimli/source/build/core/tests/unittest
"gtest:gimli_unit_tests" start time: May 29 14:57 EDT
Output:
----------------------------------------------------------
[ 95%] Built target gimli
[100%] Built target gimliUnitTest
dyld[41987]: missing symbol called
/bin/sh: line 1: 41987 Abort trap: 6           ../../../bin/gimliUnitTest
make[7]: *** [gtest] Error 134
make[6]: *** [core/tests/unittest/CMakeFiles/gtest.dir/all] Error 2
make[5]: *** [core/tests/unittest/CMakeFiles/gtest.dir/rule] Error 2
make[4]: *** [core/tests/unittest/CMakeFiles/gtest.dir/rule] Error 2
<end of output>
Test time =   0.47 sec
----------------------------------------------------------
Test Failed.
"gtest:gimli_unit_tests" end time: May 29 14:57 EDT
"gtest:gimli_unit_tests" time elapsed: 00:00:00
----------------------------------------------------------

2/2 Testing: pgtest:pygimli_tests
2/2 Test: pgtest:pygimli_tests
Command: "/usr/bin/make" "pgtest"
Directory: /Users/srkaeppler/research/data/pygimli/source/build/core/python
"pgtest:pygimli_tests" start time: May 29 14:57 EDT
Output:
----------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pygimli'
make[7]: *** [pgtest] Error 1
make[6]: *** [core/python/CMakeFiles/pgtest.dir/all] Error 2
make[5]: *** [core/python/CMakeFiles/pgtest.dir/rule] Error 2
make[4]: *** [core/python/CMakeFiles/pgtest.dir/rule] Error 2
<end of output>
Test time =   0.06 sec
----------------------------------------------------------
Test Failed.
"pgtest:pygimli_tests" end time: May 29 14:57 EDT
"pgtest:pygimli_tests" time elapsed: 00:00:00
----------------------------------------------------------

End testing: May 29 14:57 EDT

(gimli) srkaeppler@Stephens-MBP-2 source % python3 -c 'import pygimli'
Traceback (most recent call last):
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/core.py", line 11, in <module>
    from . import _pygimli_  # if it works: as pgcore, replace all _pygimli_
    ^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/srkaeppler/research/data/pygimli/source/pygimli/core/_pygimli_.so, 0x0002): symbol not found in flat namespace '_openblas_set_num_threads'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/__init__.py", line 8, in <module>
    from .core.decorators import (renamed, singleton, moduleProperty,
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/__init__.py", line 11, in <module>
    from .core import pgcore
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/core.py", line 19, in <module>
    import pgcore
ModuleNotFoundError: No module named 'pgcore'

I am not sure what the issue is.

My brew and enviornment installation includes the following packages:

(gimli) srkaeppler@Stephens-MacBook-Pro-2 build % brew list
==> Formulae
boost			gsl			mercurial		python@3.11
boost-python3		hwloc			metis			python@3.12
ca-certificates		icu4c			mpdecimal		readline
cmake			isl			mpfr			sqlite
fftw			libevent		open-mpi		suite-sparse
gcc			libidn2			openblas		triangle
gdbm			libmpc			openssl@3		wget
gettext			libomp			pcre2			xz
git			libunistring		pmix			zstd
gmp			lz4			python-gdbm@3.12

(gimli) srkaeppler@Stephens-MacBook-Pro-2 build % pip freeze
contourpy==1.2.1
cycler==0.12.1
fonttools==4.52.4
kiwisolver==1.4.5
matplotlib==3.9.0
numpy==1.26.4
packaging==24.0
pillow==10.3.0
pygccxml==2.5.0
pyparsing==3.1.2
pyplusplus==1.8.5
python-dateutil==2.9.0.post0
scipy==1.13.1
setuptools==70.0.0
six==1.16.0

Here is the full output in this file: https://www.dropbox.com/scl/fi/7antaqtoafg3ra3tsmgms/gimli-srkaeppler-Stephens-MBP-2-pygiml.txt?rlkey=rdlr69962pl34qwibilcjay22&dl=0

Florian Wagner · Answer 1 · Thu Jun 06 2024 22:45:53 GMT+0800 (China Standard Time)

Dear @srkaeppler,

not sure why the test target fails, but the installation generally seems to have worked.

Can you check if the following works

cd /Users/srkaeppler/research/data/pygimli/source/
python -c "import pygimli; print(pygimli.__version__)"

If yes, you probably just need to add the path to your .bashrc.

Best wishes
Florian

Steve Kaeppler · Answer 2 · Fri Jun 07 2024 19:44:06 GMT+0800 (China Standard Time)

@florian-wagner :

srkaeppler@Stephens-MacBook-Pro-2 foo % source ~srkaeppler/Env3/gimli/bin/activate
(gimli) srkaeppler@Stephens-MacBook-Pro-2 foo % cd ~srkaeppler/research/data/pygimli/source 
(gimli) srkaeppler@Stephens-MacBook-Pro-2 source % ls
AUTHORS.rst              INSTALLATION.rst         apps                     environment.yml
CHANGELOG.md             ISSUE_TEMPLATE.md        build                    pygimli
CITATION.cff             MANIFEST.in              core                     pygimli.egg-info
CITATION.rst             PULL_REQUEST_TEMPLATE.md dependencies.rst         setup.cfg
CMakeLists.txt           README.md                dev_requirements.txt     setup.py
CONTRIBUTING.md          README.rst               dist                     versioneer.py
COPYING.rst              __pycache__              doc
(gimli) srkaeppler@Stephens-MacBook-Pro-2 source % python -c "import pygimli; print(pygimli.__version__)"
Traceback (most recent call last):
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/core.py", line 11, in <module>
    from . import _pygimli_  # if it works: as pgcore, replace all _pygimli_
    ^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/srkaeppler/research/data/pygimli/source/pygimli/core/_pygimli_.so, 0x0002): symbol not found in flat namespace '_openblas_set_num_threads'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/__init__.py", line 8, in <module>
    from .core.decorators import (renamed, singleton, moduleProperty,
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/__init__.py", line 11, in <module>
    from .core import pgcore
  File "/Users/srkaeppler/research/data/pygimli/source/pygimli/core/core.py", line 19, in <module>
    import pgcore
ModuleNotFoundError: No module named 'pgcore'
(gimli) srkaeppler@Stephens-MacBook-Pro-2 source %

I also very much appreciate the help. I have debated whether to just use conda and bite the bullet. Brady got me onto this code and I have a particular nonlinear regularization problem I am wondering if this code could help with, so I am especially interested in the regularization notebooks, etc.

carsten-forty2 · Answer 3 · Sat Jun 08 2024 19:03:48 GMT+0800 (China Standard Time)

After some recent changes on the build toolchain its good to remove the CMakeCache.txt in the build path or even try a new clean build.

You could also give the wheel branch a try. Its on the same state like the dev branch plus a test for a build-chain for whl generation using venv. Note, this branch is not yet tested for mac and there might be some minor tweeks necessary.

In the new branch are new make targets which should produce valid whl files for local pip installation.

make whlpgcore

make whlpygimli

Both create there respective whl files and test them against a local test venv. The final whl's you can find in build/wheelhouse. If there is something wrong maybe the cmake output give some hints.

Steve Kaeppler · Answer 4 · Sat Jun 08 2024 20:37:49 GMT+0800 (China Standard Time)

@carsten-forty2 @florian-wagner
Thanks for the response. I see two potential actions here and let me outline the proposed procedure.

One:

Reclone pygimli?
rebuild following the procedure outlined above.
I am not really sure this will work unless some sort of flag has been changed in the Cmake files. This is something I am not familiar with writing. Put another way, I could basically do this right now and I have tried this by deleting the build folder and unless some other files have changed (I could git pull), I don't see why I would get a different outcome

Two:

Reclone gimli
git checkout wheel to get onto the wheel branch.
make -j=8
make whlpgcore
make whlpygimli
make pygimli J=8
Is this the procedure?

One last thing, how do I build or make pgcore maybe that is what is screwing up...

Steve Kaeppler · Answer 5 · Sat Jun 08 2024 21:21:34 GMT+0800 (China Standard Time)

@carsten-forty2 @florian-wagner
The other issue I am still concerned about is that the .so file above has some missing tags. I am not sure if this is something you need to set as a compiler option or some sort of .bashrc environment variable. I am setting some of this stuff in .zshrc since I am using a mac with a M3 chip. I have not extensively worked with openblas or any of those packages, so any help there would be appreciated.

Thank you both for helping me! Once we get this working, I am happy to post this to you in some form and you could potentially add it to your website.

Steve Kaeppler · Answer 6 · Mon Jun 10 2024 22:54:13 GMT+0800 (China Standard Time)

Just recloned this morning and getting new errors during the make install -j8 step:

  file INSTALL cannot find
  "/Users/srkaeppler/research/data/pygimli/source/build/gimli_export.h": No
  such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:42 (include)