Could not load any ArrayFire libraries

Question

Could not load any ArrayFire libraries

ivan-gusachenko opened this issue 7 years ago · comments

Hello,
I'm sorry I know this issue has been around for a while but I couldn't solve it based on the posted solutions. I'm trying to install it for Mac OSx with cpu and opencl, without cuda. So I write this in the terminal:
export DYLD_LIBRARY_PATH=/usr/local/lib:$DYLD_LIBRARY_PATH
as this is where my libaf*.dylib files are located. Still, when trying to import the module I get this message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ivangusachenko/anaconda/lib/python3.6/site-packages/arrayfire/__init__.py", line 56, in <module>
    from .library    import *
  File "/Users/ivangusachenko/anaconda/lib/python3.6/site-packages/arrayfire/library.py", line 600, in <module>
    backend = _clibrary()
  File "/Users/ivangusachenko/anaconda/lib/python3.6/site-packages/arrayfire/library.py", line 576, in __init__
    raise RuntimeError("Could not load any ArrayFire libraries.\n" + more_info_str)
RuntimeError: Could not load any ArrayFire libraries.
Please look at https://github.com/arrayfire/arrayfire-python/wiki for more information.

Pavan Yalamanchili · Answer 1 · Sat Jul 22 2017 04:20:40 GMT+0800 (China Standard Time)

@ivan-gusachenko Something about conda may be screwing up the DYLD_LIBRARY_PATH. Can you try using arrayfire using regular python?

Ivan Gusachenko · Answer 2 · Sat Jul 22 2017 04:31:17 GMT+0800 (China Standard Time)

Thank you,
I've just tried as you said, but unfortunately this didn't help.

[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import arrayfire as af
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/arrayfire/__init__.py", line 56, in <module>
    from .library    import *
  File "/Library/Python/2.7/site-packages/arrayfire/library.py", line 600, in <module>
    backend = _clibrary()
  File "/Library/Python/2.7/site-packages/arrayfire/library.py", line 576, in __init__
    raise RuntimeError("Could not load any ArrayFire libraries.\n" + more_info_str)
RuntimeError: Could not load any ArrayFire libraries.
Please look at https://github.com/arrayfire/arrayfire-python/wiki for more information.```

My export vars are:
```declare -x Apple_PubSub_Socket_Render="/private/tmp/com.apple.launchd.nYU79VcGLM/Render"
declare -x DYLD_LIBRARY_PATH="/usr/local/lib:"
declare -x HOME="/Users/ivangusachenko"
declare -x LANG="en_GB.UTF-8"
declare -x LOGNAME="ivangusachenko"
declare -x OLDPWD="/Users/ivangusachenko"
declare -x PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
declare -x PWD="/Users/ivangusachenko/Downloads"
declare -x SECURITYSESSIONID="186a7"
declare -x SHELL="/bin/bash"
declare -x SHLVL="1"
declare -x SSH_AUTH_SOCK="/private/tmp/com.apple.launchd.FJ7a2DPHRJ/Listeners"
declare -x TERM="xterm-256color"
declare -x TERM_PROGRAM="Apple_Terminal"
declare -x TERM_PROGRAM_VERSION="388.1.1"
declare -x TERM_SESSION_ID="5DFA14AE-3C10-47BC-97EB-44C32971AEDD"
declare -x TMPDIR="/var/folders/20/8fh___m52b1190hx6_0tkv9m0000gn/T/"
declare -x USER="ivangusachenko"
declare -x XPC_FLAGS="0x0"
declare -x XPC_SERVICE_NAME="0"
declare -x __CF_USER_TEXT_ENCODING="0x1F5:0x0:0x2"```

Pavan Yalamanchili · Answer 3 · Sat Jul 22 2017 04:51:41 GMT+0800 (China Standard Time)

@ivan-gusachenko can you try running otool -L /usr/local/lib/libaf.dylib and showing the output?

Ivan Gusachenko · Answer 4 · Sat Jul 22 2017 05:15:13 GMT+0800 (China Standard Time)

Thanks, it gives:
8afbf601:Downloads ivangusachenko$ otool -L /usr/local/lib/libaf.dylib /usr/local/lib/libaf.dylib: @rpath/libaf.3.dylib (compatibility version 3.0.0, current version 3.5.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

Pavan Yalamanchili · Answer 5 · Sat Jul 22 2017 05:50:41 GMT+0800 (China Standard Time)

@ivan-gusachenko can you run the same command on libafcuda.dylib, libafopencl.dylib and libafcpu.dylib ?

Ivan Gusachenko · Answer 6 · Sat Jul 22 2017 05:55:37 GMT+0800 (China Standard Time)

Here it is. I don't have cuda hardware, I have amd opencl gpu instead.

/usr/local/lib/libafcuda.dylib:
	@rpath/libafcuda.3.dylib (compatibility version 3.0.0, current version 3.5.0)
	@rpath/libcublas.8.0.dylib (compatibility version 0.0.0, current version 8.0.55)
	@rpath/libcufft.8.0.dylib (compatibility version 0.0.0, current version 8.0.55)
	@rpath/libcusparse.8.0.dylib (compatibility version 0.0.0, current version 8.0.55)
	@rpath/libcusolver.8.0.dylib (compatibility version 0.0.0, current version 8.0.55)
	@rpath/libnvrtc.8.0.dylib (compatibility version 0.0.0, current version 8.0.53)
	/usr/local/cuda/lib/libcuda.dylib (compatibility version 1.1.0, current version 8.0.46)
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)```

```8afbf601:Downloads ivangusachenko$ otool -L /usr/local/lib/libafopencl.dylib
/usr/local/lib/libafopencl.dylib:
	@rpath/libafopencl.3.dylib (compatibility version 3.0.0, current version 3.5.0)
	/System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)```

```8afbf601:Downloads ivangusachenko$ otool -L /usr/local/lib/libafcpu.dylib
/usr/local/lib/libafcpu.dylib:
	@rpath/libafcpu.3.dylib (compatibility version 3.0.0, current version 3.5.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)```

Pavan Yalamanchili · Answer 7 · Sat Jul 22 2017 06:39:10 GMT+0800 (China Standard Time)

@ivan-gusachenko how did you install arrayfire ? Do you have libforge present in /usr/local/lib ?

Ivan Gusachenko · Answer 8 · Sat Jul 22 2017 06:59:26 GMT+0800 (China Standard Time)

Big thanks for you guidance, Pavan.
I do have libforge, libforge.1, and libforge.1.0.2 in /usr/local/lib

First, I installed Anaconda.
Then I installed homebrew by doing this:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
then I downloaded binaries for Mac OS from arrayfire.org, and followed the installation procedure.
Then I downloaded a cmake installer package for Mac OS and installed it, but apparently this didn't work. So I also installed cmake from brew:
brew install cmake.
Arrayfire recommended cmake, but I don't think there's connection with python here.

Then, from terminal, I installed arrayfire-python:
pip install arrayfire
Then I tried to make it work by changing the lib path for DYLIB variable, to no avail.
Then, after your advice, I switched to the system preinstalled python distribution by removing conda directory from PATH variable, and then I had to reinstall arrayfire-python by
pip install arrayfire
Every often I check export to make sure I have /usr/local/lib in my DYLIB var.

Pavan Yalamanchili · Answer 9 · Sat Jul 22 2017 07:29:43 GMT+0800 (China Standard Time)

@ivan-gusachenko I think some dependency for libforge is not present on the system ?

Can you run otool -L libforge*

Ivan Gusachenko · Answer 10 · Sat Jul 22 2017 07:30:43 GMT+0800 (China Standard Time)

libforge.1.0.2.dylib:
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib (compatibility version 11.0.0, current version 11.2.0)
	/usr/local/lib/libglfw.dylib (compatibility version 3.0.0, current version 3.2.0)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/opt/X11/lib/libSM.6.dylib (compatibility version 7.0.0, current version 7.1.0)
	/opt/X11/lib/libICE.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current version 11.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
libforge.1.dylib:
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib (compatibility version 11.0.0, current version 11.2.0)
	/usr/local/lib/libglfw.dylib (compatibility version 3.0.0, current version 3.2.0)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/opt/X11/lib/libSM.6.dylib (compatibility version 7.0.0, current version 7.1.0)
	/opt/X11/lib/libICE.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current version 11.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
libforge.dylib:
	@rpath/libforge.1.dylib (compatibility version 1.0.0, current version 1.0.2)
	/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib (compatibility version 11.0.0, current version 11.2.0)
	/usr/local/lib/libglfw.dylib (compatibility version 3.0.0, current version 3.2.0)
	/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
	/opt/X11/lib/libSM.6.dylib (compatibility version 7.0.0, current version 7.1.0)
	/opt/X11/lib/libICE.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current version 11.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)```

Ivan Gusachenko · Answer 11 · Sat Jul 22 2017 07:31:23 GMT+0800 (China Standard Time)

sorry I don't know how to preserve the original formatting when pasting

Pavan Yalamanchili · Answer 12 · Sat Jul 22 2017 07:32:06 GMT+0800 (China Standard Time)

@ivan-gusachenko I have been editing your comments to fix the formatting. You are using a single tick (used for quoting) instead of three ticks (used for showing code).

Pavan Yalamanchili · Answer 13 · Sat Jul 22 2017 07:36:11 GMT+0800 (China Standard Time)

@ivan-gusachenko can you uninstall the current version of arrayfire and try installing the following:

http://ci.arrayfire.org/userContent/OSX/ArrayFire-no-gl-v3.5.0_OSX.pkg

Ivan Gusachenko · Answer 14 · Sat Jul 22 2017 07:45:53 GMT+0800 (China Standard Time)

Sorry, could you please tell how to uninstall it? There's no uninstaller, and its files are now all over the place, in different directories. However, when I just launch the file you've provided, the default option is to 'upgrade' all the components. Would it be OK to let the .pkg installer handle the upgrade, instead of a clean uninstall/install? Thank you

…

On Sat, Jul 22, 2017 at 12:36 AM, Pavan Yalamanchili < ***@***.***> wrote: @ivan-gusachenko <https://github.com/ivan-gusachenko> can you uninstall the current version of arrayfire and try installing the following: http://ci.arrayfire.org/userContent/OSX/ArrayFire-no-gl-v3.5.0_OSX.pkg — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#153 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIycptlI3j2DSLTgvu5Re6fR1kTD4GqPks5sQTXsgaJpZM4Of1wT> .

-- Dr. Ivan Gusachenko

Pavan Yalamanchili · Answer 15 · Sat Jul 22 2017 08:00:06 GMT+0800 (China Standard Time)

@umar456 can you reply to this ^?

Umar Arshad · Answer 16 · Sat Jul 22 2017 08:20:58 GMT+0800 (China Standard Time)

I really need to script this. It requires a few steps.

The installer creates a package IDs for the files created on the system. You can query all the arrayfire packages on your system:

> pkgutil --pkgs | grep arrayfire
com.arrayfire.pkg.arrayfire.cpu.lib
com.arrayfire.pkg.arrayfire.cuda.lib
com.arrayfire.pkg.arrayfire.doc
com.arrayfire.pkg.arrayfire.examples
com.arrayfire.pkg.arrayfire.inc
com.arrayfire.pkg.arrayfire.libcommon
com.arrayfire.pkg.arrayfire.opencl.lib
com.arrayfire.pkg.arrayfire.unified.lib
com.arrayfire.pkg.forge.cmake
com.arrayfire.pkg.forge.doc
com.arrayfire.pkg.forge.examples
com.arrayfire.pkg.forge.inc
com.arrayfire.pkg.forge.lib

You can get the install locations for each of the packages by calling info:

> # pkgutil --pkg-info <packagename>
> pkgutil --pkg-info com.arrayfire.pkg.arrayfire.unified.lib

package-id: com.arrayfire.pkg.arrayfire.unified.lib
version: 0
volume: /
location: usr/local                       # <-------- This
install-time: 1500682018

cd into the location entry and delete the files:

cd /usr/local
pkgutil --files com.arrayfire.pkg.arrayfire.unified.lib | xargs rm

This can be made into a script of course but this is the only way to do it now unfortunately.

Umar Arshad · Answer 17 · Sat Jul 22 2017 10:11:14 GMT+0800 (China Standard Time)

OSX Uninstall script

for af_package in $(pkgutil --pkgs | grep arrayfire)
do
    echo Deleting $af_package
    vol=$(pkgutil --pkg-info $af_package | awk '/volume/ { print $2 }')
    loc=$(pkgutil --pkg-info $af_package | awk '/location/ { print $2 }')
    for af_file in $(pkgutil --only-files --files $af_package)
    do
        rm $vol$loc/$af_file
    done

    pkgutil --forget $af_package
done

This will leave some directories and symlinks that were part of the install but it will delete all the files associated with the install

Ivan Gusachenko · Answer 18 · Sat Jul 22 2017 19:01:21 GMT+0800 (China Standard Time)

Hi, thanks, it all worked! I uninstalled the arrayfire with your script, then reinstalled it from the file you've provided, and added the DYLIB path. Now arrayfire is successfully imported in python.

I don't quite understand what was wrong then. Whether my installation order was wrong, or the package downloaded directly from arrayfire.com was somewhat faulty.
Really big thanks to you guys for helping out.

Pavan Yalamanchili · Answer 19 · Sun Jul 23 2017 00:22:45 GMT+0800 (China Standard Time)

@ivan-gusachenko The version I gave you will not be able to run any graphics functions. It looks like some dependency of forge was not present on your machine causing the error.