Problem with Julia API on Ubuntu 24.04
lanctot opened this issue · comments
Hi @findmyway, as promised here is some more information on the breakage of the Julia API on Ubuntu 24.04.
Interestingly, it still works on Ubuntu-22.04 as it is included in one of our CI tests which we've run several times recently (i.e. on the current head of master).
Here's how to reproduce:
(venv) lanctot@nitro-exp:~/open_spiel$ OPEN_SPIEL_BUILD_WITH_JULIA=ON ./install.sh
.
.
.
++ bash open_spiel/scripts/jill.sh -y -v 1.6.1
JILL - Julia Installer 4 Linux - Light
Copyright (C) 2017-2023 Abel Soares Siqueira <abel.s.siqueira@gmail.com>
Distributed under terms of the GPLv3 license.
This script will:
- Try to download julia version '1.6.1'
- Create a link for julia
- Create a link for julia-VER
Download folder: '/home/lanctot/packages/julias'
Link folder: '/home/lanctot/.local/bin'
Download folder will be created if required
Downloading Julia version 1.6.1
++ [[ -f /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ]]
++ cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /home/lanctot/packages/julias/julia-1.6.1/lib/julia
+++ which julia
++ [[ -x /home/lanctot/.local/bin/julia ]]
+++ julia -e 'println(VERSION >= v"1.6.1")'
++ '[' true == true ']'
++ julia --project=/home/lanctot/open_spiel/open_spiel/julia -e 'using Pkg; Pkg.instantiate();'
Installing known registries into `~/.julia`
Added registry `General` to `~/.julia/registries/General`
Installed MacroTools ─────────── v0.5.13
Installed JLLWrappers ────────── v1.5.0
Installed Preferences ────────── v1.4.3
Installed libcxxwrap_julia_jll ─ v0.8.8+1
Installed CxxWrap ────────────── v0.11.2
Downloaded artifact: libcxxwrap_julia
Precompiling project...
✗ OpenSpiel
8 dependencies successfully precompiled in 9 seconds
1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
(venv) lanctot@nitro-exp:~/open_spiel$ which julia
/home/lanctot/.local/bin/julia
(venv) lanctot@nitro-exp:~/open_spiel$ julia --project=/home/lanctot/open_spiel/open_spiel/julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.6.1 (2021-04-23)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using Pkg; Pkg.instantiate();
Precompiling project...
✗ OpenSpiel
0 dependencies successfully precompiled in 2 seconds (8 already precompiled)
1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
julia> import Pkg; Pkg.precompile()
Precompiling project...
✗ OpenSpiel
0 dependencies successfully precompiled in 2 seconds (8 already precompiled)
ERROR: The following 1 direct dependency failed to precompile:
OpenSpiel [7e5d8dc8-489f-4be6-9273-f4c3010e6ec1]
ERROR: LoadError: could not load library "/home/lanctot/open_spiel/open_spiel/julia/deps/../../../build/julia/libspieljl"
/home/lanctot/open_spiel/open_spiel/julia/deps/../../../build/julia/libspieljl.so: cannot open shared object file: No such file or directory
Stacktrace:
[1] dlopen(s::String, flags::UInt32; throw_error::Bool)
@ Base.Libc.Libdl ./libdl.jl:114
[2] dlopen
@ ./libdl.jl:114 [inlined]
[3] readmodule(so_path::String, funcname::Symbol, m::Module, flags::Nothing)
@ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/OcN1Z/src/CxxWrap.jl:724
[4] wrapmodule(so_path::String, funcname::Symbol, m::Module, flags::Nothing)
@ CxxWrap.CxxWrapCore ~/.julia/packages/CxxWrap/OcN1Z/src/CxxWrap.jl:729
[5] top-level scope
@ ~/open_spiel/open_spiel/julia/src/OpenSpiel.jl:9
[6] include
@ ./Base.jl:386 [inlined]
[7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
@ Base ./loading.jl:1213
[8] top-level scope
@ none:1
[9] eval
@ ./boot.jl:360 [inlined]
[10] eval(x::Expr)
@ Base.MainInclude ./client.jl:446
[11] top-level scope
@ none:1
in expression starting at /home/lanctot/open_spiel/open_spiel/julia/src/OpenSpiel.jl:1
Stacktrace:
[1] pkgerror(msg::String)
@ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55
[2] precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1244
[3] precompile
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:920 [inlined]
[4] #precompile#196
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:918 [inlined]
[5] precompile()
@ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:918
[6] top-level scope
@ REPL[2]:1
julia>
(venv) lanctot@nitro-exp:~/open_spiel$ mkdir build
(venv) lanctot@nitro-exp:~/open_spiel$ cd build/
(venv) lanctot@nitro-exp:~/open_spiel/build$ OPEN_SPIEL_BUILD_WITH_JULIA=ON cmake -DPython3_EXECUTABLE=python -DCMAKE_CXX_COMPILER=clang++ ../open_spiel
.
.
.
CMake Error at julia/CMakeLists.txt:1 (find_package):
By not providing "FindJlCxx.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "JlCxx", but
CMake did not find one.
Could not find a package configuration file provided by "JlCxx" with any of
the following names:
JlCxxConfig.cmake
jlcxx-config.cmake
Add the installation prefix of "JlCxx" to CMAKE_PREFIX_PATH or set
"JlCxx_DIR" to a directory containing one of the above files. If "JlCxx"
provides a separate development package or SDK, be sure it has been
installed.
-- Configuring incomplete, errors occurred!
(venv) lanctot@nitro-exp:~/open_spiel/build$
Note: I tried updating the versions of Julia and CxxWrap in Project.toml but that didn't seem to work.
Note: you should not need to build with python, so you can safely remove -DPython3_EXECUTABLE=python.. I'm just used to always including it.
Got it, I'll look into it soon.
I finally got a workable environment. Still working on it.
Here's what I found with a fresh new docker image of tag ubuntu:24.04:
- The error with
install.shcan be safely ignored. Because at that time, the openspiel lib is not built yet.
Based on discussions here: https://discourse.julialang.org/t/how-to-disable-automatic-precompilation-of-packages-in-julia-1-6/57053
We can set the JULIA_PKG_PRECOMPILE_AUTO environment variable to 0 to stop the precompiling here.
open_spiel/open_spiel/scripts/install.sh
Line 219 in 1208f83
- The above line needs to be updated:
- cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 $HOME/packages/julias/julia-1.6.1/lib/julia
+ cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /opt/julias/julia-1.6.1/bin/../lib/julia/libstdc++.so.6Hmm, I retried and I am still getting that CMake error above that it cannot find the configuration file.
There is also nothing in /opt, because the installer is putting it into $HOME/packages/julias like before.
This is on a fresh install of Ubuntu 24.04.
Hmm, I retried and I am still getting that CMake error above that it cannot find the configuration file.
So I assume you executed like this?
(venv) lanctot@nitro-exp:~/open_spiel/build$ OPEN_SPIEL_BUILD_WITH_JULIA=ON cmake -DPython3_EXECUTABLE=python -DCMAKE_CXX_COMPILER=clang++ ../open_spiel
Well that is not enough since some extra environment variables needs to be set. See
You can simply execute build_and_run_tests.sh to build the lib.
There is also nothing in /opt, because the installer is putting it into $HOME/packages/julias like before.
I see, in that case we don't need that modification.
Ohhh, I had totally forgotten about that extra environment variable. Good call, and I just checked: it works! I will add a test that re-enables the Julia API test. Thanks for helping with this!