Order of `dream` and `caqti-driver-postgresql` in libraries stanza cause build to fail or succeed on MacOS
NilsIrl opened this issue · comments
On MacOS, I have been trying to build a project that depends on dream
and caqti-driver-postgresql
but get the following error:
% dune build
File "bin/dune", line 3, characters 7-11:
3 | (name main)
^^^^
Undefined symbols for architecture arm64:
"_EVP_MD_get_size", referenced from:
_ocaml_ssl_digest in libssl_stubs.a(ssl_stubs.o)
"_SSL_get1_peer_certificate", referenced from:
_ocaml_ssl_get_certificate in libssl_stubs.a(ssl_stubs.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)
The problem is caused by the order of which things are being linked. Once I ran the underyling ocamlopt.opt
command run by dune
and using -ccopt
to force -lssl
to be included at the beginning and it worked. This problem does not exist on linux.
Now switching the order of dream
and caqti-driver-postgresql
in the dune
file cause it to build without issues:
diff --git a/bin/dune b/bin/dune
index 59504b5..1391fc2 100644
--- a/bin/dune
+++ b/bin/dune
@@ -1,5 +1,5 @@
(executable
(public_name datebridge)
(name main)
- (libraries datebridge dream caqti-driver-postgresql)
+ (libraries datebridge caqti-driver-postgresql dream)
(preprocess (pps lwt_ppx)))
Thank you! @paurkedal Would you be able to comment on this issue? Do we need to document this, or is there a way to address it technically?
Does it compile if the caqti-driver-postgresql
dependency is completely removed?
(caqti-driver-postgresql
brings in libssl
though the dependency of the libpq
at the C level. The obvious but likely wrong explanation would be that something earlier in the library list depends on libssl
but does not have the correct C options.)
When caqti-driver-postgresql
is removed it still compiles but then there is a runtime error: "Neither caqti-driver-postgresql nor the dynamic linker is linked into the application."
Logs:
02.11.23 16:33:58.286 Running at http://localhost:8080
02.11.23 16:33:58.286 Type Ctrl+C to stop
02.11.23 16:33:59.500 dream.log INFO REQ 1 GET / 127.0.0.1:58870 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0
02.11.23 16:33:59.500 dream.sql ERROR REQ 1 Dream.sql_pool: cannot create pool for 'postgres://root:hunter2@localhost/datebridge': Failed to load driver for <postgres://root:_@localhost/datebridge>: Neither caqti-driver-postgresql nor the dynamic linker is linked into the application.
02.11.23 16:33:59.500 dream.log WARN REQ 1 Aborted by: Failure("Dream.sql_pool: cannot create pool for 'postgres://root:hunter2@localhost/datebridge': Failed to load driver for <postgres://nils:_@localhost/datebridge>: Neither caqti-driver-postgresql nor the dynamic linker is linked into the application.")
02.11.23 16:33:59.500 dream.log WARN Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
02.11.23 16:33:59.500 dream.log WARN Called from Lwt.Sequence_associated_storage.with_value in file "src/core/lwt.ml", line 804, characters 19-23
02.11.23 16:33:59.500 dream.log WARN Re-raised at Lwt.Sequence_associated_storage.with_value in file "src/core/lwt.ml", line 809, characters 6-15
02.11.23 16:33:59.500 dream.log WARN Called from Lwt.Sequential_composition.try_bind in file "src/core/lwt.ml", line 2139, characters 10-14
02.11.23 16:33:59.500 dream.http ERROR REQ 1 Failure("Dream.sql_pool: cannot create pool for 'postgres://root:hunter2@localhost/datebridge': Failed to load driver for <postgres://root:_@localhost/datebridge>: Neither caqti-driver-postgresql nor the dynamic linker is linked into the application.")
02.11.23 16:33:59.500 dream.http ERROR REQ 1 Raised at Stdlib__Map.Make.find in file "map.ml", line 141, characters 10-25
02.11.23 16:33:59.500 dream.http ERROR REQ 1 Called from Logs.Tag.find in file "src/logs.ml", line 154, characters 14-32
You can reproduce the issue by running opam switch create . --deps-only && dune exec datebridge
at https://gitlab.developers.cam.ac.uk/ucms/datebridge/-/commit/f8b1fe91e111406280d9011884e3fc37130ff5dd (the child commit on the trunk branch is the one that fixes the issue by re-ordering dream
and caqti-driver-postgresql
in bin/dune
)
Yes, the runtime error is expected, but that confirms that the postgresql driver or library is involved in some way. The next I would try would be to repeat with postgresql
and substituted for caqti-driver-postgresql
. If that shows the same pattern, try the same with the ssl
library for comparison, since it's linked directly to libssl instead of via libpq.
I don't have MacOS, but I can try to reproduce on Linux tomorrow if it's not ruled out.
AFAIK the issue does not exist on Linux. (I initially was developing on Linux and only found out about the issue when testing to MacOS).
Ah yes, you already wrote that in the OP. Nice work with minimizing the code.
Could the problem have the same underlying cause as facebookincubator/velox#5631? That is, two packages linking against different versions of libssl?
It may also be instructive to try a further minimization, by reducing to (libraries datebridge ssl postgresql)
(but is datebridge
needed?) and replacing the main program with
let _digest = Ssl.digest
let _connect () = new Postgresql.connection ()
In light of the previous comment, note that the OpenSSL version used for the ssl
library is decided by conf-libssl
and that used for the postgresql
library is decided by the libpq
build.
I reduced the program to the 2 line you suggested and replaced the libraries with the one you suggested and got the following error when building:
% dune build
File "bin/dune", line 3, characters 7-11:
3 | (name main)
^^^^
ld: Undefined symbols:
_EVP_MD_get_size, referenced from:
_ocaml_ssl_digest in libssl_stubs.a[2](ssl_stubs.o)
_SSL_get1_peer_certificate, referenced from:
_ocaml_ssl_get_certificate in libssl_stubs.a[2](ssl_stubs.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)
I then flipped the order of ssl
and postgresql
in bin/dune
to give (libraries postgresql ssl))
and it built without issue. datebridge
is not necessary and does not affect the error.
It seems almost clear now; at least the fact that some postgresql
link option override those of the ssl
library. As far as I can see postgresql
does not add -lssl
to the command line, but it does add an -L
option, which points to the output of pg_config --libdir
. Could it be that this directory contains an incompatible OpenSSL library?
This solved my issue although I was using cryptokit, same thing though so if anyone else stumbles on this in the future it was the ordering as well for me.
I had:
(libraries lwt cryptokit petrol caqti caqti-lwt caqti-driver-postgresql)
and got the following error on build:
File "bin/dune", line 3, characters 7-11:
3 | (name main)
^^^^
ld: Undefined symbols:
_EVP_MD_get_size, referenced from:
_ocaml_ssl_digest in libssl_stubs.a[2](ssl_stubs.o)
_SSL_get1_peer_certificate, referenced from:
_ocaml_ssl_get_certificate in libssl_stubs.a[2](ssl_stubs.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)
Just changing the ordering of the libraries in the dune file to:
(libraries lwt petrol caqti caqti-lwt caqti-driver-postgresql cryptokit)
and everything built without an error. Wasted maybe 8 hours on this and went down a lot of rabbit holes with openssl and other fun stuff.
Thank you! What system did you observe this on? Also macOS?
yup I'm on macOS 14.2 (Sonoma)
@paurkedal I'd like to close this issue here in this repo, as it seems not to be caused by Dream, though it's good that it started here because Dream's users are affected by it and can search for it. Where should it be reported? Caqti? postgresql
?
Caqti is also just an intermediary is this, as far as I understand the problem. Though, I'm not sure what the postgresql-ocaml library can do if, as I assumed above, the issue is that the -L
option needed to locate libpq
also means that a wrong version of libssl
is picked up. Further, does the ssl
library maybe omit an -L
option since it's a standard location, but thus not overriding a non-standard location pushed to the front of the search path by postgresql
? And if that is the case, maybe it can be solved with changes to linking options in either ssl
or postgresql
.
cc @toots, @anmonteiro, @mmottl about the linking options. Do you know what is causing this ordering issue on macOS?
It would be nice if we could have the full linking command in the broken and working case to have a look at them.