pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails)
monacoremo opened this issue · comments
Describe the bug
Trying to build GHC 8.8.3 on pkgsMusl (in order to build static Haskell executables) fails with:
...
Installing library in /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/ghc-8.6.5
"/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg" --force --global-package-db "/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/package.conf.d" update rts/dist/package.conf.install
/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg: /nix/store/1i32nv0sm71qr3qak4jg9bqmlq4pq1la-ncurses-6.2-abi5-compat/lib/libtinfo.so.5: no version information available (required by /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg)
/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg: /nix/store/1i32nv0sm71qr3qak4jg9bqmlq4pq1la-ncurses-6.2-abi5-compat/lib/libtinfo.so.5: no version information available (required by /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/../terminfo-0.4.1.2/libHSterminfo-0.4.1.2-ghc8.6.5.so)
Inconsistency detected by ld.so: dl-lookup.c: 111: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:991: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/ywiy47vvr8b9blq32kw2zcqqqpqrs7kc-ghc-8.6.5-binary.drv' failed with exit code 2
cannot build derivation '/nix/store/0aivgs76j9xrf9flbbs5mhlfc0gfqyk9-ghc-8.8.3.drv': 1 dependencies couldn't be built
To Reproduce
On a recent pin of nixpkgs-unstable (rev 10100a9):
nix-build -A pkgsMusl.haskell.compiler.ghc883
Expected behavior
GHC 8.8.3 is built based on Musl
Notify maintainers
Metadata
- system:
"x86_64-linux" - host os:
Linux 5.0.0-38-generic, Ubuntu, 19.04 (Disco Dingo) - multi-user?:
no - sandbox:
yes - version:
nix-env (Nix) 2.3.1 - channels(remo):
"nixos-20.03pre204199.3140fa89c51, nixpkgs-20.03pre204216.cc6cf0a96a6" - nixpkgs:
/home/remo/.nix-defexpr/channels/nixpkgs
@monacoremo Thanks for creating this issue.
Unfortunately, we don't always check to make sure the musl builds succeed before merging stuff in.
If you wanted to fix this issue, we'd happily accept a PR.
Also, @nh2 is doing a lot with the musl builds, especially in https://github.com/nh2/static-haskell-nix. If you absolutely need a musl'd GHC, I'd suggest to use https://github.com/nh2/static-haskell-nix.
To add to that: I pin a nixpkgs that works well in a submodule:
and the same in the nixpkgs.file:
On the issue: We've seen build failures on latest nixpkgs master with musl recently, hopefulyy they will be fixed soon!
Just for the record, current versions of master (and haskell-updates) still have this issue. We'll try to build the package with 3c7ef6b reverted to check whether the update of the boot compiler caused this problem.
Many thanks all for your help and quick responses!
As @peti suggested I reverted 3c7ef6b and pkgsMusl.haskell.compiler.ghc883 is currently building, getting much further than on ghc865-binary - looking good! I'll update once the build is done, takes quite a bit of time...
Edit: Confirmed, successfully built /nix/store/x03kyh5mfrapm1mjrh5a5v4ygly6rzwz-ghc-8.8.3 !
I would like to contribute a fix if possible, but I'm not sure what direction would be best to take - GHC 8.8.3 binaries are now available, would it make sense to bootstrap from there?
Edit2: Maybe it just needs s/ncurses5/ncurses6/ in https://github.com/nixos/nixpkgs/blob/master/pkgs/development/compilers/ghc/8.6.5-binary.nix? Trying that out right now. Would that be likely to break anything else?
Edit3: Couldn't get the libtinfo.so dependency from ncurses to work for 8.6.5-binary on pkgsMusl... @dtzWill , I saw that you worked on related things, do you have an idea maybe how to solve this?
CC @bgamari
It looks to me like this is a bug in the ncurses-abi-compat derivation. In particular, the libtinfo it exposes has the incorrect soname. This is likely because is isn't libtinfo at all; rather, it's simply a symlink to libncursesw. In general I don't see how this could work.
We work around this in mpickering/ghc-artefact-nix via https://github.com/mpickering/ghc-artefact-nix/blob/99fcf89d36d306beb243acccc87d1f6e153ad50f/artifact.nix#L104.
Collecting some info for myself:
- The
ncurses-abi-compatderivation @bgamari refers to (called ratherncurses-*-abi5-compatin the issue description) is
Question:
@bgamari Do you know what a fix could look like? Instead of a symlink, should it be a copy of the .so file, on which patchelf --replace-needed is subsequently run, as shown in your link? If yes, should that happen unconditionally or only for abiVersion == "5"?
I don't know anything about the ncurses 5->6 switch, so I have no idea why this sophisticated mechanism is there in the first place.
The ideal solution would be to rather use a GHC binary distribution which expects ncurses ABI 6. For instance, this patch appears to work around the issue locally:
diff --git a/pkgs/development/compilers/ghc/8.6.5-binary.nix b/pkgs/development/compilers/ghc/8.6.5-binary.nix
index 97793d91289..188b12f5dad 100644
--- a/pkgs/development/compilers/ghc/8.6.5-binary.nix
+++ b/pkgs/development/compilers/ghc/8.6.5-binary.nix
@@ -1,6 +1,6 @@
{ stdenv
, fetchurl, perl, gcc
-, ncurses5, gmp, glibc, libiconv
+, ncurses6, gmp, glibc, libiconv
, llvmPackages
}:
@@ -11,7 +11,7 @@ let
useLLVM = !stdenv.targetPlatform.isx86;
libPath = stdenv.lib.makeLibraryPath ([
- ncurses5 gmp
+ ncurses6 gmp
] ++ stdenv.lib.optional (stdenv.hostPlatform.isDarwin) libiconv);
libEnvVar = stdenv.lib.optionalString stdenv.hostPlatform.isDarwin "DY"
@@ -38,8 +38,8 @@ stdenv.mkDerivation rec {
sha256 = "1p2h29qghql19ajk755xa0yxkn85slbds8m9n5196ris743vkp8w";
};
x86_64-linux = {
- url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-deb9-linux.tar.xz";
- sha256 = "1pqlx6rdjs2110g0y1i9f8x18lmdizibjqd15f5xahcz39hgaxdw";
+ url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-fedora27-linux.tar.xz";
+ sha256 = "18dlqm5d028fqh6ghzn7pgjspr5smw030jjzl3kq6q1kmwzbay6g";
};
aarch64-linux = {
url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-aarch64-ubuntu18.04-linux.tar.xz";
@@ -100,9 +100,9 @@ stdenv.mkDerivation rec {
'' +
# Rename needed libraries and binaries, fix interpreter
stdenv.lib.optionalString stdenv.isLinux ''
- find . -type f -perm -0100 -exec patchelf \
+ find . -type f -perm -0100 \
+ -exec patchelf \
--replace-needed libncurses${stdenv.lib.optionalString stdenv.is64bit "w"}.so.5 libncurses.so \
- --replace-needed libtinfo.so libtinfo.so.5 \
--interpreter ${glibcDynLinker} {} \;
sed -i "s|/usr/bin/perl|perl\x00 |" ghc-${version}/ghc/stage2/build/tmp/ghc-stage2
Is it feasible to add pkgsMusl.haskell.compiler.ghc* to hydra to prevent this from happening?
FWIW this also happens with the ghc-8.10.2 binary used to bootstrap ghc901:
$ nb -A pkgsMusl.haskell.compiler.ghc901
...
Inconsistency detected by ld.so: dl-lookup.c: 105: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:973: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/d6a8i0q7yiv97rsi9yz38p7f8ykhn7c6-ghc-8.10.2-binary.drv' failed with exit code 2
cannot build derivation '/nix/store/z3kk34qqqnncsc132v899yscxanfrpb6-ghc-9.0.0.20200925.drv': 1 dependencies couldn't be built
error: build of '/nix/store/z3kk34qqqnncsc132v899yscxanfrpb6-ghc-9.0.0.20200925.drv' failed
I built things with @bgamari's patch above and it all went very smoothly, is there a reason against including it in nixpkgs, even if just for musl builds?