Add customized libstdc++ to "clang (experimental P1144)"
Quuxplusone opened this issue · comments
I'm interested in expanding my "clang (experimental P1144)"/clang-relocatable compiler ( https://godbolt.org/z/c3Pa56GM7 ) to build not only Clang and libc++, but also libstdc++. My goal is that a user selecting "clang (experimental P1144)" and passing -stdlib=libstdc++ will get a libstdc++ that is built from https://github.com/Quuxplusone/gcc/tree/trivially-relocatable , just as when passing -stdlib=libc++ they already get a libc++ that's built from https://github.com/Quuxplusone/llvm-project/tree/trivially-relocatable .
I suspect that the right way to go about this is for me to open a GitHub issue here, and wait for someone with more knowledge to take it on. :) I would guess that the solution might involve taking part of the current workflow for "gcc (trunk)" and grafting it onto the existing clang-relocatable workflow; but I wouldn't be surprised either if that was a bad idea for some reason. As of this writing, I don't even know how to build libstdc++ locally. So my step 1 is to open this issue and ask for help. :)
Sorry it took so long to get to this 😊
I think that "might" be as simple as passing --gcc-root=<the right thing>. We already point each clang at a "compatible" gcc.
It doesn't look like we currently have a "trivially relocatable" gcc build around, so the first part would be to PR aginst the gcc builder. So I think something like (deep breath...so many steps and so many repos...sorry)
- PR against https://github.com/compiler-explorer/gcc-builder/blob/main/build/build.sh and add the relevant magic incantations to build the trivially relocatable gcc
- PR against https://github.com/compiler-explorer/compiler-workflows/blob/main/compilers.yaml and add this new gcc to the list of daily-built compilers
- PR against and add the new gcc there (it will be installed every night too)
Lines 1160 to 1169 in cb5c288
- [optional] PR to add the new GCC to the list of compilers in CE itself. Might as well
- Finally PR to the https://github.com/compiler-explorer/compiler-explorer/blob/477db1b7943fc6675d499ec119ad5e93a1993933/etc/config/c%2B%2B.amazon.properties#L410-L413 and change
compiler.clang_p1144.options=--gcc-toolchain=/opt/compiler-explorer/gcc-snapshot -stdlib=libc++ -g0
to
compiler.clang_p1144.options=--gcc-toolchain=/opt/compiler-explorer/gcc-relocatble-trunk -stdlib=libc++ -g0
``` (or similar)
Once again, sorry for this! If any of those steps are unclear, LMK and I'll try and get to it, but in the spirit of teaching fishing...here you go!! :)
@mattgodbolt: I've opened PRs for steps (1), (2), (3). It is kinda painful. :P I don't want to do step (4) because so far there is no compiler implementation of P1144 for gcc, so there's no sense in exposing the not-actually-implemented compiler to the user.
Orthogonally, but related: I've chosen the name gcc-p1144 for the gcc fork, and I think it would be appropriate to rename the Clang fork also to clang-p1144 (as opposed to the current name clang-relocatable). I wouldn't trust myself to find all the places that need to change in order for that to happen, nor would I trust the PRs to get merged atomically enough to not-break-anything. Maybe I could prevail upon you to do that renaming yourself? :)
If you did, that would merge-conflict with step (5); so I haven't made a PR for step (5) yet. You have my permission to do step (5) if you want to; or, I can submit a PR for it "later."
Thanks for your patience, I'm mergin 1-3 and doing the remaining little bits on my side.
Understood re: 4.
Let's get things built etc and then I'll make the change on step 5
[rename the] Clang fork also to clang-p1144 (as opposed to the current name clang-relocatable)
I'm not going to do this; all the remaining clang forks are not named so.
@Quuxplusone Sadly the build failed:
declaration of '__is_trivially_relocatable' must be available [-fpermissive]
2023-12-10T19:58:45.4765064Z 825 | : public __bool_constant<__is_trivially_relocatable(_Tp)>
2023-12-10T19:58:45.4765563Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~
2023-12-10T19:58:45.4766699Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:825:30: note: (if you use '-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated)
2023-12-10T19:58:45.4768124Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:825:61: error: template argument 1 is invalid
2023-12-10T19:58:45.4768885Z 825 | : public __bool_constant<__is_trivially_relocatable(_Tp)>
2023-12-10T19:58:45.4769306Z | ^
2023-12-10T19:58:45.5128118Z In file included from /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/bits/stl_pair.h:60,
2023-12-10T19:58:45.5129411Z from /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/bits/stl_algobase.h:64,
2023-12-10T19:58:45.5130547Z from /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/algorithm:60,
2023-12-10T19:58:45.5131686Z from /opt/compiler-explorer/gcc-build/gcc-p1144-trunk-20231210/libstdc++-v3/include/precompiled/stdc++.h:51:
2023-12-10T19:58:45.5133697Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:825:30: error: there are no arguments to '__is_trivially_relocatable' that depend on a template parameter, so a declaration of '__is_trivially_relocatable' must be available [-fpermissive]
2023-12-10T19:58:45.5135236Z 825 | : public __bool_constant<__is_trivially_relocatable(_Tp)>
2023-12-10T19:58:45.5135656Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~
2023-12-10T19:58:45.5136782Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:825:30: note: (if you use '-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated)
2023-12-10T19:58:45.5138262Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:825:61: error: template argument 1 is invalid
2023-12-10T19:58:45.5139018Z 825 | : public __bool_constant<__is_trivially_relocatable(_Tp)>
2023-12-10T19:58:45.5139441Z | ^
2023-12-10T19:58:45.5260803Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:3255:84: error: expected primary-expression before ')' token
2023-12-10T19:58:45.5262027Z 3255 | inline constexpr bool is_trivially_relocatable_v = __is_trivially_relocatable(_Tp);
2023-12-10T19:58:45.5262709Z | ^
2023-12-10T19:58:45.5264472Z /opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include/type_traits:3255:54: error: there are no arguments to '__is_trivially_relocatable' that depend on a template parameter, so a declaration of '__is_trivially_relocatable' must be available [-fpermissive]
2023-12-10T19:58:45.5265894Z 3255 | inline constexpr bool is_trivially_relocatable_v = __is_trivially_relocatable(_Tp);
2023-12-10T19:58:45.5266423Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~
2023-12-10T19:58:48.5073963Z make[4]: *** [Makefile:1927: x86_64-linux-gnu/bits/stdc++.h.gch/O2ggnu++0x.gch] Error 1
2023-12-10T19:58:48.5074666Z make[4]: *** Waiting for unfinished jobs....
2023-12-10T19:58:49.9665956Z make[4]: *** [Makefile:1932: x86_64-linux-gnu/bits/stdc++.h.gch/O2g.gch] Error 1
2023-12-10T19:58:49.9667502Z make[4]: Leaving directory '/opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3/include'
2023-12-10T19:58:49.9674108Z make[3]: *** [Makefile:576: all-recursive] Error 1
2023-12-10T19:58:49.9674798Z make[3]: Leaving directory '/opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3'
2023-12-10T19:58:49.9676720Z make[2]: *** [Makefile:501: all] Error 2
2023-12-10T19:58:49.9677573Z make[2]: Leaving directory '/opt/compiler-explorer/gcc-build/objdir/x86_64-linux-gnu/libstdc++-v3'
2023-12-10T19:58:49.9683064Z make[1]: *** [Makefile:14039: all-target-libstdc++-v3] Error 2
2023-12-10T19:58:49.9683846Z make[1]: Leaving directory '/opt/compiler-explorer/gcc-build/objdir'
2023-12-10T19:58:49.9690067Z make: *** [Makefile:1067: all] Error 2
2023-12-10T19:58:53.1257046Z ##[error]Process completed with exit code 2.
2023-12-10T19:58:53.1307657Z ##[group]Run ./post-run.sh "gcc_p1144" "" "" "1702237864"
2023-12-10T19:58:53.1308108Z �[36;1m./post-run.sh "gcc_p1144" "" "" "1702237864"�[0m
2023-12-10T19:58:53.1317351Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
[rename the] Clang fork also to clang-p1144 (as opposed to the current name clang-relocatable)
I'm not going to do this; all the remaining clang forks are not named so.
clang-p1061 is named so; and I think it's a good convention, for forks specifically associated with active proposals. (Although I admit my fork implements more than just P1144.) The other reason is that the word "relocatable" is overloaded (e.g. PIC/PIE executables are "relocatable" in a different sense, right?).
So I think you ought to agree with my conclusion. :) But since I'm not offering to do the rename myself, I'll end up going along with whatever you decide, either way.
@Quuxplusone Sadly the build failed:
declaration of '__is_trivially_relocatable' must be available [-fpermissive] 2023-12-10T19:58:45.4765064Z 825 | : public __bool_constant<__is_trivially_relocatable(_Tp)> 2023-12-10T19:58:45.4765563Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~
Uh-oh, I think this might indicate a problem with the high-level approach; I really want libstdc++ to be compiled with (the P1144 fork of) Clang, not compiled with GCC at all. However, I'll need to look closer at this first; it might be that I can deal with it (that is, support compiling with both Clang and GCC, in a way that doesn't defeat my purpose) just by adding some additional #ifdefs to my libstdc++ code.
clang-p1061 is named so; and I think it's a good convention
bollo:~/d/c/infra (main|✔) $ git grep clang-p1061
bollo:~/d/c/infra (main|✔) $
Where is it named that? I think I'm missing something :)
Uh-oh, I think this might indicate a problem with the high-level approach;
Ok cool: In theory you can use the docker container locally to build exactly as we do and so then you can debug and see the issues without a roundtrip to github etc. Instructions should be in the README. If you need a build of another compiler there are ways to ce_install another CE compiler as part of the build too.
Where is it named that? I think I'm missing something :)
https://github.com/search?q=org%3Acompiler-explorer%20p1061&type=code
However, I'll need to look closer at this first; it might be that I can deal with it (that is, support compiling with both Clang and GCC, in a way that doesn't defeat my purpose) just by adding some additional #ifdefs to my libstdc++ code.
Okay, I pushed the branch again, in a way that might be better or might be worse. :) Will try to investigate the Docker thing later — ideally with some hints from you. ;)
Where is it named that? I think I'm missing something :)
https://github.com/search?q=org%3Acompiler-explorer%20p1061&type=code
Got it! thanks
Okay, I pushed the branch again, in a way that might be better or might be worse. :) Will try to investigate the Docker thing later — ideally with some hints from you. ;)
https://github.com/compiler-explorer/compiler-workflows/actions/workflows/build-daily-gcc_p1144.yml seems to have done the trick :)
Have done step 5....and will deploy shortly to see whether it all fits together :)
This is now deployed: let me know how it works out!
@mattgodbolt: Looks good to me at first glance! 🎉 🙏
Here's an example showing the new behavior: https://godbolt.org/z/evex4exoW
The left compiler pane is passing -stdlib=libstdc++ so that it compiles my patched libstdc++ with my patched Clang; this allows types like std::unique_ptr<int> to be recognized as trivially relocatable no matter which library is being used.
So, unless I find something on second glance that doesn't look right, I think we can call this "mission accomplished"! Thanks a ton!