Installation issue: MPICH package not configured with the right Fortran compiler
jeffhammond opened this issue · comments
Steps to reproduce the issue
$ spack spec -l mpich device=ch4 netmod=ucx hcoll=false
Input spec
--------------------------------
- mpich~hcoll device=ch4 netmod=ucx
Concretized
--------------------------------
- r4fetpt mpich@4.2.1%clang@19.0.0git~argobots~cuda+fortran~hcoll+hwloc+hydra+libxml2+pci~rocm+romio~slurm~vci~verbs+wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ucx pmi=default arch=linux-ubuntu22.04-zen4
[+] wpk6ask ^findutils@4.9.0%gcc@13.2.0 build_system=autotools patches=440b954 arch=linux-ubuntu22.04-zen4
[+] 5ztf3wp ^gcc-runtime@13.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu22.04-zen4
[e] eovh7js ^glibc@2.35%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] srobu4x ^gmake@4.4.1%gcc@13.2.0~guile build_system=generic arch=linux-ubuntu22.04-zen4
[+] hwbdfbj ^hwloc@2.9.1%gcc@13.2.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-ubuntu22.04-zen4
[+] wuh4h5w ^ncurses@6.4%gcc@13.2.0~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-ubuntu22.04-zen4
[+] ulltwdw ^libpciaccess@0.17%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] friofj2 ^libtool@2.4.7%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] kk5tq47 ^util-macros@1.19.3%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] okokkeq ^libxml2@2.10.3%gcc@13.2.0+pic~python+shared build_system=autotools arch=linux-ubuntu22.04-zen4
[+] 5ps2cyo ^libiconv@1.17%gcc@13.2.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-zen4
[+] 2ubf5w7 ^xz@5.4.6%gcc@13.2.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu22.04-zen4
[+] 5ndf5pz ^zlib-ng@2.1.6%gcc@13.2.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu22.04-zen4
[+] oj3siih ^pkgconf@1.9.5%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] kjsupd2 ^ucx@1.16.0%clang@19.0.0git~assertions~backtrace_detail~cma~cuda~dc~debug~dm+examples~gdrcopy~gtest~ib_hw_tm~java~knem~logging~mlx5_dv+openmp+optimizations~parameter_checking+pic~rc~rdmacm~rocm~thread_multiple~ucg~ud~verbs~vfs~xpmem build_system=autotools libs=shared,static opt=3 simd=auto arch=linux-ubuntu22.04-zen4
[+] 5cqnvaf ^yaksa@0.3%clang@19.0.0git~cuda~rocm build_system=autotools arch=linux-ubuntu22.04-zen4
[+] 7ueq3fm ^autoconf@2.72%clang@19.0.0git build_system=autotools arch=linux-ubuntu22.04-zen4
[+] 6fy7gl2 ^perl@5.38.0%gcc@13.2.0+cpanm+opcode+open+shared+threads build_system=generic patches=714e4d1 arch=linux-ubuntu22.04-zen4
[+] pdh6riv ^berkeley-db@18.1.40%gcc@13.2.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu22.04-zen4
[+] yxnivmi ^automake@1.16.5%clang@19.0.0git build_system=autotools arch=linux-ubuntu22.04-zen4
[+] 7najtcf ^m4@1.4.19%gcc@13.2.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-ubuntu22.04-zen4
[+] vxfnmwx ^diffutils@3.10%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] hk5425x ^libsigsegv@2.14%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] xcohooy ^python@3.11.7%gcc@13.2.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=linux-ubuntu22.04-zen4
[+] 6soib6y ^bzip2@1.0.8%gcc@13.2.0~debug~pic+shared build_system=generic arch=linux-ubuntu22.04-zen4
[+] k5p35fw ^expat@2.6.2%gcc@13.2.0+libbsd build_system=autotools arch=linux-ubuntu22.04-zen4
[+] v4ju4mm ^libbsd@0.12.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] 5egs7o6 ^libmd@1.0.4%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] ymcrfbb ^gdbm@1.23%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] wb2xz4h ^gettext@0.22.4%gcc@13.2.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu22.04-zen4
[+] lbcdm5z ^tar@1.34%gcc@13.2.0 build_system=autotools zip=pigz arch=linux-ubuntu22.04-zen4
[+] hnm7e6c ^pigz@2.8%gcc@13.2.0 build_system=makefile arch=linux-ubuntu22.04-zen4
[+] gfc3c4x ^zstd@1.5.6%gcc@13.2.0+programs build_system=makefile compression=none libs=shared,static arch=linux-ubuntu22.04-zen4
[+] sdurl22 ^libffi@3.4.6%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
[+] eqcspjn ^libxcrypt@4.4.35%gcc@13.2.0~obsolete_api build_system=autotools patches=4885da3 arch=linux-ubuntu22.04-zen4
[+] l5uuw33 ^openssl@3.2.1%gcc@13.2.0~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu22.04-zen4
[+] spgyryd ^ca-certificates-mozilla@2023-05-30%gcc@13.2.0 build_system=generic arch=linux-ubuntu22.04-zen4
[+] 26ulxrd ^readline@8.2%gcc@13.2.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu22.04-zen4
[+] aygi42j ^sqlite@3.43.2%gcc@13.2.0+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-ubuntu22.04-zen4
[+] cs5noep ^util-linux-uuid@2.38.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-zen4
MPICH maintainers: @raffenet @yfguo
Error message
Error message
==> Error: ProcessError: Command exited with status 1:
'/tmp/jehammond/spack-stage/spack-stage-mpich-4.2.1-r4fetptlct6yns3z36sb7k43cuxwicfx/spack-src/configure' '--prefix=/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/mpich-4.2.1-r4fetptlct6yns3z36sb7k43cuxwicfx' '--disable-maintainer-mode' '--disable-silent-rules' '--enable-shared' '--with-pm=hydra' '--enable-romio' '--without-ibverbs' '--enable-wrapper-rpath=yes' '--with-yaksa=/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/yaksa-0.3-5cqnvaf3ijrvqlqjcvvu72zaywxmopbb' '--with-hwloc=/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/gcc-13.2.0/hwloc-2.9.1-hwbdfbjmmyakcb4kjki7mdqa7nagj5v7' '--with-slurm=no' '--without-cuda' '--without-hip' '--with-device=ch4:ucx' '--with-ucx=/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/ucx-1.16.0-kjsupd2vmqn5snejiuvjp3nfonkt5mkm' '--enable-libxml2' '--with-datatype-engine=auto'
3 errors found in build log:
167 checking Python 3... python
168 checking whether we are using the GNU Fortran compiler... no
169 checking whether /home/jehammond/SPACK/spack/lib/spack/env/clang/flang accepts -g... no
170 checking if libtool supports shared libraries... yes
171 checking whether to build shared libraries... yes
172 checking whether to build static libraries... yes
>> 173 libtool.m4: error: problem compiling FC test program
174 checking for /home/jehammond/SPACK/spack/lib/spack/env/clang/flang option to produce PIC... -fPIC
175 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang PIC flag -fPIC works... no
176 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang static flag -static works... no
177 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang supports -c -o file.o... no
178 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang supports -c -o file.o... (cached) no
179 checking whether the /home/jehammond/SPACK/spack/lib/spack/env/clang/flang linker (/usr/bin/ld -m elf_x86_
64) supports shared libraries... yes
...
759 checking how to hardcode library paths into programs... immediate
760 checking whether we are using the GNU Fortran compiler... no
761 checking whether /home/jehammond/SPACK/spack/lib/spack/env/clang/flang accepts -g... no
762 checking if libtool supports shared libraries... yes
763 checking whether to build shared libraries... yes
764 checking whether to build static libraries... yes
>> 765 libtool.m4: error: problem compiling FC test program
766 checking for /home/jehammond/SPACK/spack/lib/spack/env/clang/flang option to produce PIC...
767 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang static flag works... no
768 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang supports -c -o file.o... no
769 checking if /home/jehammond/SPACK/spack/lib/spack/env/clang/flang supports -c -o file.o... (cached) no
770 checking whether the /home/jehammond/SPACK/spack/lib/spack/env/clang/flang linker (/usr/bin/ld -m elf_x86_
64) supports shared libraries... yes
771 checking dynamic linker characteristics... (cached) GNU/Linux ld.so
...
1047 checking for Fortran 77 libraries of /home/jehammond/SPACK/spack/lib/spack/env/clang/flang...
1048 checking whether /home/jehammond/SPACK/spack/lib/spack/env/clang/flang accepts the FLIBS found by autoconf
... no
1049 checking for valid entries in FLIBS...
1050 checking whether /home/jehammond/SPACK/spack/lib/spack/env/clang/clang links with FLIBS found by autoconf.
.. yes
1051 checking whether Fortran 77 and C objects are compatible... no
1052 checking for file... file
>> 1053 configure: error: **** Incompatible Fortran and C Object File Types! ****
1054 F77 Object File Type produced by "/home/jehammond/SPACK/spack/lib/spack/env/clang/flang " is : : cannot o
pen `' (No such file or directory).
1055 C Object File Type produced by "/home/jehammond/SPACK/spack/lib/spack/env/clang/clang -O2" is : : ELF
64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped.
Information on your system
$ spack debug report
* **Spack:** 0.22.0.dev0 (d210425eef832c10a6c74365068a3262d20b576b)
* **Python:** 3.10.12
* **Platform:** linux-ubuntu22.04-zen4
* **Concretizer:** clingo
Based on the following, I expect FC=/opt/llvm/latest/bin/flang-new to be passed to the MPICH configure script. It is not.
~/SPACK/spack$ spack compilers
==> Available compilers
-- clang ubuntu22.04-x86_64 -------------------------------------
clang@19.0.0git clang@17.0.6 clang@14.0.0
-- gcc ubuntu22.04-x86_64 ---------------------------------------
gcc@9.5.0 gcc@13.2.0 gcc@12.3.0 gcc@11.4.0 gcc@10.5.0
-- nvhpc ubuntu22.04-x86_64 -------------------------------------
nvhpc@24.3 nvhpc@24.1 nvhpc@23.9 nvhpc@23.7 nvhpc@23.11
~/SPACK/spack$ spack compiler info clang@19.0.0git
clang@19.0.0git:
paths:
cc = /opt/llvm/latest/bin/clang
cxx = /opt/llvm/latest/bin/clang++
f77 = /opt/llvm/latest/bin/flang-new
fc = /opt/llvm/latest/bin/flang-new
modules = []
operating system = ubuntu22.04
~/SPACK/spack$ spack config add "packages:all:compiler:[clang@19.0.0git]"
Additional information
spack-build-env.txt
spack-build-out.txt
General information
- I have run
spack debug reportand reported the version of Spack/Python/Platform - I have run
spack maintainers <name-of-the-package>and @mentioned any maintainers - I have uploaded the build log and environment files
- I have searched the issues of this repo and believe this is not a duplicate
this is certainly problematic...
$ /home/jehammond/SPACK/spack/lib/spack/env/clang/flang -v
/home/jehammond/SPACK/spack/lib/spack/env/clang/flang: 3: eval: Syntax error: word unexpected (expecting ")")
...but i have no idea what the problem is since my compiler configuration appears reasonable...
~/SPACK/spack$ spack config add "packages:all:compiler:[clang@19.0.0git]"
~/SPACK/spack$ spack compilers
==> Available compilers
-- clang ubuntu22.04-x86_64 -------------------------------------
clang@19.0.0git clang@17.0.6 clang@14.0.0
-- gcc ubuntu22.04-x86_64 ---------------------------------------
gcc@9.5.0 gcc@13.2.0 gcc@12.3.0 gcc@11.4.0 gcc@10.5.0
-- nvhpc ubuntu22.04-x86_64 -------------------------------------
nvhpc@24.3 nvhpc@24.1 nvhpc@23.9 nvhpc@23.7 nvhpc@23.11
~/SPACK/spack$ spack compiler info clang@19.0.0git
clang@19.0.0git:
paths:
cc = /opt/llvm/latest/bin/clang
cxx = /opt/llvm/latest/bin/clang++
f77 = /opt/llvm/latest/bin/flang-new
fc = /opt/llvm/latest/bin/flang-new
modules = []
operating system = ubuntu22.04
$ /opt/llvm/latest/bin/flang-new -v
flang-new version 19.0.0git (https://github.com/llvm/llvm-project.git 05c1447b3eabe9cc4a27866094e46c57350c5d5a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/llvm/latest/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/11
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/12
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/12
Candidate multilib: .;@m64
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda, version
You can't run the compiler wrapper outside the build env. It used explain that in an error message, but a recent change broke that.
To run stuff inside the build env, open a shell like this:
$ spack build-env gmake%clang@19 -- bash
# flang -v
And for checking the effective compiler call:
# SPACK_TEST_COMMAND=dump-args flang hello.f
will probably tell you flang-new is called.
You can also run the (failing) build with spack -d install ... and it'll dump compiler wrapper input args to spack-cc*.in.log and effective compiler call to spack-cc*.out.log in the current working dir.
this all looks fine. i was hoping something here would go wrong so i'd understand what is broken when MPICH is built.
$ spack build-env gmake%clang@19 -- bash
$ flang -v
flang-new version 19.0.0git (https://github.com/llvm/llvm-project.git 05c1447b3eabe9cc4a27866094e46c57350c5d5a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/llvm/latest/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/11
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/12
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/12
Candidate multilib: .;@m64
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda, version
$ SPACK_TEST_COMMAND=dump-args flang hello.f
/opt/llvm/latest/bin/flang-new
-march=znver4
-mtune=znver4
-Wl,--disable-new-dtags
-Wl,-rpath,/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/gmake-4.4.1-tf3wgc2wkychmb4fpjpmpzf4bcenduwh/lib
-Wl,-rpath,/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/gmake-4.4.1-tf3wgc2wkychmb4fpjpmpzf4bcenduwh/lib64
-Wl,-rpath,/opt/llvm/latest/lib/x86_64-unknown-linux-gnu
hello.f
Maybe one of those files has details:
find $(spack location -S) -name config.log
Thanks! I should have looked more carefully at the logs.
flang-new is obnoxious about not supporting flags and then failing when it sees them. I'll try to fix this.
configure:38497: /home/jehammond/SPACK/spack/lib/spack/env/clang/flang -o conftest -L/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/clang-19.0.0git/ucx-1.16.0-kjsupd2vmqn5snejiuvjp3nfonkt5mkm/lib -L/home/jehammond/SPACK/spack/opt/spack/linux-ubuntu22.04-zen4/gcc-13.2.0/hwloc-2.9.1-hwbdfbjmmyakcb4kjki7mdqa7nagj5v7/lib conftest.f pac_conftest.o >&5
flang-new: error: unknown argument: '-mtune=znver4'
flang-new: error: unknown argument: '-fallow-argument-mismatch'
I have solved this. Part of it is a hack but I don't see a better way for now.