Can't configure on Frontier with amd/5.7.0
khuck opened this issue · comments
The subject says it all... here's what I am doing:
module reset
module unload perftools-base
module load cmake
module load PrgEnv-amd
module swap amd amd/5.7.0
module swap cray-mpich cray-mpich/8.1.27
module load craype-accel-amd-gfx90a
module unload darshan-runtime
git clone https://github.com/ginkgo-project/ginkgo.git
cd ginkgo
cmake -B `pwd`/build -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=`pwd`/install -D GINKGO_BUILD_OMP=OFF -D GINKGO_BUILD_HIP=ON -D GINKGO_BUILD_HWLOC=OFF -D GINKGO_BUILD_EXAMPLES=OFF -D GINKGO_BUILD_TESTS=OFF -D GINKGO_FAST_TESTS=OFF -D GINKGO_BUILD_BENCHMARKS=OFF -D HIPRAND_PATH=${ROCM_PATH} -D ROCRAND_PATH=${ROCM_PATH} .
I get the following output:
-- The C compiler identification is Clang 17.0.0
-- The CXX compiler identification is Clang 17.0.0
-- Cray Programming Environment 2.7.19 C
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/cray/pe/craype/2.7.19/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Cray Programming Environment 2.7.19 CXX
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/cray/pe/craype/2.7.19/bin/CC - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found hipconfig: /opt/rocm-5.7.0/hip/bin/hipconfig
-- Found MPI_CXX: /opt/cray/pe/craype/2.7.19/bin/CC (found suitable version "3.1", minimum required is "3.1")
-- Found MPI: TRUE (found suitable version "3.1", minimum required is "3.1") found components: CXX
-- Enabling MPI support
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- Could NOT find PAPI (missing: PAPI_LIBRARY PAPI_INCLUDE_DIR sde) (Required is at least version "7.0.1.0")
-- HIP platform set to amd
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- hip::amdhip64 is SHARED_LIBRARY
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- Found ROCTX: /opt/rocm-5.7.0/lib/libroctx64.so
-- Performing Test Ginkgo_C_COVERAGE_SUPPORTED
-- Performing Test Ginkgo_C_COVERAGE_SUPPORTED - Success
-- Performing Test Ginkgo_C_TSAN_SUPPORTED
-- Performing Test Ginkgo_C_TSAN_SUPPORTED - Success
-- Performing Test Ginkgo_C_ASAN_SUPPORTED
-- Performing Test Ginkgo_C_ASAN_SUPPORTED - Success
-- Performing Test Ginkgo_C_LSAN_SUPPORTED
-- Performing Test Ginkgo_C_LSAN_SUPPORTED - Success
-- Performing Test Ginkgo_C_UBSAN_SUPPORTED
-- Performing Test Ginkgo_C_UBSAN_SUPPORTED - Success
-- Performing Test Ginkgo_CXX_COVERAGE_SUPPORTED
-- Performing Test Ginkgo_CXX_COVERAGE_SUPPORTED - Success
-- Performing Test Ginkgo_CXX_TSAN_SUPPORTED
-- Performing Test Ginkgo_CXX_TSAN_SUPPORTED - Success
-- Performing Test Ginkgo_CXX_ASAN_SUPPORTED
-- Performing Test Ginkgo_CXX_ASAN_SUPPORTED - Success
-- Performing Test Ginkgo_CXX_LSAN_SUPPORTED
-- Performing Test Ginkgo_CXX_LSAN_SUPPORTED - Success
-- Performing Test Ginkgo_CXX_UBSAN_SUPPORTED
-- Performing Test Ginkgo_CXX_UBSAN_SUPPORTED - Success
-- Performing Test Ginkgo_HIP_COVERAGE_SUPPORTED
-- Performing Test Ginkgo_HIP_COVERAGE_SUPPORTED - Success
-- Performing Test Ginkgo_HIP_TSAN_SUPPORTED
-- Performing Test Ginkgo_HIP_TSAN_SUPPORTED - Success
-- Performing Test Ginkgo_HIP_ASAN_SUPPORTED
-- Performing Test Ginkgo_HIP_ASAN_SUPPORTED - Success
-- Performing Test Ginkgo_HIP_LSAN_SUPPORTED
-- Performing Test Ginkgo_HIP_LSAN_SUPPORTED - Success
-- Performing Test Ginkgo_HIP_UBSAN_SUPPORTED
-- Performing Test Ginkgo_HIP_UBSAN_SUPPORTED - Success
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Could NOT find VTune (missing: VTune_EXECUTABLE VTune_LIBRARY VTune_INCLUDE_DIR)
-- Could NOT find METIS (missing: METIS_LIBRARY METIS_INCLUDE_DIR)
CMake Error at hip/CMakeLists.txt:103 (hip_add_library):
Unknown CMake command "hip_add_library".
-- Configuring incomplete, errors occurred!
See also "/lustre/orion/world-shared/phy122/khuck/xgc-dependencies/src/ginkgo2/build/CMakeFiles/CMakeOutput.log".
See also "/lustre/orion/world-shared/phy122/khuck/xgc-dependencies/src/ginkgo2/build/CMakeFiles/CMakeError.log".
If I do module swap amd/5.7.0 amd/5.4.3 it configures just fine. I don't know if this is a CMake bug, a hip/rocm bug, or what - but I am submitting it here so you are aware of it...
This is an issue related to the location of HIP_PATH and FindHIP.cmake, which used to point to /opt/rocm/hip and in more recent versions points to /opt/rocm in dpkg installs, it's probably similar in the spack installation.
Thanks, I thought that might be related. Is there a workaround for now?
maybe try export HIP_PATH=$ROCM_PATH/hip and then rebuild ginkgo
that works, thanks!