How to analyze code with mpi?

Question

How to analyze code with mpi?

frankroeder opened this issue 7 years ago · comments

Hello,

recently I've tried to analyze code containing mpi with scan-build. If I pass mpicc to --use-cc it won't work.
For the already named example of https://github.com/0ax1/MPI-Checker/tree/master/examples
there are two possibility:

In case of CMake it is required to generate the compilation database with Cmake itself and invoke analyze-build afterwards.
In case of make, its required to set environmental variables like export LDFLAGS=mpicc --showme:link, export CFLAGS=mpicc --showme:compile and export CXXFLAGS=mpicc --showme:compile, before scan-build can be invoked with --use-cc=clang. Otherwise he won't be able the realize "mpi.h".

László Nagy · Answer 1 · Mon Jul 03 2017 03:15:27 GMT+0800 (China Standard Time)

I don't have much experience with MPI. But as I understood from the @0ax1 description, in case of make you can still generate the compilation database with intercept-build and then run analyze-build... As a rule of thumb, I always recommend to run intercept-build and analyze-build instead of scan-build... But give me some time, next week will try to make some experiment with it.

Alexander Droste · Answer 2 · Mon Jul 03 2017 05:10:45 GMT+0800 (China Standard Time)

Currently, both scan-build and intercept-build do not work. intercept-build simply produces an empty compile_commands.json whereas scan-build gives an empty report . When glancing at compilation.py my suspicion is that it might be because the MPI compilers (mpicc, mpic++ etc.) are treated as regular compilers but not as compiler wrappers which is what they are. In particular, they also implicitly set MPI related link and include flags which is why using --use-cc or --use-c++ might lead to headers or symbols not being found.

Alexander Droste · Answer 3 · Mon Jul 03 2017 05:12:16 GMT+0800 (China Standard Time)

Thanks for having a look at this! 👍

László Nagy · Answer 4 · Sat Jul 08 2017 23:21:13 GMT+0800 (China Standard Time)

Hey Guys, was playing with this. Not sure I've fixed it. Need your feedback. The change is on fix branch. (Did not merged to master yet.) The intercept-build produces a non empty compilation database for MPI compilers too. (Was running against @0ax1 example project.)

Alexander Droste · Answer 5 · Mon Jul 10 2017 16:23:35 GMT+0800 (China Standard Time)

Hey László, thanks again for looking into this. I just gave the fix branch a try but somehow intercept-build still produces an empty compilation database. Which MPI version did you use? I tested it with MPICH 3.2 and Open MPI 2.1.1 . Did you add any flags to intercept-build make?

László Nagy · Answer 6 · Mon Jul 10 2017 18:16:46 GMT+0800 (China Standard Time)

Thanks @0ax1 , was testing with MPICH 3.2 on Fedora 25. No extra flags. Used your repo examples. Will look at it again and try with OPENMPI too.

László Nagy · Answer 7 · Wed Aug 09 2017 18:55:47 GMT+0800 (China Standard Time)

Sorry to be late with this... I did re-check with OpenMPI and MPICH on Fedora 26. Both cases it was generate a non empty compilation database, with the expanded flags from mpicc -show command.
This is released as 2.0.8 on master.

Alexander Droste · Answer 8 · Sat Aug 12 2017 15:29:40 GMT+0800 (China Standard Time)

No problem. Great to hear that it is working with multiple MPI implementations. I was wondering why it still wasn't working on my current macOS setup but then remembered from your libEAR page that it is necessary to disable system integrity protection on macOS in order to produce a compilation database. So after disabling system integrity protection, intercept-build also works on macOS. I think it would be great if this information would be in the README of this repo. What's strange is that scan-build still doesn't produce a report (scan-build: Removing directory ... because it contains no report.). Do you have an idea why this is the case?

One thing I noticed about 5af4396 is that the open_mpi_compiler test was commented out instead of removed: 5af4396#diff-e1dbf5104a4e733ff3e792a2b25272cbR54. Was that intentional?

Thanks again for making this work!

László Nagy · Answer 9 · Sun Aug 13 2017 10:45:24 GMT+0800 (China Standard Time)

@0ax1 thanks for your comment! You touched multiple issues, I hope I can answer them all. ;)

Because now the mpi compiler wrappers generates random flags (in a sense that it's not deterministic for every installation), it is hard to write a unit test which asserts expected flags. I might need to rework that for proper dependency injection, to make it properly testable. Until then it is commented out.

The README file talks about different behavior of intercept-build depends on the OS security protection. When dynamic linker pre-load is not available it falls back to compiler wrappers. The wrapper solution is sensitive about the build system. The build system shall use CC and CXX environment variable to call the compiler. If it does intercept-build will work as expected and be able to build the compilation database. (Same story about the scan-build.)

Alexander Droste · Answer 10 · Mon Aug 14 2017 15:11:41 GMT+0800 (China Standard Time)

I think you did :). Good to know how these aspects make a difference.

Ramakrishnan · Answer 11 · Thu May 23 2019 23:25:38 GMT+0800 (China Standard Time)

Sorry to be late with this... I did re-check with OpenMPI and MPICH on Fedora 26. Both cases it was generate a non empty compilation database, with the expanded flags from mpicc -show command.
This is released as 2.0.8 on master.

Hi,

I am using intercept-build to generate the compilation data base. Our code use OpenMPI. I use the following line to generate the .JSON file:

intercept-build --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make

JSON file created. But the created file is empty. Could you tell me how to use the intercept-build with OpenMPI?

László Nagy · Answer 12 · Fri May 24 2019 08:55:58 GMT+0800 (China Standard Time)

Hey @Raamkrishnan , the command you've used looks good to me. But the devil is in the details... Feel free to create another ticket.

Which OS do you use? Which version of this tool you are using? How did you installed?

The other relevant information can be, how is your project looks like? What build system it's using? Does the build respect the CC and CXX environment variables for picking the compiler?

You can also run the same command with an extra -vvvv append to the flags. This will make intercept-build write verbose logs, which you can attach to the ticket.

Ramakrishnan · Answer 13 · Fri May 24 2019 19:43:18 GMT+0800 (China Standard Time)

Thank you for your reply. I use Ubuntu 18.04.2 LTS. I am using scan-build-2.0.16 and I downloaded that using pip3 (python3.6). I use openmpi-1.10.7.

I want to generate the compilation data base for IBAMR code (open source code to simulate fluid flows). Then I will read this database in some editors so that I can debug the code. This project build system is based on GNU Make. This project relies upon several other open-source libraries such as PETSc, SAMRAI, LibMesh.

I am just trying to generate compilation database for PETSc now. I used the following commands:

**intercept_make="make"

if [ $SETUP_PETSC -eq 1 ]; then
echo "setting up petsc"
cd $PETSC_DIR
echo "working directory is: $PWD"
echo "PETSC_ARCH is: $PETSC_ARCH"
make PETSC_ARCH=$PETSC_ARCH clean
rm $PWD/compile_commands.json
intercept-build --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make PETSC_ARCH=$PETSC_ARCH all
compdb -p . list > compile_commands-new.json
cp $PWD/compile_commands-new.json $PROJECT_ROOT/petsc-compile_commands.json
if [ $REBUILD_LIBRARIES -eq 1 ]; then
make PETSC_ARCH=$PETSC_ARCH all
fi
cd $PROJECT_ROOT
fi**

compile_commands.json file is created. but it is empty. Like you suggested, I also ran with the following line:

intercept-build -vvvv --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make

The results is same. I am attaching the output of above lines in the terminal. Please let me know your suggestions

output.TXT

László Nagy · Answer 14 · Fri May 24 2019 20:20:17 GMT+0800 (China Standard Time)

Thanks to come back. Was try to read the PETSc make file, but failed to understand the relevant details. :/

What I noticed that you forced the tool to use the "compiler wrapper" to intercept the compilation commands with the --override-compiler flag. But the build not necessarily pick that up. Is there any reason why you want to run like this?

I would recommend instead the intercept-build -vvvv $intercept_make command. Could you try this?

Ramakrishnan · Answer 15 · Sat May 25 2019 02:23:53 GMT+0800 (China Standard Time)

Thank you for your suggestion. It works for me. I used this command:

intercept-build --use-cc=mpicc --use-c++=mpicxx $intercept_make

It creates non empty compilation data base. Could you tell me what was happening when I used the --override-compiler flag?

László Nagy · Answer 16 · Sat May 25 2019 07:35:02 GMT+0800 (China Standard Time)

I'm glad that it worked... for more details please read the readme file in this project. It has a section to explain the differences between the modes... I think specifying the --use-cc and the --use-cxx is also not necessary. If you are interested to make the command shorter.

Ramakrishnan · Answer 17 · Sun May 26 2019 15:48:10 GMT+0800 (China Standard Time)

Yse. --use-cc and --use-cxx are also redundant in this case. I tried it. Just I have to use:

intercept-build $intercept_make

to create the non empty compilation database. Thank you for your suggestions.