Add support for OpenACC

[Nicholas Chaimov]

TAU now supports OpenACC with PGI/nvhpc compilers. To test this in TAU:

• Log in to a system that has compatible compilers, such as voltar.nic.uoregon.edu
• Load the appropriate compiler module, e.g., module load nvhpc/20.9
• Load CUDA: module load cuda/11.4
• Build TAU with -openacc -cc=nvc -c++=nvc++ -fortran=nvfortran -cuda=/packages/cuda/11.4 -bfd=download -unwind=download; make install -j8
• cd examples/openacc
• make
• tau_exec -T serial,acc,nvhpc,cupti -openacc -cupti ./jacobi

We want to support this TAU configuration in TAU Commander. Adding this support will involve:

• Adding a boolean OpenACC argument to the Application model, indicating whether the application uses OpenACC; see for example

  taucmdr/packages/taucmdr/model/application.py

  Lines 75 to 87 in eae3ae7

   'openmp': { 
       'type': 'boolean', 
       'description': 'application uses OpenMP', 
       'default': False, 
       'argparse': {'flags': ('--openmp',)}, 
       'rebuild_required': True 
   }, 
   'pthreads': { 
       'type': 'boolean', 
       'description': 'application uses pthreads', 
       'default': False, 
       'argparse': {'flags': ('--pthreads',)}, 
       'rebuild_required': True 

  for other boolean options already in the Application model.
• Adding a similar argument to the Measurement to control whether to measure OpenACC. A constraint should be on this requiring that the Application use OpenACC and that a compatible compiler be used. See

  taucmdr/packages/taucmdr/model/measurement.py

  Lines 183 to 193 in eae3ae7

   'mpi': { 
       'type': 'boolean', 
       'default': False, 
       'description': 'use MPI library wrapper to measure time spent in MPI methods', 
       'argparse': {'flags': ('--mpi',)}, 
       'compat': {True: 
                  (Target.require(MPI_CC.keyword), 
                   Target.require(MPI_CXX.keyword), 
                   Target.require(MPI_FC.keyword), 
                   Application.require('mpi', True), 
                   Measurement.exclude('baseline', True))}, 

  for an example of constraints on a Measurement attribute.
• Modifying tau_installation.py to build TAU with -openacc when the Application specifies OpenACC.
• Modifying tau_installation.py to expect the tag acc when TAU is built with OpenACC.
• Modifying tau_installation.py to run applications through tau_exec with -openacc when the Measurement specifies OpenACC.
• Adding a test of OpenACC support to the test suite and verifying that it passes on a system that supports OpenACC. The test should be configured so that it will not run if the system doesn't have an appropriate compiler.

[Nicholas Chaimov]

One other probable task for this: the PGI compilers have been re-named and are now the NVHPC compilers. They used to be in executables named pgcc, pgCC, and pgfortran but are now nvc, nvc++, and nvfortran. The compiler detection logic in TAU Commander will probably need to be updated to support the new names. The existing code for detection of PGI compilers is at:

taucmdr/packages/taucmdr/cf/compiler/host.py

Lines 48 to 49 in eae3ae7

 PGI = HOST_COMPILERS.add('PGI', family_regex=r'The Portland Group|NVIDIA CORPORATION', 
                          CC='pgcc', CXX=('pgCC', 'pgc++', 'pgcxx'), FC=('pgfortran', 'pgf90', 'pgf77')) 

Also, the tag that is used with TAU is now nvhpc instead of pgi when the new versions are used, so the logic to construct a tag from the compiler will also need to be updated. That is at:

taucmdr/packages/taucmdr/cf/software/tau_installation.py

Lines 1037 to 1039 in eae3ae7

 def _compiler_tags(self): 
     return {host_compilers.INTEL: 'intel' if self.tau_magic.operating_system is CRAY_CNL else 'icpc', 
             host_compilers.PGI: 'pgi'} 

[Nicholas Chaimov]

Added in #416. Thanks @vikram8128!