MPI Programs: Debugging

• Finding errors in parallel programs is difficult. This is the sequence for attaching the debugger to an MPI program:
  • mpirun -np N ssrun -hang program prog-args
    • this will stop execution when the shepherd process is created
  • obtain the process id of the program with ps -ef | grep program
  • attach debugger to that process: cvd -pid #procid
  • in cvd select the menus Admin->Multiprocess View->Config->Preferences… and enable “Attach to forked processes”.
  • Continue execution. The Multiprocess View will show a list of processes that will be created at that time. Stop All, change focus, etc.
• Alternatively, MPI program can be instrumented to stop at the desired place (e.g. put read *,dummy in the code) and debugger attached to any thread after the creation.
• Last resort are the print statements in the code.

Previous slide

Next slide

Back to first slide

View graphic version