Motivation

PMPI: Success Story for HPC Tools

  • Transparently intercept all MPI calls.
  • Achieved by prelinking the tool to the application.
  • Standardized interface.
  • Used by many tools.

Shortcomings of PMPI

  • Supports only single concurrent tool.
  • Requires pre-linking.
  • Prevents tool modularity and code reuse.

Our Approach

Approach
  • Virtualize PMPI interface.
  • Enable dynamic linking of multiple PMPI tools.
  • Extensions for modularity.
  • Reuse existing binary PMPI tools.
  • Allow dynamic tool chain selection.

Architecture

Architecture

Itself Implemented as a PMPI Tool

  • Thin, low-overhead wrapper library.
  • Automatically generated from mpi.h file.
  • Can be linked by default.

PnMPI Core

  • Only activated if some loaded tool intercepts routine.
  • Dynamically created tool chain.

Configuration and Module Loading

  • Module library.
  • Binary rewriter for existing PMPI tools.
  • Configuration file to select modules.

Publish/Subscribe Interface for Tool Services

  • PnMPI-aware modules can register services.
  • Other modules can query services and use them.
  • Can be used to implement generic tool services such as:
    • Datatype walking.
    • Request tracking.
    • Generic communication interception.

Dynamic Tool Chain Selection

  • Each module can point to arbitrary successor module/tool stack.
  • Decision can be done at runtime, e.g., based on routine arguments.
  • Enables dynamic tool scopes and/or tool multiplexing