Software releases on GitHub
SAMRAI (Structured Adaptive Mesh Refinement Application Infrastructure) is an object-oriented C++ software library that enables exploration of numerical, algorithmic, parallel computing, and software issues associated with applying structured adaptive mesh refinement (SAMR) technology in large-scale parallel application development. SAMRAI provides software tools for developing SAMR applications that involve coupled physics models, sophisticated numerical solution methods, and which require high performance parallel computing hardware. SAMRAI enables integration of SAMR technology into existing codes and simplifies the exploration of SAMR methods in new application domains. Due to judicious application of object-oriented design, SAMRAI capabilities are readily enhanced and extended to meet specific problem requirements. The SAMRAI team collaborates with application researchers at LLNL and other institutions. These interactions motivate the continued evolution of the SAMRAI library.
Two central design goals have been the focus of SAMRAI development from its beginning: (1) to allow customization and extension of patch-based data structures while reusing the existing parallel communication infrastructure; and (2) to provide a flexible algorithmic framework to explore solution methods for new applications. SAMRAI exploits many modern software design and implementation techniques to increase flexibility and extensibility. Object-oriented design features, such as "design patterns," are ubiquitous in the SAMRAI library. Careful attention to software design and implementation has enabled SAMRAI to be used in multiple, diverse application codes and exploit a significant amount of design and code re-use across those applications. Our aim is to assemble application codes from algorithmic and numerical "building blocks" both found in the framework and developed specifically for individual applications. Many software elements in SAMRAI may be enhanced or altered without changing underlying framework source code or re-compiling the library. For example, new patch data types, such as particles for discrete methods, can be introduced without modifying existing parallel communication routines, data structures, or algorithm support. Also, smaller parts of larger algorithms, such as load balance routines and time integration methods, may be replaced with application-specific versions without disrupting other software units that interact with those entities. This claim is demonstrated by the fact that different application codes based on SAMRAI share a significant amount of higher-level algorithmic code; i.e., above the level of basic mesh and data structure classes.
SAMRAI Functionality and Library Organization
The SAMRAI library provides a rich set of reusable, extensible software components for SAMR application development. The capabilities provided by SAMRAI include:
- Automatic (user-controlled) dynamic mesh refinement
- Uniform, non-uniform, and user-defined load balancing
- Various array data types for representing simulation quantities on a mesh with different centerings (e.g., node, face, cell, etc.), and support for data defined on irregular sets of cell indices
- Support for user-defined data on a SAMR mesh hierarchy with full to parallel data communication functionality (w/o recompiling the library)
- Customizable adaptive meshing and integration algorithms (via object-oriented composition and inheritance)
- Support for meshes with arbitrary spatial dimension
- Multiblock AMR allowing irregular block connectivity
- Interfaces to solver libraries, such as hypre, PETSc, and SUNDIALS
- Flexible parallel restart (HDF5) and input parser
- Tools for measuring performance, gathering statistics
- Visualization support via VisIt
The SAMRAI library is partitioned into a collection of software "packages". Each package is a set of logically-related C++ classes that constitutes a functional role in SAMR application development. See Figure.
The following list briefly describes the functionality provided by each SAMRAI package:
- The Toolbox package provides a collection of basic utility classes that are used throughout the library and in application development. The utilities include: memory management via arena mechanisms; smart pointers, arrays and other container classes; basic MPI classes; tools for managing input and restart files; and event logging, tracing, and timing.
- The Hierarchy package contains patch, patch level, and patch hierarchy containers and support for abstract index space and box calculus operations on which most SAMR patch hierarchy structure and data manipulation operations depend. Interface classes for managing variables and data living on the hierarchy also reside here.
- The Transfer package provides classes used to manage and perform inter-patch data communication on a structured AMR hierarchy. Typical data transfer operations include moving data between patches on the same level and coarsening and refining data between different levels. The package also contains base classes for operators that refine and coarsen data spatially, interpolate data in time, and implement physical boundary conditions
- The Patchdata package provides support for various patch data types used to represent simulation data. Data types found here include a variety of array-based quantities (cell-centered, node-centered, face-centered, etc.) as well as an "index" type for managing data associated with an arbitrary collection of cell indices (e.g., irregular structures like embedded boundaries or lists of particles).
- The Math Operations package supports basic arithmetic and other operations, such as dot products and norms that are often required for vector kernels in solver libraries. These operations apply to data living on a single patch, a single level, or a subset of levels within an SAMR patch hierarchy. This support is provided for all array-based data types in SAMRAI.
- The Mesh package contains classes that support construction and adaptive regridding of a SAMR patch hierarchy. Routines include clustering of tagged cells into box regions and load balancing based on either spatially-uniform or non-uniform workload estimates.
- The Algorithm package houses classes that are useful for constructing solution algorithms associated with certain types of PDE systems, such as local time cycling on hierarchy levels.
- The Solvers package contains support for applying linear and nonlinear solver methods to problems defined on an AMR patch hierarchy. Vector structures for SAMR patch hierarchy data are provided here. Also included are SAMRAI interfaces to the PETSc and KINSOL nonlinear solver libraries, the PVODE ODE solver library, and the hypre preconditioning library. An FAC algorithm for solving Poisson problems on an AMR hierarchy is also included.
- The Geometry package supports specific coordinate systems on patches in an SAMR patch hierarchy. This includes a variety of spatial refine/coarsen operators and time interpolation operators for array-based patch data types.
- The Application Utilities package contains utilities that are useful when construction applications with SAMRAI, such tools for generating visualization data.
SAMRAI Computational Environment
SAMRAI is used and supported primarily on the high performance parallel computing platforms at LLNL. It is regularly tested on these systems.
Full SAMRAI functionality relies on several external software libraries and tools such as:
- Visualization software. The SAMRAI distribution includes a homegrown visualization tool called Vizamrai. More extensive visualization capabilities are provided by the VisIt system.
- The HDF5 library is required for restart functionality and generating VisIt visualization files.
- Parallelism requires some form of MPI library, such as MPICH or hardware vendor supplied.
- The linear and nonlinear solver interfaces within SAMRAI require the appropriate solver package. These include PETSc, KINSOL and PVODE, and hypre.
- SAMRAI provides capabilities for using the TAU and VAMPIR performance analysis tools.
- SAMRAI compilation requires configuration using the autoconf system.
- Generation of the SAMRAI source code documentation requires the Doxygen system