ETHOS | Computing

The Enabling Technologies for High-Order Simulations (ETHOS) project performs research of fundamental mathematical technologies for next-generation high-order simulations algorithms. The current focus is on advancing the theoretical understanding and practical utility of unstructured meshes with arbitrarily high-order curvilinear elements by researching key challenges associated with high-order mesh quality optimization and simulation-driven adaptivity.

With support from the Applied Mathematics research program in the Department of Energy (DOE) Office of Science, the ETHOS team performs fundamental research that is broadly applicable to many high-order simulation approaches, including high-order finite elements and tightly coupled arbitrary Lagrangian-Eulerian (ALE) simulations of interest to the DOE. Our research builds on the Target-Matrix Optimization Paradigm (TMOP) and nonlinear variational minimization to develop new node-movement strategies for high-order mesh quality optimization and adaptation in both purely geometric settings, as well as in the context of a given physical simulation.

Project summary

High-order meshes are difficult to control

High-order simulations rely on high-order meshes, and bad mesh quality leads to small time steps and applications failures.
In addition to good geometric quality, applications can require high-order mesh adaptivity to dynamic simulation features.

Our solution: Develop theory that rigorously defines high-order mesh quality

We have extended the Target Matrix Optimization Paradigm (TMOP) by Knupp to high-order meshes, defining pointwise quality based on sub-zonal information, and optimizing the mesh node positions with respect to an aggregated quality measure. The TMOP variational minimization technique is based on optimizing an objective function that depends on a quality metric measuring the difference between the current and target geometric parameters of the mesh.
We are currently exploring (nonlinear) solvers for the global optimization problem, incorporating research in constrained optimization, linear and nonlinear solvers, and preconditioners.
We are developing the TMOP-based variational formulation for the upcoming exascale architectures.
We are also interested in mesh generation, surface optimization, hrp-refinement, interface fitting, and more.

Impact: Mesh optimization is relevant to a wide range of applications. Some innovations under the ETHOS project include:

Target construction: Methods for systematic generation of geometric target parameter values (size, aspect-ratio, skew, and orientation) that vary over mesh elements or sample points. Permits flexible and precise control over essential geometric quality features of mesh, tailored to any specific application.
Simulation-driven r-adaptivity: Simulation-based adaptivity to optimize meshes during runtime using the solution of the PDE. This approach has already demonstrated significant impact in improving solution accuracy and computational efficiency for ALE hydrodynamics.
hr-adaptivity: TMOP-based energy estimator to augment r-adaptivity with anisotropic mesh refinement. This allows us to satisfy size and aspect-ratio targets in cases where the topology of the original mesh does not allow the geometric targets to be met.
Interface fitting: Penalty-based formulation to align elements in a mesh to internal interfaces. This approach allows us to take Cartesian meshes and align them to domains with complicated curvilinear interfaces.
Adaptive node limiting: Penalty-based formulation to optimize the geometric quality of the mesh while minimizing the change associated with a discrete function defined on the original mesh. This functionality was inspired by multi-material applications where it is crucial to preserve material mass for each zone while optimizing the mesh quality.
TMOP-based adaptivity has been used for a wide variety of applications ranging from inertial confinement fusion, where symmetry preservation is important, to tokamak magnetohydrodynamics, where mesh alignment with the magnetic fields helps accelerate the convergence of the solution.
The ETHOS algorithms are freely available on the MFEM website.

Team

Vladimir Tomov (LLNL)
Patrick Knupp (Dihedral, LLC)
Tzanio Kolev (LLNL) – Project Leader
Veselin Dobrev (LLNL)
Ketan Mittal (LLNL)

Software

Many of the high-order mesh optimization algorithms developed in the ETHOS project are freely available in a user-friendly form in the MFEM finite element library.
See in particular the Mesh Optimizer miniapp in the miniapps/meshing directory and the TMOP sources in the fem directory.

Publications

J-S. Camier, V. Dobrev, P. Knupp, Tz. Kolev, K. Mittal, R. Rieben, and V. Tomov, Accelerating high-order mesh optimization using finite element partial assembly on GPUs. Journal of Computational Physics, 474, (2023).
P. Knupp, Worst case mesh quality in the target matrix optimization paradigm. Engineering with Computers, 38(6), (2022).
P. Knupp, Geometric parameters in the target matrix mesh optimization paradigm. Partial Differential Equations in Applied Mathematics, 100390, (2022).
V. Dobrev, P. Knupp, Tz. Kolev, K. Mittal, and V. Tomov. hr-Adaptivity for nonconforming high-order meshes with the target matrix optimization paradigm. Engineering with Computers, 38(4), 3721-3737, (2022).
P. Knupp, Tz. Kolev, K. Mittal, V. Tomov, Adaptive Surface Fitting and Tangential Relaxation for High-Order Mesh Optimization. International Meshing Roundtable, (2021).
V. Dobrev, P. Knupp, Tz. Kolev, K. Mittal, V. Tomov, hr-adaptivity for nonconforming high-order meshes with the target-matrix optimization paradigm. Engineering with Computers, (2021).
P. Knupp, Metric Type in the Target-matrix Mesh Optimization Paradigm (No. LLNL-TR-817490). Lawrence Livermore National Laboratory, Livermore, CA (2020).
V. Dobrev, P. Knupp, Tz. Kolev, K. Mittal, R. Rieben, and V. Tomov, Simulation-Driven Optimization of High-Order Meshes in ALE Hydrodynamics. Computers & Fluids, 104602, (2020).
P. Knupp, Target formulation and construction in mesh quality improvement (No. LLNL-TR-795097). Lawrence Livermore National Laboratory, Livermore, CA, (2019).
V. Dobrev, P. Knupp, Tz. Kolev, and V. Tomov, Towards Simulation-Driven Optimization of High-Order Meshes by the Target-Matrix Optimization Paradigm, In International Meshing Roundtable (pp. 285-302). Springer, Cham (2019).
V. Dobrev, P. Knupp, Tz. Kolev, K. Mittal, and V. Tomov, The Target-Matrix Optimization Paradigm for High-Order Meshes, SIAM J. Sci. Comput., 41(1), pp. B50–B68, (2019).
R. Anderson, V. Dobrev, Tz. Kolev, R. Rieben, and V. Tomov, High-Order Multi-Material ALE Hydrodynamics, SIAM J. Sci. Comp., Vol. 40(1), pp. B32–B58, (2018).
P. Knupp, Introducing the target-matrix paradigm for mesh optimization by node movement, Engineering with Computers 28(4), pp. 419–429, (2012).

Presentations

Gallery

Benchmark featuring Kershaw meshes shows that finite element partial assembly on GPUs leads to significant speed-up (30-40x) in comparison to full assembly on CPUs. [paper]