MACSio: A Multi-purpose, Application-Centric, Scalable I/O Proxy Application
I/O Performance Characteristics
We also show (in yellow) an I/O request histogram for a typical restart dump as a percentage of total bytes in the dump. A given bar indicates the percent of total bytes in the dump that were transferred at that request size. In fact, we show two different categories of requests. Those that originate from the application itself (solid yellow) as well as those that originate from one or more of the lower layers (hashed yellow) in the I/O stack on behalf of the application (typically metadata associated with the abstractions). In this example a majority of the smaller requests originated from the application itself. This suggests that the application could be adjusted to aggregate many of its smaller requests into a single larger request and experience improved performance. By appropriate use of timing and request size information gathered from within MACSio and its I/O plugins, this kind of detailed application I/O emulation and performance analysis is possible.
Download MACSio at GitHub
MACSio design document
“Replicating HPC I/O workloads with proxy applications” | 2016 1st Joint International Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems (PDSW-DISCS) | James Dickson, Steven Wright, Stephen Jarvis (University of Warwick), Satheesh Maheswaran, Andy Herdman (UK Atomic Weapons Establishment), Mark Miller (Lawrence Livermore National Laboratory)