Topic: Data Compression

An LLNL-led effort in data compression was one of nine projects recently funded by the DOE for research aimed at shrinking the amount of data needed to advance scientific discovery. Under the project — ComPRESS: Compression and Progressive Retrieval for Exascale Simulations and Sensors — LLNL scientists will seek better understanding of data-compression errors.

High-precision numerical data from computer simulations, observations, and experiments is often represented in floating point and can easily reach terabytes to petabytes of storage.

fpzip is a library for lossless or lossy compression of multidimensional floating-point arrays. It was primarily designed for lossless compression.

Highlights include Computation’s annual external review, machine learning for ALE simulations, CFD modeling for low-carbon solutions, seismic modeling, and an in-line floating point compression tool.

zfp is an open-source C/C++ library for compressed floating-point and integer arrays that support high throughput read and write random access.

hzip 1.0.1 is a C++ library for lossless compression of structured and unstructured meshes composed of cells with hypercube topology.