zfp is a BSD licensed open-source library for compressed floating-point arrays that support high throughput read and write random access. zfp is primarily written in C and C++ but also has Python, Fortran, and other bindings. zfp is loosely based on the algorithm described in the following paper:

Peter Lindstrom, "Fixed-Rate Compressed Floating-Point Arrays," IEEE Transactions on Visualization and Computer Graphics, 20(12): 2674–2683, December 2014, doi:10.1109/TVCG.2014.2346458.

zfp was designed to achieve high compression ratios and therefore uses lossy but optionally error-bounded compression. Bit-for-bit lossless compression of integer and floating-point arrays is also supported. zfp provides high-quality compression and is fast, achieving throughputs of up to 2 GB/s per CPU core and 800 GB/s aggregate throughput on recent GPUs. zfp supports several different back-ends, including OpenMP, CUDA, and HIP. An FPGA implementation is also available.

zfp is hosted as open source on GitHub and can also be installed as conda, spack, and RPM packages. Separate conda and pip packages are available for zfPy, the Python interface to zfp. HDF5 users may be interested in the H5Z-ZFP compression plugin. zfp is supported by software tools and libraries like Intel®IPP, HDF5®, ADIOS, BLOSC, E4S, MVAPICH2, OpenInventor™, TTK, VTK-m, and Zarr.

zfp development is supported by the U.S. Department of Energy's Exascale Computing Project, by the Advanced Scientific Computing Research Program, and by the Advanced Simulation and Computing Program. Advanced features such as variable-rate random-access arrays were investigated on LLNL's Variable Precision Computing Project.

zfp adaptivity simulation
Figure: Illustration of zfp's ability to vary the compression ratio to any desired setting, from 10:1 on the left to 250:1 on the right, where the partitioning of the data set into small blocks is evident. The visualization shows the density field from a Rayleigh-Taylor instability simulation.