In early November, Lawrence Livermore National Laboratory (LLNL) hosted the High Performance Storage System (HPSS) User Forum, marking the first time in the collaboration’s 33-year history the global HPSS user community gathered in Livermore.

HPSS is a data archive and retrieval system for managing upwards of 100s of petabytes of data on disk and tape (which is often housed in robotic tape libraries). It provides hierarchical storage management services to move data between different tiers of storage. Originally conceptualized and developed primarily at Livermore, HPSS is one of the Department of Energy’s (DOE’s) longest running multilaboratory and industry collaborations. It is the result of over three decades of collaboration among five DOE laboratories and IBM.

“The DOE labs’ broad mission space and vast data assets required extraordinary data storage solutions well before industry confronted similar challenges,” explains Todd Heer, the HPSS Executive Committee representative for LLNL. “HPSS was created to gather similarly chartered organizations around a single solution for addressing this ever-present challenge at scale. It was built on the concept of a shared software development model long before the open source initiative. But our goals today are the same as they were at the system’s inception in the 1990s—to keep scientists from having to wrestle with their data and to be responsive to user needs.”

Today, HPSS is adopted by some of the largest, most notable data-crunching organizations in the world in research, government, and academia. Livermore remains a primary steward and co-developer.

The HPSS collaboration makes it a priority to regularly gather its users and developers to get everyone on the same page. Traditionally, the HPSS User Forum (HUF) is held once a year for five days, nearly always hosted by a deployment site. HUF offers the global HPSS community—existing HPSS users, future users, developers, system engineers, support personnel, project managers, and HPSS leaders—an opportunity to collectively discuss best practices, preview upcoming features and software roadmaps, expand professional networks, and share best practices in the uniquely challenging space of exascale computing—and more recently AI—data storage.

“It’s great to hear about HPSS’s successes during the HUF,” says Heer, “but the more important outcomes are the candid discussions about areas for improvement. We walk away energized from the collaboration and with a list of major feature requests and bug fixes.”

This year’s event was held at University of California Livermore Collaboration Center. The forum drew 58 participants from across the globe, with most attending in person and a few joining remotely. “We deliberately scheduled this year’s HUF later in the year than usual, so that we’d go back-to-back with two other major HPC events—the IBM Future of Tape Summit and SC25. We wanted our international participants to be able to attend all the fun in only one overseas trip,” explains Geoff Cleary, Scalable Storage Group leader and HUF cohost.

The meeting format is designed to maximize expert interaction, particularly between HPSS deployment specialists, development teams, and site administrators. The LLNL-hosted HUF included interactive polling throughout the day to spark discussions among the entire attendee group as well as social gatherings and dinners off site.

This year’s theme—Disruptive Tech— reflects the evolving landscape of data storage, with sessions exploring the critical intersection of on-premises, scaled-out, streaming-tape deployments in consideration of transformative forces such as DevOps, containerization, AI and machine learning, data-centric computing, digital transformation/engineering, and cloud. LLNL’s Brian Spears, director of LLNL’s AI Innovation Incubator and technical director of DOE’s Genesis Mission, gave a keynote on the intersection of AI and storage.

Heer and Cleary’s hosting duties were supported by an exceptional coordination team including Ali Bischoff, Miranda Wang, Carly Bond, and Jennifer Humphreys.

Next year, the HUF will be hosted by Japan’s national space agency, JAXA, one of the HPSS deployment sites.

—Deanna Willis