Livermore Scientists Win Best Paper Award
Livermore scientists Erik Draeger, Xavier Andrade, Abhinav Bhatele, and Alfredo Correa, together with colleagues from the IBM Thomas J. Watson Research Center and the University of Illinois at Urbana–Champaign, have won a Best Paper Award (Applications Track) for their paper “Massively Parallel First-Principles Simulation of Electron Dynamics in Materials.” The team will be presented with their award at the International Parallel and Distributed Processing Symposium (IPDPS) 2016, taking place this May 23–27. IPDPS provides a forum in which a unique international gathering of computer scientists from around the world present their latest research findings in all aspects of parallel computation.
The winning paper presents the demonstration of a highly scalable, parallel implementation of first-principles electron dynamics coupled with molecular dynamics (MD). This capability will enable first-principles simulation of transport properties such as electrical and heat conductivity as well as simulation of electron diffusion under high currents and simulation of electronic stopping power, particularly in the extreme non-linear domain where perturbative theories cannot be used. “Electron dynamics makes it possible to explore highly excited states of matter with computer simulations,” said Alfredo Correa, who leads the science applications of the project. For this study in particular, the team generated simulations with the help of a code called Qbox, a first-principles molecular dynamics code that is used to compute the electronic structure of atoms, molecules, solids, and liquids within the Density Functional Theory (DFT) formalism.
The paper describes how the team used optimized kernels, network topology aware communication, and full distribution of all terms in the time-dependent Kohn-Sham equation to demonstrate unprecedented time to solution for disordered aluminum systems of 2,000 atoms (22,000 electrons) and 5,400 atoms (59,400 electrons), with wall clock time as low as 7.5 seconds per MD time step.
Despite a significant amount of non-local communication required in every iteration, the team achieved excellent strong scaling and sustained performance on the Sequoia Blue Gene/Q supercomputer. They obtained up to 59% of the theoretical sustained peak performance on 16,384 nodes and performance of 8.75 petaFLOPS (43% of theoretical peak) on the full 98,304-node machine (1,572,864 cores).
Other avenues of research made available by scalable explicit electron dynamics include the mechanism of light absorption and energy transfer within photosynthetic complexes—which requires simulations of thousands of atoms and many thousands of time steps—and electromigration, or the displacement of atoms in a conductor due to the momentum transfer from a current of electrons. Scalable explicit electron dynamics also allows for the study of phenomena beyond the reach of standard first-principles MD, especially materials subject to strong or rapid perturbations such as pulsed electromagnetic radiation, particle irradiation, or strong electric currents. “This is an exciting new capability that lets us study a whole new class of materials properties,” said Draeger, lead author. “We’re eager to apply these advances to important problems of interest here at the Lab.”
As supercomputers continue to grow more powerful, it is predicted that the trend toward heterogeneity and complexity will also continue, with Qbox code’s heavy reliance on general kernels providing a clear path to take full advantage of new hardware architectures such as the next generation of graphics processing unit architectures.