Supercomputer Achieves a Thousand Trillion Calculations Per Second

The National Institute for Computational Sciences’ (NICS’s) Cray XT5 supercomputer—Kraken—has been upgraded to become the first academic system to surpass a thousand trillion calculations a second, or one petaflop, a landmark achievement that will greatly accelerate science and place Kraken among the top five computers in the world.

Managed by the University of Tennessee (UT) for the National Science Foundation (NSF), the system came online Oct. 5 with a peak performance of 1.03 petaflops. It features more than 16,000 six-core 2.6-GHz AMD Istanbul processors with nearly 100,000 compute cores.

In addition, an upgrade to 129 terabytes of memory (the equivalent of more than 13 thousand movies on DVD) effectively doubles the size of Kraken for researchers running some of the world’s most sophisticated 3-D scientific computing applications. Simulation has become a key tool for researchers in a number of fields, from climate change to materials.

“At over a petaflop of peak computing power, and the ability to routinely run full machine jobs, Kraken will dominate large-scale NSF computing in the near future,” said NICS Project Director Phil Andrews. “Its unprecedented computational capability and total available memory will allow academic users to treat problems that were previously inaccessible.”

For example, understanding the mechanism behind the explosion of core-collapse supernovas will reveal much about our universe (these cataclysmic events are responsible for more than half the elements in the universe). Essentially three phenomena are being simulated to explore these explosions: hydrodynamics, nuclear burning or fusion, and neutrino transport, said UT astrophysicist Bronson Messer.

At the terascale, or trillions of calculations per second, Messer and his team were forced to simulate the star in 1-D as a perfect sphere and with unrealistic fusion physics. “Now, however, we are getting closer to physical reality,” said Messer. “With petascale capability, we can simulate all three phenomena simultaneously with significant realism. This brings us closer to understanding the explosion mechanism and being able to make meaningful predictions.”

From the physical makeup of the universe to the causes of global warming to the roles of proteins in disease, Kraken’s increased computing muscle will reach far and wide.

As the main computational resource for NICS, the new system is linked to the NSF-supported TeraGrid, a network of supercomputers across the country that is the world’s largest computational platform for open scientific research.

The system and the resulting NICS organization are the result of an NSF Track 2 award of $65 million to the University of Tennessee and its partners to provide for next-generation high-performance computing (HPC). The award was won in an open competition among HPC institutions vying to guarantee America’s continued competitiveness through the next generation of supercomputers (systems greater than 10 teraflops and into the petascale).

“While reaching the petascale is a remarkable achievement in itself, the real strides will be made in the new science that petascale computing will enable,” said Thomas Zacharia, NICS principal investigator, professor in electrical and computer engineering at the University of Tennessee and deputy director for science and technology at Oak Ridge National Laboratory. “Kraken is a game changer for research.”