The U.S. Department of Energy's (DOE) Oak Ridge National Laboratory launched a new era of scientific supercomputing today with Titan, a system capable of churning through more than 20,000 trillion calculations each second—or 20 petaflops—by employing a family of processors called graphic processing units first created for computer gaming. Titan will be 10 times more powerful than ORNL's last world-leading system, Jaguar, while overcoming power and space limitations inherent in the previous generation of high-performance computers.
Oak Ridge National Laboratory is home to Titan, the world’s most powerful supercomputer for open science with a theoretical peak performance exceeding 20 petaflops (quadrillion calculations per second). That kind of computational capability—almost unimaginable—is on par with each of the world’s 7 billion people being able to carry out 3 million calculations per second.
Titan, which is supported by the Department of Energy, will provide unprecedented computing power for research in energy, climate change, efficient engines, materials and other disciplines and pave the way for a wide range of achievements in science and technology.
The Cray XK7 system contains 18,688 nodes, with each holding a 16-core AMD Opteron 6274 processor and an NVIDIA Tesla K20 graphics processing unit (GPU) accelerator. Titan also has more than 700 terabytes of memory. The combination of central processing units, the traditional foundation of high-performance computers, and more recent GPUs will allow Titan to occupy the same space as its Jaguar predecessor while using only marginally more electricity.
"One challenge in supercomputers today is power consumption," said Jeff Nichols, associate laboratory director for computing and computational sciences. "Combining GPUs and CPUs in a single system requires less power than CPUs alone and is a responsible move toward lowering our carbon footprint. Titan will provide unprecedented computing power for research in energy, climate change, materials and other disciplines to enable scientific leadership."
Because they handle hundreds of calculations simultaneously, GPUs can go through many more than CPUs in a given time. By relying on its 299,008 CPU cores to guide simulations and allowing its new NVIDIA GPUs to do the heavy lifting, Titan will enable researchers to run scientific calculations with greater speed and accuracy.
"Titan will allow scientists to simulate physical systems more realistically and in far greater detail," said James Hack, director of ORNL's National Center for Computational Sciences. "The improvements in simulation fidelity will accelerate progress in a wide range of research areas such as alternative energy and energy efficiency, the identification and development of novel and useful materials and the opportunity for more advanced climate projections."
Titan will be open to select projects while ORNL and Cray work through the process for final system acceptance. The lion's share of access to Titan in the coming year will come from the Department of Energy's Innovative and Novel Computational Impact on Theory and Experiment program, better known as INCITE.
Researchers have been preparing for Titan and its hybrid architecture for the past two years, with many ready to make the most of the system on day one. Among the flagship scientific applications on Titan:
Materials Science The magnetic properties of materials hold the key to major advances in technology. The application WL-LSMS provides a nanoscale analysis of important materials such as steels, iron-nickel alloys and advanced permanent magnets that will help drive future electric motors and generators. Titan will allow researchers to improve the calculations of a material's magnetic states as they vary by temperature.
"The order-of-magnitude increase in computational power available with Titan will allow us to investigate even more realistic models with better accuracy," noted ORNL researcher and WL-LSMS developer Markus Eisenbach.
Combustion The S3D application models the underlying turbulent combustion of fuels in an internal combustion engine. This line of research is critical to the American energy economy, given that three-quarters of the fossil fuel used in the United States goes to powering cars and trucks, which produce one-quarter of the country's greenhouse gases.
Titan will allow researchers to model large-molecule hydrocarbon fuels such as the gasoline surrogate isooctane; commercially important oxygenated alcohols such as ethanol and butanol; and biofuel surrogates that blend methyl butanoate, methyl decanoate and n-heptane.
"In particular, these simulations will enable us to understand the complexities associated with strong coupling between fuel chemistry and turbulence at low preignition temperatures," noted team member Jacqueline Chen of Sandia National Laboratories. "These complexities pose challenges, but also opportunities, as the strong sensitivities to both the fuel chemistry and to the fluid flows provide multiple control options which may lead to the design of a high-efficiency, low-emission, optimally combined engine-fuel system."
Nuclear Energy Nuclear researchers use the Denovo application to, among other things, model the behavior of neutrons in a nuclear power reactor. America's aging nuclear power plants provide about a fifth of the country's electricity, and Denovo will help them extend their operating lives while ensuring safety. Titan will allow Denovo to simulate a fuel rod through one round of use in a reactor core in 13 hours; this job took 60 hours on the Jaguar system.
Climate Change The Community Atmosphere Model-Spectral Element simulates long-term global climate. Improved atmospheric modeling under Titan will help researchers better understand future air quality as well as the effect of particles suspended in the air.
Using a grid of 14-kilometer cells, the new system will be able to simulate from one to five years per day of computing time, up from the three months or so that Jaguar was able to churn through in a day.
"As scientists are asked to answer not only whether the climate is changing but where and how, the workload for global climate models must grow dramatically," noted CAM-SE team member Kate Evans of ORNL. "Titan will help us address the complexity that will be required in such models."
ORNL is managed by UT-Battelle for the Department of Energy. The Department of Energy is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.