CASE STUDY
Using Cray System “Blue Waters,” Researchers Break the HIV Capsid Code
and Open the Door to New Virus-Busting Therapies
Situation
Organization
National Center for Supercomputing Applications
Urbana-Champaign, IL
www.ncsa.illinois.edu
About Blue Waters
Housed at the National Center for
Supercomputing Applications (NCSA) at the
University of Illinois at Urbana-Champaign
and supported by the National Science
Foundation, the Blue Waters supercomputer is
one of the most powerful supercomputers in
the world and the fastest system anywhere on
a university campus. Scientists and engineers
use the computing and data power of Blue
Waters to tackle a wide range of challenging
problems, from predicting the behavior of
complex biological systems to simulating the
evolution of the cosmos. The system is based
on Cray® XE6™ and XK7™ technology and
uses hundreds of thousands of computational
cores to achieve over 13 petaflops of peak
performance.
System Overview
•System: Cray XE/XK hybrid supercomputer
•Cabinets: 288
•Peak Performance: 13 PF
•System Memory: 1.5 PB
•XE Compute Nodes: 22,640 AMD 6276
“Interlagos” processors
•XK Compute Nodes: 4,224 NVIDIA® GK110
“Kepler” GPU accelerators
•Interconnect: Gemini
Cray Inc.
901 Fifth Avenue, Suite 1000
Seattle, WA 98164
Tel: 206.701.2000
Fax: 206.701.2500
www.cray.com
The HIV virus has proven so challenging to treat because it has a high mutation rate — a
characteristic that makes it quickly resistant to drug therapies. As a result, researchers
have long sought to understand how the HIV capsid — a protein shell that protects
the virus’s genetic material — is constructed. Knowing its structure in detail will enable
scientists to develop more effective drugs against it.
Many studies have chipped away at the mystery of the HIV capsid. Researchers have used
a variety of laboratory techniques — cryo-electron microscopy, cryo-EM tomogrpahy,
nuclear magnetic resonance spetroscopy and X-ray crystallography — to look at individual
parts of the capsid or to get a sense of the whole. But no one had been able to piece
together the entire HIV capsid.
Challenge
Researchers had amassed an enormous amount of information on HIV, in particular the
architecture of the virus and the various steps it undergoes in living cells. They knew it
was an assemblage of more than 1,300 identical proteins.
They also knew from earlier work on the fullerene cone analogy that the proteins were
arranged into pentagons and hexagons, and they hypothesized that the pentagons
formed the most tightly rounded corners of the capsid shape seen under an electron
microscope.
But they did not know how many of these protein buildling blocks were needed, or how
the pentagons and hexagons fit together to form the capsid. And no one could piece
together the entire HIV capsid in atomic-level detail.
“In experiments we [could] look at the same thing at different size scales and detail, but
you can only look at small pieces. You cannot look at the whole assembly — or [you can
look at] the whole assembly at low resolution,” says University of Illinois postdoctoral
researcher Juan Perilla who, with physics professor Klaus Schulten, conducted molecular
simulations on Blue Waters that integrated data from laboratory experiments performed
by colleagues at the University of Pittsburgh and Vanderbilt University. “We had to find a
way to put all of this [data] together to build a capsid.”
“It was very clear that it would require a huge amount of simulation — the largest
simulation ever published — involving 64 million atoms,” says Schulten.
“Blue Waters was essential to the success of the project. It’s the only machine where we
could have run a simulation of this size.”
—Juan R. Perilla
Postdoctoral Researcher, University of Illinois
©2014 Cray Inc. All rights reserved. Cray is a registered trademark of Cray Inc.
All other trademarks mentioned herein are the properties of their respective owners. 20140529_V1KJL
CASE STUDY
Page #2
Solution
Right as the HIV capsid project was developing, the petascale Blue Waters supercomputer was coming online at NCSA. The Blue Waters
system is a Cray® XE6™/XK7™ hybrid machine composed of AMD 6276 “Interlagos” processors and NVIDIA® GK110 “Kepler” GPU accelerators
all connected by the Cray Gemini torus interconnect. It also has a dedicated storage system that includes 26 petabytes of usable online
storage for quick access while jobs are running based on the Cray® Sonexion® storage system for Lustre. The entire system is purpose built
to provide the computational capability to make it possible for investigators to tackle larger and more complex research challenges across a
wide spectrum of domains.
We were very, very fortunate,” says Schulten. “We knew on this computer we could carry out the necessary simulation. We could simulate
up to a 100-million atom system.”
Perilla and Schulten used the data from experiments carried out by their colleagues and from their own simulations of the interactions
between the hexamers and pentamers to conduct a series of large-scale computer simulations that accounted for the structural properties
of the capsid’s building blocks.
“The work of matching the overall capsid, made of 64 million atoms, to the diverse experimental data can only be done through computer
simulation using a methodology we have developed called molecular dynamic flexible fitting,” Schulten says. “You basically simulate the
physical characteristics and behavior of large biological molecules but you also incorporate the data into the simulation so that the model
actually drives itself toward agreement with the data.”
The simulations revealed that the HIV capsid contained 216 protein hexagons and 12 protein pentagons arranged just as the experimental
data had indicated. The proteins that composed these pentagons and hexagons were all identical, and yet the angles of attachment between
them varied from one region of the capsid to another.
The pentagons “induced acute surface curvature,” the researchers reported, allowing the capsid to be a closed structure that would not have
been possible if the capsid were composed only of hexagons.
Having a chemically detailed structure of the HIV capsid will allow researchers to further investigate how it functions, with implications for
pharmacological interventions to disrupt that function. “[Blue Waters] was essential to the success of the project because it’s only machine
where we could have run a simulation of this size,” Perilla says.
For more information on NCSA’s HIV capsid work, visit http://news.illinois.edu/news/13/0529HIVcapsid_KlausSchulten.html.
Content courtesy of National Center for Supercomputing Applications at the University of Illinois Urbana-Champaign