The Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility, has selected six projects for its Theta Early Science Program (ESP), a collaborative effort designed to help prepare scientific applications for the architecture and scale of the new supercomputer. 

Theta, an early production system based on Intel’s second-generation Xeon Phi processor, will arrive in 2016 and serve as a bridge between the ALCF’s current supercomputer, Mira, and its next leadership-class supercomputer, Aurora, which is scheduled for delivery in 2018.

The Theta ESP brings together computational scientists, code developers, and computing hardware experts to optimize key applications for Theta, and to solidify libraries and infrastructure to pave the way for other applications to run on the system.

Modeled after the ALCF’s highly successful ESP for Mira, the program also gives researchers substantial allocations of pre-production compute time on Theta to pursue innovative computational science calculations that push the boundaries of what’s possible with leadership-class supercomputers.

“Not only did our ESP for Mira help the system to hit the ground running on day one, it also produced invaluable lessons learned and some very interesting new science, including the first accurately computed values for the bulk properties of solid argon, a noble gas element,” said Tim Williams, an ALCF computational scientist who manages the ESP. “We’re looking forward to seeing what the new ESP projects can do with Theta’s leading-edge architecture.”

Like the typical ALCF workload, the six selected ESP projects, known as Tier 1 projects, represent a wide spectrum of scientific areas and numerical methods (see project descriptions below).

To help develop and optimize their software for Theta, project teams will collaborate with ALCF staff, as well as vendor staff through the ALCF Intel-Cray Center for Excellence (AICCE). Four of the six projects will also be assigned a dedicated postdoctoral researcher.

In addition, the ALCF will host ESP training sessions, including a virtual kick-off workshop on system hardware and programming, and a hands-on workshop for testing and debugging of project applications.

Prior to Theta’s availability, the ALCF will offer access to Theta simulator software, and provide allocations on Mira for development work that does not depend on having the new hardware (e.g., new algorithms, new physics modules, basic introduction of threads). ESP project teams will also have access to training and hardware at the Oak Ridge Leadership Computing Facility (OLCF) and the National Energy Research Supercomputing Center (NERSC) as alternative development platforms to encourage application code portability among heterogeneous architectures.

Because of the strong response to the call for proposals, the ALCF is expanding the Theta ESP to include six additional Tier 2 projects to help prepare other applications for Theta. These projects will not receive allocations for science runs, but they will have access to ESP training, an ESP discussion forum, Theta simulator software, early hardware, and to Theta itself for porting, tuning, and debugging.

Tier 1 Projects

Scale-Resolving Simulations of Wind Turbines with SU2
PI: Juan J. Alonso, Stanford University
Code: SU2

Alonso will use Theta to develop a simulation capability to design better wind turbines and to lay out large wind farms for maximum energy extraction and improved turbine fatigue life. To do so, his research team will generate a database of large eddy simulations of various single and multiple wind turbine settings.

Large-Scale Simulation of Brain Tissue: Blue Brain Project, EPFL
PI: Fabien Delalondre, Ecole Federale Polytechnique de Lausanne
Code: CoreNeuron

Delalondre will use Theta to improve our understanding of the brain using simulations of brain plasticity—experience-dependent changes in synaptic connectivity. Other work will include rodent somatosensory cortex simulation, and simulating the electrical activity of the largest possible brain model for several seconds of biological time.

First-Principles Simulations of Functional Materials for Energy Conversion
PI: Giulia Galli, Argonne National Laboratory/University of Chicago
Codes: Qbox, WEST

Galli will combine ab initio molecular dynamics and post-density functional theory methods to optimize properties of nanostructured materials for use in solar and thermal energy conversion devices at an unprecedented level of accuracy. The ultimate goal is to provide a truly predictive tool for device performance within a Materials Genome Initiative design framework.

Next-Generation Cosmology Simulations with HACC: Challenges from Baryons
PI: Katrin Heitmann, Argonne National Laboratory
Code: HACC

Heitmann’s project aims to further our understanding of astrophysical processes by performing detailed simulations of the universe for comparison with the latest observational data. The simulations will disentangle astrophysical processes (e.g., galaxy evolution) from fundamental physics (e.g., dark energy), helping mitigate one of the major sources of systematic uncertainties for upcoming cosmological surveys.

Direct Numerical Simulations of Flame Propagation in Hydrogen-Oxygen Mixtures in Closed Vessels
PI: Alexei Khokhlov, University of Chicago
Code: HSCD

Khokhlov will perform direct numerical simulations of the flame acceleration and the deflagration-to-detonation transition process in hydrogen-oxygen mixtures in closed spherical vessels—exactly matching experimental apparatus. This research is aimed at improving the industrial and public safety of hydrogen fuels and certain types of water-cooled nuclear reactors in which hydrogen can accumulate.

Free Energy Landscapes of Membrane Transport Proteins
PI: Benoit Roux, Argonne National Laboratory/University of Chicago
Code: NAMD

Roux will carry out molecular dynamics simulations to provide detailed visualizations of the large conformational changes of membrane transport proteins and quantitative predictions of the energetics of these processes. This atomistic picture of membrane transport proteins stands to improve our understanding of a broad range of biological functions.

Tier 2 Projects

Electronic Structure Based Discovery of Hybrid Photovoltaic Materials on Next-Generation HPC Platforms
PI: Volker Blum, Duke University
Code: FHI-aims, GAtor

Flow, Mixing and Combustion of Transient Turbulent Gaseous Jets in Confined Cylindrical Geometries
PI: Christos Frouzakis, Swiss Federal Institute of Technology Zurich (ETHZ)
Code: Nek5000

Advanced Electronic Structure Methods for Heterogeneous Catalysis and Separation of Heavy Metals
PI: Mark Gordon, Iowa State University
Code: GAMESS

Extreme Scale Unstructured Adaptive CFD: From Multiphase Flow to Aerodynamic Flow Control
PI: Kenneth Jansen, University of Colorado Boulder
Code: PHASTA

The Hadronic Contribution to the Anomalous Magnetic Moment of the Muon
PI: Paul Mackenzie, Fermilab
Codes: MILC, CPS

Quantum Monte Carlo Calculations in Nuclear Theory
PI: Steven Pieper, Argonne National Laboratory
Code: GFMC

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.  For more information, please visit science.energy.gov.