A team led by researchers at the Argonne Leadership Computing Facility (ALCF) was recognized with the Best Paper Award for an innovative scientific visualization study presented at the 2023 In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization (ISAV) Workshop on November 13 at the SC23 conference in Denver, Colorado. The ALCF is a U.S. Department of Energy (DOE) Office of Science user facility at DOE’s Argonne National Laboratory.

The team’s paper, “Scaling Computational Fluid Dynamics: In Situ Visualization of NekRS using SENSEI,” details their efforts to equip the NekRS computational fluid dynamics (CFD) code with in situ analysis and visualization capabilities. Led by ALCF’s Victor Mateevitsi, the team includes Argonne colleagues Nicola Ferrier, Paul Fischer, Joseph Insley, Yu-Hsiang Lan, Misun Min, Michael Papka, Saumil Patel, and Silvio Rizzi; and Mathis Bode, Jens Henrik Göbbert, and Jonathan Windgassen from the Jülich Supercomputing Centre (JSC) in Germany. The collaboration operates under the Joint Laboratory for Extreme Scale Computing (JLESC), which is focused on addressing software challenges in high performance computing (HPC). 

In situ processing allows researchers to analyze and visualize simulation data on the fly while computations are being carried out. This has emerged as an important capability for large-scale computational workloads, such as CFD simulations, as the processing power of new HPC systems has significantly outpaced that of I/O speeds. Traditional methods of saving simulation data to disk, known as checkpointing, can cause significant slowdowns, prompting a choice between reducing checkpoint frequency or lowering resolution. The in situ approach allows more frequent data “snapshots” to be taken directly from memory, eliminating the need for disruptive checkpointing. 

In their ISAV paper, the team explored the benefits of instrumenting the NekRS code with SENSEI, an open-source in situ interface developed with support from DOE’s Exascale Computing Project. The researchers demonstrated the capabilities and scalability of their approach by running two real scientific use cases on the ALCF’s Polaris supercomputer and JSC’s JUWELS Booster supercomputer. They found that the ability to “see” the data in real time significantly augmented their analytical capacities.

“Our findings demonstrate that with careful configuration, integration of advanced tools, and an understanding of the underlying phenomena, we can achieve computational efficiency without compromising on visualization efficacy. As computational simulations become even more intricate and demand increased resources, such approaches will be pivotal in advancing scientific understanding,” the paper concludes.