Blue Waters Project

Project Description

Extending the Spatiotemporal Scales of Physics-based Seismic Hazard Analysis

PI: T. H. Jordan (USC); Co-PIs: J. Bielak (CMU), K. Olsen (SDSU), Y. Cui (SDSC)

Earthquake simulations at the spatiotemporal scales required for probabilistic seismic hazard analysis (PSHA) present some of the toughest computational challenges in geoscience. PSHA is the scientific basis for many engineering and social applications: performance-based design, seismic retrofitting, resilience engineering, insurance-rate setting, disaster preparation and warning, emergency response, and public education. This project will extend deterministic earthquake simulations to seismic frequencies of 2 Hz and greater with the goal of reducing the epistemic uncertainties in physics-based PSHA. The research will address fundamental scientific problems that limit the scale range in current representations of source physics, anelasticity, and geologic heterogeneity. The research will improve the physical representations of earthquake processes and the deterministic codes for simulating earthquakes, which will benefit earthquake system science worldwide. The consequent decrease in mean exceedance probabilities, which could be up to an order of magnitude at high hazard levels, would have a broad impact on the prioritization and economic costs of risk-reduction strategies.

Previous research on Blue Waters has verified the scalability and computational readiness of the simulation codes. These codes will be used to advance physics-based PSHA through a coordinated program of numerical experimentation and large-scale simulation targeted at three primary objectives: (1) validation of high-frequency simulations against seismic recordings of historical earthquakes; (2) computation of high-frequency CyberShake hazard models for the Los Angeles region to support the development of high-resolution urban seismic hazard maps by the U. S. Geological Survey and the Southern California Earthquake Center (SCEC), and (3) high-frequency simulation of a M7.8 earthquake on the San Andreas fault to revise the 2008 Great California ShakeOut scenario and improve the risk analysis developed in detail for that scenario. The plan to accomplish this research has five computational milestones: (a) dynamic rupture simulations up to 8 Hz that include fault roughness and near-fault plasticity; (b) simulations of historic earthquakes up to 4 Hz for model validation; (c) simulations of the 1994 Northridge Earthquake up to 8 Hz for verification and validation; (d) extension of the 2008 ShakeOut scenario to 4 Hz; and (e) calculation of a complete CyberShake hazard model for the Los Angeles region up to 2 Hz. Blue Waters is a Cray XE6/XK7 system consisting of more than 22,500 XE6 compute nodes (each containing two AMD Interlagos processors) augmented by more than 4200 XK7 compute nodes (each containing one AMD Interlagos processor and one NVIDIA GK110 "Kepler" accelerator) in a single Gemini interconnection fabric. This configuration enables extremely large simulations on hundreds of thousands of traditional CPUs for science and engineering discovery, while also supporting development and optimization of cutting-edge applications capable of leveraging the compute power of thousands of GPUs.

Acknowledgement

This research is part of the Blue Waters sustained-petascale computing project, which is supported by the National Science Foundation (awards OCI-0725070 and ACI-1238993) and the state of Illinois. Blue Waters is a joint effort of the University of Illinois at Urbana-Champaign and its National Center for Supercomputing Applications. This work is also part of the "Extending the Spatiotemporal Scales of Physics-based Seismic Hazard Analysis" PRAC allocation support by the National Science Foundation (award number OCI TBD).

PRAC 2014 Award

Extending the Spatiotemporal Scales of Physics-based Seismic Hazard Analysis (OCI-1440085)

• NSF PRAC Award 2014

PRAC 2009 Award

Petascale Research in Earthquake System Science on Blue Waters (OCI-0832698)

• NSF PRAC Award 2009

SCEC at 2015 Blue Waters Symposium

Philip Maechling attended the 2015 meeting for SCEC and presented a summary of work this year.

• Maechling SCEC Presentation 2015 (30 MB pptx)
• Blue Waters Symposium Web Page

SCEC At Blue Waters Symposium 2014

Thomas H. Jordan attended the 2014 Blue Waters Symposium and presented SCEC research summary for 2014.

• Jordan SCEC Presentation 2014 (pdf)

SCEC At Blue Waters Symposium 2013

E. Poyraz attended the 2013 NEIS-P2 meeting and presented progress developing GPU codes for Blue Waters

• SCEC NEIS-P2 Progress Summary 2013 (E. Poyraz) (43MB pptx)

Blue Waters User Portal

• Blue Waters User Portal

Blue Waters Home Page

• Blue Waters Home Page

Blue Waters User Wiki Site

• User Acct Logins

File System Organization

The three file systems are provided to the users as follows:

• /u – storage for home directories
• /projects – storage for project home directories
• /scratch – high performance, high capacity transient storage for applications

Related Material

• Jordan High Frequency Simulation Presentation (pdf)
• Jordan NSF HPC Talk 2014 (pdf)

Related Links

• Geoinformatics Project
• SEISM Project
• CME Project
• Blue Waters Project