CyberShake Study 15.4

CyberShake Study 15.3 is a computational study to calculate one physics-based probabilistic seismic hazard model for Southern California at 1 Hz, using CVM-S4.26, the GPU implementation of AWP-ODC-SGT, the Graves and Pitarka (2015) rupture variations with uniform hypocenters, and the UCERF2 ERF. The SGT calculations will be split between NCSA Blue Waters and OLCF Titan, and the post-processing will be done entirely on Blue Waters. The goal is to calculate the standard Southern California site list (286 sites) used in previous CyberShake studies so we can produce comparison curves and maps, and data products for the UGMS Committee.

Preparation for production runs

1. Check list of Mayssa's concerns
2. Update DAX to support separate MD5 sums
3. Add MD5 sum job to TC
4. Evaluate topology-aware scheduling
5. Get DirectSynth working at full run scale, verify results
6. Modify workflow to have md5sums be in parallel
7. Test of 1 Hz simulation with 2 Hz source - 2/27
8. Add a third pilot job type to Titan pilots - 2/27
9. Run test of full 1 Hz SGT workflow on Blue Waters - 3/4
10. Add cleanup to workflow and test - 3/4
11. Test interface between Titan workflows and Blue Waters workflows - 3/4
12. Add capability to have files on Blue Waters correctly striped - 3/6
13. Add restart capability to DirectSynth - 3/6
14. Add DirectSynth to workflow tools - 3/6
15. Implement and test parallel version of reformat_awp - 3/6
16. Simulate curves for 3 sites with final configuration; compare curves and seismograms - 3/11
17. File ticket for 90-day purged space at Blue Waters - 3/13
18. File ticket for reservation at Blue Waters, along with justification - 3/13
19. Follow up on high priority jobs at Titan - 3/13
20. Create study description file for Run Manager - 3/13
21. Science readiness review - 3/18
22. Technical readiness review - 3/18

Computational Status

We are hoping to begin this study in early March.

Goals

Science Goals

1. Calculate a 1 Hz hazard map of Southern California.
2. Produce a contour map at 1 Hz for the UGMS committee.
3. Compare the hazard maps at 0.5 Hz and 1 Hz.

Technical Goals

1. Show that Titan can be integrated into our CyberShake workflows.
2. Demonstrate scalability for 1 Hz calculations.
3. Show that we can split the SGT calculations across sites.

Verification

DirectSynth

A comparison of 1 Hz results with SeisPSA to results with DirectSynth for WNGC. SeisPSA results are in magenta, DirectSynth results are in black. They're so close it's difficult to make out the magenta.

2s	3s	5s	10s

Sites

We are proposing to run 286 sites around Southern California. Those sites include 46 points of interest, 27 precarious rock sites, 23 broadband station locations, 43 20 km gridded sites, and 147 10 km gridded sites. All of them fall within the Southern California box except for Diablo Canyon and Pioneer Town. You can get a CSV file listing the sites here. A KML file listing the sites is available here.

Fig 1: Sites selected for Study 2.3 Purple are gridded sites, red are precarious rocks, orange are SCSN stations, and yellow are sites of interest.