Broadband CyberShake Validation

This page follows on CyberShake BBP Verification, moving from 1D comparisons to 3D CyberShake comparisons with both the BBP and observations.

Process

These are the steps involved in comparing CyberShake and BBP results to observations for a verification event.

BBP

1. Create a working directory.
2. Create an src_files directory inside it.
3. Copy the src file from the BBP validation directory into the src_files directory.
4. Copy the station list into the working directory.
5. Run run_bbp.py with the '--expert -g' arguments to create an XML file description of the run. Instead of using the defaults for the validation event, point to the station list file in the working directory and the src file in the src_files directory. Include the GP GOF, but not FAS.
6. Create 64 realizations by copying over the initial src file and changing the random seed.
7. Create 64 directories, one for each source.
8. Copy over the setup_lsf_files.sh script. Edit the xml_file_timestamp value to the XML file generated with the run_bbp.py script. You may also need to edit the find/replace lines to reflect the correct path to the .src and .srf files.
9. Edit the run_bbp_src.lsf script. Change the xml_file_timestamp, the sim_id, and the name of the event.
10. Run setup_lsf_files.sh, which will copy over the run_bbp_src.lsf script into every realization directory and make the needed changes for that realization index.
11. Submit all 64 src_*/run_bbp_src*.lsf files.
12. Once all 64 run successfully, make a purple_plot/Sims directory, and logs, indata, tmpdata, and outdata directories inside it.
13. Edit stage_purple_plot.sh to have the right sim_id.
14. Run stage_purple_plot.sh .
15. Run utils/batch/combine_gof_gen.py -d purple_plot -o . -c gp
16. Examine the .png file produced to make sure it looks OK.

CyberShake

1. Create a working directory.
2. Copy the src_files directory from the Broadband validation run directory
3. Create the src_* directories, one for each of the 64 realizations.
4. Copy in the *.stl file from the BBP for this validation event.
5. Create site_list.txt, with one CyberShake site per line.
6. From another CyberShake validation directory, copy in the following scripts:
   • import.lsf
   • prep.lsf
   • prep_stat.sh
   • rename.py
   • run_gof.sh
   • stage_purple_plot.sh

Northridge

1D BBP comparisons

We calculated 64 realizations for Northridge for these 38 stations.

Some of the differences between the 1D BBP and 3D CyberShake results can be attributed to differences in site response, which is calculated based on the reference velocity ('vref') and the vs30 of the site. A spreadsheet with vref and Vs30 for both the BBP and CyberShake is available File:BBP v CyberShake Northridge vs30.xls

V1 (2/14/22)

Initially, we used vref=500 for both the high-frequency and low-frequency site response. However, this is incorrect; vref for the low-frequency should vary depending on the properties of the 3D velocity mesh.

V2 (2/25/22)

We continued to use vref=500 for the high-frequency site response. For the low-frequency site response, we are now using the same vref we used in Study 15.12:

vref_LF_eff = Vs30 * [ VsD5H / Vs5H ]

Vs30 = 30 / Sum (1/(Vs sampled from [0.5,29.5] in 1 meter increments))

H = grid spacing

Vs5H = travel time averaged Vs, computed from the CVM in 1 meter increments down to a depth of 5*H

VsD5H = discrete travel time averaged Vs computed from 3D velocity mesh used in the SGT calculation over the upper 5 grid points

So, for H=100m Vs5H would be:

Vs500 = 500 / ( Sum ( 1 / Vs sampled from [0.5,499.5] in 1 meter increments ))

And then VsD5H is given by:

VsD500 = 5/{ 0.5/Vs(Z=0m) + 1/Vs(Z=100m) + 1/Vs(Z=200m) + 1/Vs(Z=300m) + 1/Vs(Z=400m) + 0.5/Vs(Z=500m) }

Below are the ts_process plots for a subset of 10 stations, comparing the 3D CyberShake with 1D BBP results.

Site	TS Process plot
SCE
SYL
LDM
PAC
PKC
SPV
WON
KAT
RO3
ANA

V3 (3/2/22)

Next, we recalculated the CyberShake results for 10 sites, using the BBP vs30 values for both the low-frequency and high-frequency elements. Note that the vref low-frequency value for CyberShake is still being derived from the velocity model.

Site	TS Process plot
SCE
SYL
LDM
PAC
PKC
SPV
WON
KAT
RO3
ANA

Observational Comparisons

We calculated goodness-of-fit results for both Broadband CyberShake and the BBP against observations for Northridge, using the 64 realizations and 38 stations.

TS process plots comparing the 3 results are available here: v1 ts process plots.

Broadband Platform

Overall GoF	GoF, best realization (#7)	GoF, worse realization (#15)

CyberShake v4_8

Overall GoF	GoF, best realization (#53)	GoF, worse realization (#54)

Broadband Platform, updated frequency bands

We realized that our GoF comparisons were run using hard-coded frequency bands of [0.05, 50] Hz. This was correct when doing CS-to-BBP comparisons, but not correct when using observations. We updated the frequency bands and reran the GoF.

Overall GoF	GoF, best BBP realization (#47)	GoF, worst BBP realization (#25)	GoF, best CS realization (#40)	GoF, worst CS realization (#37)

CyberShake v4_8, updated frequency bands

As above, reran the GoF.

Overall GoF	GoF, best CS realization (#40)	GoF, worst CS realization (#37)	GoF, best BBP realization (#47)	GoF, worst BBP realization (#25)

CyberShake v4_28, BBP Vs30

We recalculated the CyberShake GoF results, using the BBP Vs30 values.

Overall GoF	GoF, best CS realization (#40)	GoF, worst CS realization (#51)	GoF, best BBP realization (#47)	GoF, worst BBP realization (#25)

BBP, FAS

CyberShake v4_7, FAS

Observational Comparisons, BBP v22.4

We repeated the comparisons, using BBP v22.4 for both the 1D BBP and 3D CyberShake calculations.

BBP

Overall GoF	GoF, best BBP realization (#6)	GoF, worst BBP realization (#51)	GoF, best CS realization (#40)	GoF, worst CS realization (#51)

CyberShake

Overall GoF	GoF, best CS realization (#40)	GoF, worst CS realization (#51)	GoF, best BBP realization (#6)	GoF, worst BBP realization (#51)

Chino Hills

For our second event, we selected Chino Hills.

Sites

The BBP validation event for Chino Hills has 40 stations. A KML file with the stations is available here.

BBP

Overall GoF	GoF, best realization (#24)	GoF, worst realization (#35)	GoF, worst CS realization (#34)

CyberShake

Overall GoF	GoF, best realization (#24)	GoF, worst realization (#34)	GoF, worst BBP realization (#35)

CyberShake, BBP Vs30 values

Overall GoF	GoF, best realization (#24)	GoF, worst realization (#34)	GoF, worst BBP realization (#35)

CyberShake, BBP v22.4

Overall GoF	GoF, best realization (#24)	GoF, worst realization (#34)	GoF, best BBP realization (#24)	GoF, worst BBP realization (#35)

Whittier

Our third event is Whittier.

Sites

The BBP validation event for Whittier has 39 stations, one of which is a duplicate. A KML file with the 38 stations used in these tests is available here.

BBP

Overall GoF	GoF, best realization (#15)	GoF, worst realization (#7)	GoF, best CS realization (#62)	GoF, worst CS realization (#3)

CyberShake

These results are incomplete, without 2 of the stations (VER and PMN).

Overall GoF	GoF, best realization (#62)	GoF, worst realization (#3)	GoF, best BBP realization (#15)	GoF, worst BBP realization (#7)

File:CS-GoF-WH-FAS-combined.png 400px

CyberShake, BBP Vs30 values

Overall GoF	GoF, best realization	GoF, worst realization	GoF, best BBP realization	GoF, worst BBP realization
File:CS-BBPvs30-GoF-WH-combined.png 400px

File:CS-BBPvs30-GoF-WH-FAS-combined.png 400px

CyberShake, BBP v22.4

Overall GoF	GoF, best realization (#62)	GoF, worst realization (#39)	GoF, best BBP realization (#15)	GoF, worst BBP realization (#7)