Difference between revisions of "Goodness of Fit"

From SCECpedia
Jump to navigationJump to search
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
SCEC computational scientists often need to compare simulation parameters against observed geophysical and seismic observations. We often need to define what parameters we compare and how to evaluate groups of comparisons.
 
SCEC computational scientists often need to compare simulation parameters against observed geophysical and seismic observations. We often need to define what parameters we compare and how to evaluate groups of comparisons.
 
Examples of Goodness of Fit in SCEC research includes Broadband Platform development, in the SCEC CVM development, in CISN EEW evaluation, and CSEP short term earthquake forecast testing.
 
Examples of Goodness of Fit in SCEC research includes Broadband Platform development, in the SCEC CVM development, in CISN EEW evaluation, and CSEP short term earthquake forecast testing.
−
 
  
 
== Mayhew and Olsen ==
 
== Mayhew and Olsen ==
Line 9: Line 8:
 
*Olsen, K.B., and J.E. Mayhew (2010). Goodness-of-fit Criteria for Broadband Synthetic Seismograms, With Application to the 2008 Mw5.4 Chino Hills, CA, Earthquake,  Seism. Res. Lett., revised.
 
*Olsen, K.B., and J.E. Mayhew (2010). Goodness-of-fit Criteria for Broadband Synthetic Seismograms, With Application to the 2008 Mw5.4 Chino Hills, CA, Earthquake,  Seism. Res. Lett., revised.
 
* [http://www-rohan.sdsu.edu/~kbolsen/PUBL_dir/JOUR.html Kim Olsen's Site for Paper Download]
 
* [http://www-rohan.sdsu.edu/~kbolsen/PUBL_dir/JOUR.html Kim Olsen's Site for Paper Download]
 +
*[http://hypocenter.usc.edu/research/CME_Meetings/Olsen_ssa11_gof.ppt Goodness of Fit SSA 2011] (65Mb) - Kim Olsen
 +
 +
The algorithm computes:
 +
 +
#PGA, PGV, and PGD
 +
#The response spectral acceleration averaged for periods between 0.1 and 10 s (RS)
 +
#Fourier spectrum (smoothed to reduce variance; FS, Equation S6)
 +
#Spectral acceleration at 16 individual periods used by recent NGA relations (SA16)
 +
#Energy duration (DUR)
 +
#Cumulative energy (ENER)
 +
#Cross-correlation (Xcor)
 +
#IE ratio is the ratio between the maximum inelastic and elastic displacements plotted against the strength reduction factor R
 +
 +
== Broadband Platform ==
 +
[[File:Gof-seis-match.png|256px|thumb|right|Matched Seismogram Comparison]]
 +
[[File:Nr_PGA_map.png|256px|thumb|right|Northridge MO-GoF PGA Map Plot]]
 +
[[File:Match_Acc_overlay.png|256px|thumb|right|Seismogram Overlay with PGA]]
 +
===URS GoF===
 +
[[Broadband Platform]] v11.2 has integrated a bias calculation comparing forecast peak ground motion amplitudes against observed peak amplitudes. These bias plots are calculated for validation events for different code bases.
 +
 +
===SDSU GoF===
 +
The SDSU - Mayhew & Olsen goodness of fit (MO-GoF) is being integrated into the next release of BBP. This GoF will be available as a selectable option in BBP workflow and the user can choose to run URS-GoF and/or MO-GoF.
 +
 +
====Matching Seismograms====
 +
MO-GoF module requires the observed and synthetic seismograms to have the same sampling rate and record length before GoF metrics are calculated. Some metrics like cross correlation, energy duration and cumulative energy, require the observed and synthetic seismograms to be trace aligned manually to produce meaningful results. These three metrics will be disabled in the initial release of the module.
 +
 +
The BBP MO-GoF module includes code to match the observed and synthetic seismograms for each station in the station list before GoF is calculated. The trace matching code compares the observed and synthetic seismogram to determine if their sample rate or record length differ.
 +
 +
* Sample Rate: If the sample rate of one of the seismograms is lower than the other, the seismogram with the lower sample rate is up-sampled to match the sample rate of the other seismogram.
 +
* Record Length: After the sample rate of a station's observed and synthetic seismogram is matched, the seimograms are checked for matching record lengths. If one of the seismograms has a shorter record length, it is padded with the last data value found in the file to match the length of the other seismogram.
 +
 +
====GoF Metric Calculation====
 +
 +
The MO-GoF algorithm can calculate up to 12 GoF metrics. The following table lists the MO-GoF metrics that will be available in the next release of BBP through MO-GoF module.
 +
{| class="wikitable" border="1"
 +
|-
 +
! GoF Metric
 +
! Status
 +
|-
 +
| PGA
 +
| Available
 +
|-
 +
| PGV
 +
| Available
 +
|-
 +
| PGD
 +
| Available
 +
|-
 +
| PSA
 +
| Available
 +
|-
 +
| Spectral Fit
 +
| Available
 +
|-
 +
| Spectral Acceleration
 +
| Available
 +
|-
 +
| In-Elastic - Elastic Fit
 +
| Available
 +
|-
 +
| Fourier Spectrum
 +
| Available
 +
|-
 +
| Cumulative Energy
 +
| Pending
 +
|-
 +
| Energy Duration
 +
| Pending
 +
|-
 +
| Cross Correlation
 +
| Pending
 +
|-
 +
| Overall Site Fit
 +
| Available
 +
|}
  
 +
====GoF Metric Map Plots====
 +
BBP platform will generate map plots for each of the GoF metrics that are selected. The sparse GoF metric data for the stations in simulation station list is smoothed out by interpolating over the region defined by the smallest-perimeter polygon enclosing all the stations in the simulation station list.
  
−
== CVM-T ==
+
== SCEC CVM Developments ==
  
 
SCEC CVM development involves improving the underlying earth structure model, adding new material databases like geotechnical layer information, and codes for meshing and populating large scale meshes.
 
SCEC CVM development involves improving the underlying earth structure model, adding new material databases like geotechnical layer information, and codes for meshing and populating large scale meshes.
  
−
*[[CVM Toolkit]]
+
*[[UCVM]]
 +
*[[CVM-H]]
 +
*[[CVM Evaluation]]
 +
 
 +
== Other Goodness of Fit Measurements ==
 +
 
 +
*[http://www.iitk.ac.in/nicee/wcee/article/13_243.pdf John Anderson 2004 Paper on Synthetics to Observations]
 +
*[http://www.rsmas.miami.edu/personal/glin/Research.html Guoqing Lin's Research Page]

Latest revision as of 23:47, 7 October 2011

SCEC computational scientists often need to compare simulation parameters against observed geophysical and seismic observations. We often need to define what parameters we compare and how to evaluate groups of comparisons. Examples of Goodness of Fit in SCEC research includes Broadband Platform development, in the SCEC CVM development, in CISN EEW evaluation, and CSEP short term earthquake forecast testing.

Mayhew and Olsen

Mayhew and Olsen have developed and published the following paper on goodness of fit for ground motion modeling groups.

The algorithm computes:

  1. PGA, PGV, and PGD
  2. The response spectral acceleration averaged for periods between 0.1 and 10 s (RS)
  3. Fourier spectrum (smoothed to reduce variance; FS, Equation S6)
  4. Spectral acceleration at 16 individual periods used by recent NGA relations (SA16)
  5. Energy duration (DUR)
  6. Cumulative energy (ENER)
  7. Cross-correlation (Xcor)
  8. IE ratio is the ratio between the maximum inelastic and elastic displacements plotted against the strength reduction factor R

Broadband Platform

Matched Seismogram Comparison
Northridge MO-GoF PGA Map Plot
Seismogram Overlay with PGA

URS GoF

Broadband Platform v11.2 has integrated a bias calculation comparing forecast peak ground motion amplitudes against observed peak amplitudes. These bias plots are calculated for validation events for different code bases.

SDSU GoF

The SDSU - Mayhew & Olsen goodness of fit (MO-GoF) is being integrated into the next release of BBP. This GoF will be available as a selectable option in BBP workflow and the user can choose to run URS-GoF and/or MO-GoF.

Matching Seismograms

MO-GoF module requires the observed and synthetic seismograms to have the same sampling rate and record length before GoF metrics are calculated. Some metrics like cross correlation, energy duration and cumulative energy, require the observed and synthetic seismograms to be trace aligned manually to produce meaningful results. These three metrics will be disabled in the initial release of the module.

The BBP MO-GoF module includes code to match the observed and synthetic seismograms for each station in the station list before GoF is calculated. The trace matching code compares the observed and synthetic seismogram to determine if their sample rate or record length differ.

  • Sample Rate: If the sample rate of one of the seismograms is lower than the other, the seismogram with the lower sample rate is up-sampled to match the sample rate of the other seismogram.
  • Record Length: After the sample rate of a station's observed and synthetic seismogram is matched, the seimograms are checked for matching record lengths. If one of the seismograms has a shorter record length, it is padded with the last data value found in the file to match the length of the other seismogram.

GoF Metric Calculation

The MO-GoF algorithm can calculate up to 12 GoF metrics. The following table lists the MO-GoF metrics that will be available in the next release of BBP through MO-GoF module.

GoF Metric Status
PGA Available
PGV Available
PGD Available
PSA Available
Spectral Fit Available
Spectral Acceleration Available
In-Elastic - Elastic Fit Available
Fourier Spectrum Available
Cumulative Energy Pending
Energy Duration Pending
Cross Correlation Pending
Overall Site Fit Available

GoF Metric Map Plots

BBP platform will generate map plots for each of the GoF metrics that are selected. The sparse GoF metric data for the stations in simulation station list is smoothed out by interpolating over the region defined by the smallest-perimeter polygon enclosing all the stations in the simulation station list.

SCEC CVM Developments

SCEC CVM development involves improving the underlying earth structure model, adding new material databases like geotechnical layer information, and codes for meshing and populating large scale meshes.

Other Goodness of Fit Measurements

Retrieved from "https://strike.scec.org/scecwiki/index.php?title=Goodness_of_Fit&oldid=3733"