Difference between revisions of "CTC Results"

From SCECpedia
Jump to navigationJump to search
 
(71 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Image:SCEC_NSF-USGS-words_logom.png|left|150px]][[Image:CISNlogos.png|right|250px]][[Image:usgs-logo-color.jpg|right|150px]]
+
[[Image:SCEC_NSF-USGS-words_logom.png|left|150px]]
 +
[[Image:CISNlogos.png|right|250px]]
 +
[[Image:Swiss_Seismological_Service-Logo.png‎|right|40px]]
  
[[CISN Testing Center]] evaluation results are presented as reference performance metrics. An important goal of a collaborative testing capabilities is the ability to compare equivalent results.  
+
The CISN Testing Center (CTC) consists of scientific analysis software, CISN EEW algorithm performance reports, and earthquake observational data.
  
== Testing Center Design Goals ==
+
== ShakeAlert Event Summaries ==
 +
Event-oriented ShakeAlert performance summaries are generated for each M3.5+ California earthquake found in the ANSS catalog. These ShakeAlert Event summaries show time series of EEW parameters logged by ShakeAlert during real-time operations.
  
CTC is designed to meet scientific forecast testing goals described in Schorlemmer Gerstenberger (2005):
+
The ShakeAlert algorithms log their trigger information, and the ShakeAlert Decision Module plots the earthquake parameters released to ShakeAlert users.The event summaries plots show the performance of both the algorithms (Elarms, Virtual Seismologist, and OnSite) as well as the ShakeAlert Decision Modules.
# Transparency
 
# Controlled Environment
 
# Comparability
 
# Reproducibility
 
  
For transparency, CTC evaluation summaries are identified with a CTC software version. When a CTC results is presented, all input data sets used in the calculation, all authorized data sets (observational data), and all source code is accessible to researchers. All calculations can be checked.
+
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results/ ShakeAlert Daily Event Summaries]
  
For controlled environment, CTC evaluation summaries are based on EEW forecasts created during earthquake processing and transmitted to the CTC testing center with no human intervention. Performance summaries are produced by CTC testing group, independent of algorithm developers. Observational data used is obtained from approved and authorized data sources (ANSS Catalog and ShakeMap).
+
== ShakeAlert Cumulative Summaries ==
 +
ShakeAlert cumulative summaries are based on a "selected" start date (current selected at 1 Sept 2011), through current date. The CTC retrieves all events in this time range from ANSS, then filters them in two ways. First, it removes all events less than M3.5. Second, it removes all events outside of the California CSEP Testing area. The CSEP testing area the includes all of California, and a little room on each edge including Nevada, Mexico, and the Pacific ocean.
  
For comparability, CTC evaluation summaries integrate information from multiple algorithms. CTC processing combines information from multiple algorithms producing comparable performance information for difference algorithms for same earthquakes.
+
The four cumulative summaries are:
 +
#Speed Test - Shows time system took to produce a warning. Does not include distribution time from ShakeAlert to users. (Time decision Module has event forecast - event origin time)
 +
#Location Error - Shows difference between final observed location and forecast location (in km). Uses difference between epicenter locations (rather than hypocenter locations).
 +
#Magnitude - Shows difference between (Observed - Forecast) final magnitude. Values above zero show over underestimated magnitudes. Negative values show overestimated magnitudes
 +
#Magnitude Distribution - Shows x-y plot with observed mag on x and forecast mag on y. Accurate forecasts will fall on a diagonal line.
  
For reproducibility, CTC results can be reproduced by retrieving the CTC software version used to produced the summaries. The CTC software can be run interactively, which enables any use to reproduce CTC results.
+
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results/ ShakeAlert Cumulative Summaries] At bottom of page
 +
*[http://scec.usc.edu/research/eew/summary/California/shake-alert-ElarmsV2/results ShakeAlert Cumulative Summaries since Elarms2] at bottom of page
  
== CTC Results ==
+
== Filtered ANSS Catalog of Earthquakes ==
 +
Many of the ShakeAlert evaluation tests are based on comparing ShakeAlert triggers against an observational earthquake catalog. ANSS catalog has been selected as the authoritative catalog.
  
*[[CTC 19 June 2011]]
+
Two catalog filter stages are applied to the raw catalog. First, all events outside the CSEP California Testing Region are dropped. Second, all events below M3.5 are dropped. The resulting catalog is the authoritative catalog used to compare ShakeAlert triggers against.
  
== See Also ==
+
The ANSS earthquake catalog is plotted on California maps for different time frames, showing events over last one month, two months,four months, and six months. Maps of these event catalogs are posted here:
 +
*[http://scec.usc.edu/research/eew/summary/California/anss-tests/observations/ Filtered ANSS Catalog]
  
*[[EEW Evaluation]]
+
The filtered ShakeAlert catalog contains only M3.5 and larger events within the geographical state of California. These catalogs for specific magnitude ranges and date ranges are plotted here:
*[[EEW]]
+
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/observations/ ANSS Raw and ShakeAlert Filtered Event Catalog]
 +
 
 +
== Related Entries ==
 +
*[[CTC Results Pre-2012]]
 +
*[[CISN Testing Center]]
 +
*[[Earthquake Early Warning]]
 +
*[[CSEP]]

Latest revision as of 18:50, 27 February 2013

SCEC NSF-USGS-words logom.png
CISNlogos.png
Swiss Seismological Service-Logo.png

The CISN Testing Center (CTC) consists of scientific analysis software, CISN EEW algorithm performance reports, and earthquake observational data.

ShakeAlert Event Summaries

Event-oriented ShakeAlert performance summaries are generated for each M3.5+ California earthquake found in the ANSS catalog. These ShakeAlert Event summaries show time series of EEW parameters logged by ShakeAlert during real-time operations.

The ShakeAlert algorithms log their trigger information, and the ShakeAlert Decision Module plots the earthquake parameters released to ShakeAlert users.The event summaries plots show the performance of both the algorithms (Elarms, Virtual Seismologist, and OnSite) as well as the ShakeAlert Decision Modules.

ShakeAlert Cumulative Summaries

ShakeAlert cumulative summaries are based on a "selected" start date (current selected at 1 Sept 2011), through current date. The CTC retrieves all events in this time range from ANSS, then filters them in two ways. First, it removes all events less than M3.5. Second, it removes all events outside of the California CSEP Testing area. The CSEP testing area the includes all of California, and a little room on each edge including Nevada, Mexico, and the Pacific ocean.

The four cumulative summaries are:

  1. Speed Test - Shows time system took to produce a warning. Does not include distribution time from ShakeAlert to users. (Time decision Module has event forecast - event origin time)
  2. Location Error - Shows difference between final observed location and forecast location (in km). Uses difference between epicenter locations (rather than hypocenter locations).
  3. Magnitude - Shows difference between (Observed - Forecast) final magnitude. Values above zero show over underestimated magnitudes. Negative values show overestimated magnitudes
  4. Magnitude Distribution - Shows x-y plot with observed mag on x and forecast mag on y. Accurate forecasts will fall on a diagonal line.

Filtered ANSS Catalog of Earthquakes

Many of the ShakeAlert evaluation tests are based on comparing ShakeAlert triggers against an observational earthquake catalog. ANSS catalog has been selected as the authoritative catalog.

Two catalog filter stages are applied to the raw catalog. First, all events outside the CSEP California Testing Region are dropped. Second, all events below M3.5 are dropped. The resulting catalog is the authoritative catalog used to compare ShakeAlert triggers against.

The ANSS earthquake catalog is plotted on California maps for different time frames, showing events over last one month, two months,four months, and six months. Maps of these event catalogs are posted here:

The filtered ShakeAlert catalog contains only M3.5 and larger events within the geographical state of California. These catalogs for specific magnitude ranges and date ranges are plotted here:

Related Entries