Difference between revisions of "CTC Results"

From SCECpedia
Jump to navigationJump to search
 
(32 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
[[Image:Swiss_Seismological_Service-Logo.png‎|right|40px]]
 
[[Image:Swiss_Seismological_Service-Logo.png‎|right|40px]]
  
 +
The CISN Testing Center (CTC) consists of scientific analysis software, CISN EEW algorithm performance reports, and earthquake observational data.
  
The CISN Testing Center (CTC) consists of scientific analysis software, CISN EEW algorithm performance reports, and earthquake observational data. This entry is a collection point for current and past results from the CTC testing center.
+
== ShakeAlert Event Summaries ==
 +
Event-oriented ShakeAlert performance summaries are generated for each M3.5+ California earthquake found in the ANSS catalog. These ShakeAlert Event summaries show time series of EEW parameters logged by ShakeAlert during real-time operations.  
  
== Type of ShakeAlert Performance Summaries ==
+
The ShakeAlert algorithms log their trigger information, and the ShakeAlert Decision Module plots the earthquake parameters released to ShakeAlert users.The event summaries plots show the performance of both the algorithms (Elarms, Virtual Seismologist, and OnSite) as well as the ShakeAlert Decision Modules.
We current support four event oriented summaries, and four cumulative summaries.
 
  
== Cumulative Summaries ==
+
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results/ ShakeAlert Daily Event Summaries]
The cumulative summaries are based on a "selected" start date (current selected at 1 Sept 2011), through current date. The CTC retrieves all events in this time range from ANSS, then filters them in two ways. First, it removes all events less than M3.5. Second, it removes all events outside of the California CSEP Testing area. This is an area the includes all of California, and a little room on each edge including Nevada, Mexico, and the Pacific ocean.
 
  
The input catalog (that is all events in ANSS catalog), and the filtered catalog, are available
+
== ShakeAlert Cumulative Summaries ==
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/observations/ (scec.eew.Catalog.EEWANSSOneDay-EEWANSSCatalog.dat.1328657526.726369.1) ANSS Raw Event Catalog]
+
ShakeAlert cumulative summaries are based on a "selected" start date (current selected at 1 Sept 2011), through current date. The CTC retrieves all events in this time range from ANSS, then filters them in two ways. First, it removes all events less than M3.5. Second, it removes all events outside of the California CSEP Testing area. The CSEP testing area the includes all of California, and a little room on each edge including Nevada, Mexico, and the Pacific ocean.
  
*http://scec.usc.edu/research/eew/summary/California/shake-alert/observations/  (EEWANSSCatalog.dat.Magnitude3.5) ShakeAlert Filtered Event Catalog]
+
The four cumulative summaries are:
 +
#Speed Test - Shows time system took to produce a warning. Does not include distribution time from ShakeAlert to users. (Time decision Module has event forecast - event origin time)
 +
#Location Error - Shows difference between final observed location and forecast location (in km). Uses difference between epicenter locations (rather than hypocenter locations).
 +
#Magnitude - Shows difference between (Observed - Forecast) final magnitude. Values above zero show over underestimated magnitudes. Negative values show overestimated magnitudes
 +
#Magnitude Distribution - Shows x-y plot with observed mag on x and forecast mag on y. Accurate forecasts will fall on a diagonal line.
  
 +
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results/ ShakeAlert Cumulative Summaries] At bottom of page
 +
*[http://scec.usc.edu/research/eew/summary/California/shake-alert-ElarmsV2/results ShakeAlert Cumulative Summaries since Elarms2] at bottom of page
  
== Daily Event Processing ==
+
== Filtered ANSS Catalog of Earthquakes ==
The automated nightly evaluation results are posted at the following location.
+
Many of the ShakeAlert evaluation tests are based on comparing ShakeAlert triggers against an observational earthquake catalog. ANSS catalog has been selected as the authoritative catalog.  
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results CTC Automated Daily ShakeAlert Testing Results]
 
  
== CTC ShakeAlert Testing ==
+
Two catalog filter stages are applied to the raw catalog. First, all events outside the CSEP California Testing Region are dropped. Second, all events below M3.5 are dropped. The resulting catalog is the authoritative catalog used to compare ShakeAlert triggers against.
As of September 2011, [[CISN Testing Center]] (CTC) processing now evaluates performance of the prototype CISN ShakeAlert EEW system. CTC evaluations are now based on earthquake parameter forecasts as logged by the CISN ShakeAlert system at U.C. Berkeley.
 
  
This testing include inputs from three algorithms (UCB Elarms, ETH Virtual Seismologist, and Caltech Onsite) as well as processing outputs from the ShakeAlert Decision Module.  
+
The ANSS earthquake catalog is plotted on California maps for different time frames, showing events over last one month, two months,four months, and six months. Maps of these event catalogs are posted here:
 +
*[http://scec.usc.edu/research/eew/summary/California/anss-tests/observations/ Filtered ANSS Catalog]
  
The CTC automated testing center uses the following processing steps in its evaluation:
+
The filtered ShakeAlert catalog contains only M3.5 and larger events within the geographical state of California. These catalogs for specific magnitude ranges and date ranges are plotted here:
# Evaluations are run once per day.
+
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/observations/ ANSS Raw and ShakeAlert Filtered Event Catalog]
# In order to let evaluation EEW alerts against "final" "stable" earthquake parameters, evaluations are delay for two days. That is on Tuesday, evaluations are done for the previous Sunday.
 
# Evaluation processing retrieves observed earthquake information from the ANSS catalog.
 
# ANSS catalog entries are filtered and earthquake below Mag 2.5 or outside the state of California are discarded.
 
# Evaluation processing retrieves ShakeAlert log files from operational server at UCB.
 
# For each ANSS earthquake, we look for one or more ShakeAlert triggers with origin time within 45 seconds of the observed earthquake.
 
# Plots are produced showing final (observed, ANSS) magnitude as compared to algorithm alerts from VS, OnSite, and ElarmS and Decision Module alerts (as sent to users).
 
 
 
The automated nightly evaluation results are posted at the following location.
 
*[http://scec.usc.edu/research/eew/summary/California/shake-alert/results CTC Automated Daily ShakeAlert Testing Results]
 
*[http://scec.usc.edu/research/eew/summary/California/shakecast-tests/ CTC Automated Daily ShakeCast Testing Results]
 
 
 
[[File:Scec.eew.EEWTest.eewTest ShakeAlertMagnitude-Test Magnitude3.5 Event11010669.png.1316761668.413857.1.png|right|350px|Fig 1. Example CTC Performance Summary for ShakeAlert on recent California earthquake showing Algorithm performance (Virtual Seismologists) and Decision Module performance (ShakeAlert).]]
 
 
 
[[File:VS_OnSite_and_ShakeAlert.png|right|350px|Fig 2. Example CTC Performance Summary for ShakeAlert on recent California earthquake showing Algorithm performance (Virtual Seismologists and OnSite) and Decision Module performance (ShakeAlert).]]
 
 
 
== CTC Interactive Performance Summaries ==
 
Prior to September 2011, CTC testing center produced interactive test results for individual EEW algorithms. These results lack information about the integrated ShakeAlert system. The most recent CTC development, described above, now include evaluation information about both individual algorithms as well as overall ShakeAlert performance. The interactive CTC performance summaries produced in the past are posted below, but this software is no longer under active development.
 
 
 
Interactive performance summaries are produced with a specific version of the CTC software in order to manage changes and improvements to the CTC processing software. These performance summaries are cumulative, with later summaries containing more events. Interactive results are reproducible by scientists as both the input EEW performance reports, and processing source code, are available from the CTC testing center.
 
 
 
*[[CTC 19 June 2011]]
 
 
 
== See Also ==
 
  
 +
== Related Entries ==
 +
*[[CTC Results Pre-2012]]
 
*[[CISN Testing Center]]
 
*[[CISN Testing Center]]
 
*[[Earthquake Early Warning]]
 
*[[Earthquake Early Warning]]
 
*[[CSEP]]
 
*[[CSEP]]

Latest revision as of 18:50, 27 February 2013

SCEC NSF-USGS-words logom.png
CISNlogos.png
Swiss Seismological Service-Logo.png

The CISN Testing Center (CTC) consists of scientific analysis software, CISN EEW algorithm performance reports, and earthquake observational data.

ShakeAlert Event Summaries

Event-oriented ShakeAlert performance summaries are generated for each M3.5+ California earthquake found in the ANSS catalog. These ShakeAlert Event summaries show time series of EEW parameters logged by ShakeAlert during real-time operations.

The ShakeAlert algorithms log their trigger information, and the ShakeAlert Decision Module plots the earthquake parameters released to ShakeAlert users.The event summaries plots show the performance of both the algorithms (Elarms, Virtual Seismologist, and OnSite) as well as the ShakeAlert Decision Modules.

ShakeAlert Cumulative Summaries

ShakeAlert cumulative summaries are based on a "selected" start date (current selected at 1 Sept 2011), through current date. The CTC retrieves all events in this time range from ANSS, then filters them in two ways. First, it removes all events less than M3.5. Second, it removes all events outside of the California CSEP Testing area. The CSEP testing area the includes all of California, and a little room on each edge including Nevada, Mexico, and the Pacific ocean.

The four cumulative summaries are:

  1. Speed Test - Shows time system took to produce a warning. Does not include distribution time from ShakeAlert to users. (Time decision Module has event forecast - event origin time)
  2. Location Error - Shows difference between final observed location and forecast location (in km). Uses difference between epicenter locations (rather than hypocenter locations).
  3. Magnitude - Shows difference between (Observed - Forecast) final magnitude. Values above zero show over underestimated magnitudes. Negative values show overestimated magnitudes
  4. Magnitude Distribution - Shows x-y plot with observed mag on x and forecast mag on y. Accurate forecasts will fall on a diagonal line.

Filtered ANSS Catalog of Earthquakes

Many of the ShakeAlert evaluation tests are based on comparing ShakeAlert triggers against an observational earthquake catalog. ANSS catalog has been selected as the authoritative catalog.

Two catalog filter stages are applied to the raw catalog. First, all events outside the CSEP California Testing Region are dropped. Second, all events below M3.5 are dropped. The resulting catalog is the authoritative catalog used to compare ShakeAlert triggers against.

The ANSS earthquake catalog is plotted on California maps for different time frames, showing events over last one month, two months,four months, and six months. Maps of these event catalogs are posted here:

The filtered ShakeAlert catalog contains only M3.5 and larger events within the geographical state of California. These catalogs for specific magnitude ranges and date ranges are plotted here:

Related Entries