Difference between revisions of "CTC Results"
| Line 1: | Line 1: | ||
| + | [[CISN Testing Center]] evaluation results are presented as reference performance metrics. An important goal of a collaborative testing capabilities is the ability to compare equivalent results. As of 20 June 2011, CTC testing results are based on CISN EEW algorithms contributing performance reports to the SCEC data center. Current CTC performance summaries are based on CISN algorithms performance. Evaluation of CISN ShakeAlert system is under development. | ||
| + | |||
[[Image:SCEC_NSF-USGS-words_logom.png|left|150px]][[Image:CISNlogos.png|right|250px]][[Image:usgs-logo-color.jpg|right|150px]] | [[Image:SCEC_NSF-USGS-words_logom.png|left|150px]][[Image:CISNlogos.png|right|250px]][[Image:usgs-logo-color.jpg|right|150px]] | ||
| − | [[ | + | == CTC Interactive Results == |
| + | |||
| + | *[[CTC 19 June 2011]] | ||
| + | |||
| + | == CTC Automated Processing == | ||
| − | == | + | *[http://scec.usc.edu/research/eew/summary/California/shakecast-tests/ CTC Test Results Directory] |
| + | |||
| + | == CTC Design Goals == | ||
CTC is designed to meet scientific forecast testing goals described in Schorlemmer Gerstenberger (2005): | CTC is designed to meet scientific forecast testing goals described in Schorlemmer Gerstenberger (2005): | ||
| − | + | *Transparency | |
| − | + | *Controlled Environment | |
| − | + | *Comparability | |
| − | + | *Reproducibility | |
| + | |||
| + | '''Transparency''' | ||
For transparency, CTC evaluation summaries are identified with a CTC software version. When a CTC results is presented, all input data sets used in the calculation, all authorized data sets (observational data), and all source code is accessible to researchers. All calculations can be checked. | For transparency, CTC evaluation summaries are identified with a CTC software version. When a CTC results is presented, all input data sets used in the calculation, all authorized data sets (observational data), and all source code is accessible to researchers. All calculations can be checked. | ||
| + | |||
| + | '''Controlled Environment''' | ||
For controlled environment, CTC evaluation summaries are based on EEW forecasts created during earthquake processing and transmitted to the CTC testing center with no human intervention. Performance summaries are produced by CTC testing group, independent of algorithm developers. Observational data used is obtained from approved and authorized data sources (ANSS Catalog and ShakeMap). | For controlled environment, CTC evaluation summaries are based on EEW forecasts created during earthquake processing and transmitted to the CTC testing center with no human intervention. Performance summaries are produced by CTC testing group, independent of algorithm developers. Observational data used is obtained from approved and authorized data sources (ANSS Catalog and ShakeMap). | ||
| + | |||
| + | '''Comparability''' | ||
For comparability, CTC evaluation summaries integrate information from multiple algorithms. CTC processing combines information from multiple algorithms producing comparable performance information for difference algorithms for same earthquakes. | For comparability, CTC evaluation summaries integrate information from multiple algorithms. CTC processing combines information from multiple algorithms producing comparable performance information for difference algorithms for same earthquakes. | ||
| + | |||
| + | '''Reproducibility''' | ||
For reproducibility, CTC results can be reproduced by retrieving the CTC software version used to produced the summaries. The CTC software can be run interactively, which enables any use to reproduce CTC results. | For reproducibility, CTC results can be reproduced by retrieving the CTC software version used to produced the summaries. The CTC software can be run interactively, which enables any use to reproduce CTC results. | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
== See Also == | == See Also == | ||
Revision as of 06:07, 20 June 2011
CISN Testing Center evaluation results are presented as reference performance metrics. An important goal of a collaborative testing capabilities is the ability to compare equivalent results. As of 20 June 2011, CTC testing results are based on CISN EEW algorithms contributing performance reports to the SCEC data center. Current CTC performance summaries are based on CISN algorithms performance. Evaluation of CISN ShakeAlert system is under development.
CTC Interactive Results
CTC Automated Processing
CTC Design Goals
CTC is designed to meet scientific forecast testing goals described in Schorlemmer Gerstenberger (2005):
- Transparency
- Controlled Environment
- Comparability
- Reproducibility
Transparency
For transparency, CTC evaluation summaries are identified with a CTC software version. When a CTC results is presented, all input data sets used in the calculation, all authorized data sets (observational data), and all source code is accessible to researchers. All calculations can be checked.
Controlled Environment
For controlled environment, CTC evaluation summaries are based on EEW forecasts created during earthquake processing and transmitted to the CTC testing center with no human intervention. Performance summaries are produced by CTC testing group, independent of algorithm developers. Observational data used is obtained from approved and authorized data sources (ANSS Catalog and ShakeMap).
Comparability
For comparability, CTC evaluation summaries integrate information from multiple algorithms. CTC processing combines information from multiple algorithms producing comparable performance information for difference algorithms for same earthquakes.
Reproducibility
For reproducibility, CTC results can be reproduced by retrieving the CTC software version used to produced the summaries. The CTC software can be run interactively, which enables any use to reproduce CTC results.
