EEW Evaluation

Performance Evaluation Summaries

Two descriptions of performance summaries include:

• CISN Testing Document
• CTC Group Evaluation Collection

We separate the summaries into "Event" Summaries and "Catalog" Summaries where catalog means any set of earthquakes, and the summary must make it clear what criteria were used to define the catalog.

CISN On Site Trigger

An example CISN Trigger file is posted here:

We noted several things about the current formatting.

1. The report id never changes. The intention was that this uniquely identified the file. But it is currently fixed.
2. Forecasts PGV and Mag only
3. Reports both algorithm and alert time, which always seem to have a delta of 2 seconds. This may mean difference between the two is the amount of data needed to produce the alert
4. Multiple forecasts per trigger file
5. Multiple events per trigger file
6. Seems to report duplicate forecast information for each trigger. One copy of the trigger also contains observational data for the trigger. It is unclear how the trigger is match to the event id. We noted that in some cases the event id reported in the trigger file is not the final ANSS event id.
7. The event id does not have a network id so we assume it is CI network event

VS Trigger Reports

VS is running on southern California Events. We noted the following.

1. Reports one event per trigger file
2. Each trigger file reports source of report
3. Each trigger file contains a set of time series
4. It seems that all time series in the trigger report have the same timestamps
5. Event oriented time series include:
   1. magnitude
   2. lat
   3. lon
   4. origin time
6. Region-oriented time series include
   1. pgv
   2. pga

ShakeMap Data

We are currently working with the ShakeMap stationlist txt file. We may convert to the stationlist xml files, but it was easier to parse the column oriented to start.

We use the ShakeMap data as reference observational data for station amplitude values.

We have identified several characteristics to the ShakeMap observational reports.

1. Two main formats are the CI: and NN: events, and ECX events. ECX events have their pgv and pga values in different columns than the others, so this type of ShakeMap report must be checked for while parsing.
2. ShakeMap files also have different number of columns populated. Currently we require that both PGA and PGV are listed before we use it as a authorized data observation. The goal is to maximize the number of observations we have to compare to forecasts. Right now, we don't have forecasts at PGV or PGA only sites. Once we do, we should add small modification to track observations even if they only have one component of data.
3. We pull event ID from the header of the Shakemap file. However, it does not list the network, which we need to match it up in the ANSS catalog.

Processing Flow

Current design is based on processing one or more trigger reports. Currently two types of trigger reports are supported CIT On Site, and ETH Virtual Seismologist.

1. Init CISN Station list, ANSS EQ catalog, ShakeMap catalog
2. For each Trigger File
   1. Mapped to One or More ANSS Events and observed data from ShakeMap
3. For each event in trigger file
   1. write event summary
   2. Add to catalog summaries
4. write catalog summaries

ANSS Catalog Files

We create a catalog of ANSS events based on the ANSS web-based catalog search capabilities. The query parameters for the ANSS catalog are in the header of the file. This catalog represents the authorized reference data for events in the testing region. While we use ShakeMap data files to get station information, we use ANSS catalog as the reference event catalog.

• Example ANSS Catalog File

Software Download

CTC Software is posted in git repository on github. maechlin:ctc

Related Entries

• Earthquake Early Warning
• M8 EEW Analysis

See Also

• Main Page