Difference between revisions of "CSEP 2 CATALOG FORMAT"

From SCECpedia
Jump to navigationJump to search
Line 2: Line 2:
  
 
== Summary ==
 
== Summary ==
The general philosophy is that the file type should be human readable and easy to use for both researchers and developers. We should support multiple readers to ingest various catalog formats into the PyCSEP codebase.
+
The general philosophy is the file format should be human readable and easy to use for both researchers and developers. (The actual storage of these catalogs in CSEP testing centers might be different than this proposed format).
  
The catalog will initially be defined as an ASCII .csv format where each row corresponds to a single event. This file will have the following headers:<br><br>
+
The catalog will be defined as a (ASCII/utf-8) text-file in csv format. Each row corresponds to a single event. This file will have the following headers:<br><br>
 
<code>
 
<code>
longitude, latitude, '''M''', time_string format="%Y-%m-%d %H:%M:%S.%f", depth, [catalog_id]
+
longitude, latitude, '''M''', time_string format="%Y-%m-%dT%H:%M:%S.%f", depth, [catalog_id]
 
</code>
 
</code>
  
 
<code>'''longitude'''</code>: longitude in decimal degrees</br>
 
<code>'''longitude'''</code>: longitude in decimal degrees</br>
 
<code>'''latitude'''</code>: latitude in decimal degrees</br>
 
<code>'''latitude'''</code>: latitude in decimal degrees</br>
<code>'''M'''</code>: magnitude arbitrary units</br>
+
<code>'''M'''</code>: magnitude</br>
<code>'''time string (UTC)'''</code>: year-month-day hour-minute-second.fraction_second. strptime format="%Y-%m-%d %H:%M:%S.%f": example: 1985-01-01 00:00:00.0</br>
+
<code>'''time string (UTC)'''</code>: year-month-day hour-minute-second.fraction_second. strptime format="%Y-%m-%dT%H:%M:%S.%f": example: 1985-01-01T00:00:00.0</br>
 
<code>'''depth'''</code>: hypocenter depth in (km)</br>
 
<code>'''depth'''</code>: hypocenter depth in (km)</br>
 
<code>'''catalog_id'''</code>: indicates type of catalog</br>
 
<code>'''catalog_id'''</code>: indicates type of catalog</br>
Line 18: Line 18:
 
* simulated: [0, n_cat-1]
 
* simulated: [0, n_cat-1]
  
This format can be easily extended to additional file types, including more advanced storage types like HDF5 or databases. Modelers can chose an arbitrary number of catalogs to store within a file. Events will be mapped to catalogs through the ''catalog_id'' field.
+
This format can be easily extended to additional file types, including more advanced storage including binary, HDF5, or database representations. Modelers can chose an arbitrary number of catalogs to store within a file. Events will be mapped to catalogs through the ''catalog_id'' field, which would prevent the build-up of many, but potentially empty, catalog files.
  
 
== Example Catalog ==
 
== Example Catalog ==

Revision as of 19:07, 26 May 2020

Summary

The general philosophy is the file format should be human readable and easy to use for both researchers and developers. (The actual storage of these catalogs in CSEP testing centers might be different than this proposed format).

The catalog will be defined as a (ASCII/utf-8) text-file in csv format. Each row corresponds to a single event. This file will have the following headers:

longitude, latitude, M, time_string format="%Y-%m-%dT%H:%M:%S.%f", depth, [catalog_id]

longitude: longitude in decimal degrees
latitude: latitude in decimal degrees
M: magnitude
time string (UTC): year-month-day hour-minute-second.fraction_second. strptime format="%Y-%m-%dT%H:%M:%S.%f": example: 1985-01-01T00:00:00.0
depth: hypocenter depth in (km)
catalog_id: indicates type of catalog

  • observed: -1
  • simulated: [0, n_cat-1]

This format can be easily extended to additional file types, including more advanced storage including binary, HDF5, or database representations. Modelers can chose an arbitrary number of catalogs to store within a file. Events will be mapped to catalogs through the catalog_id field, which would prevent the build-up of many, but potentially empty, catalog files.

Example Catalog

Downloadable Example

lon,lat,M,epoch_time,depth,catalog_id,event_id
lon,lat,M,time_string,depth,catalog_id -117.43017,35.616665,4.73,2019-07-06 03:22:35.630000,9.35,-1
-117.7365,35.891,4.64,2019-07-06 03:22:48.300000,9.1,-1
-117.617836,35.803165,4.84,2019-07-06 03:23:50.720000,11.44,-1
-117.67083,35.86067,4.61,2019-07-06 03:25:27.970000,10.32,-1
-117.72583,35.913834,4.5,2019-07-06 03:27:07.010000,8.0,-1
-117.431335,35.530334,4.57,2019-07-06 03:27:11.370000,3.83,-1
-117.7115,35.902832,4.51,2019-07-06 03:29:32.080000,3.18,-1
-117.35817,35.556667,4.49,2019-07-06 03:30:25.050000,8.71,-1
-117.73033,35.890335,4.17,2019-07-06 03:32:46.660000,6.15,-1
-117.505165,35.714832,4.13,2019-07-06 03:33:09.850000,1.74,-1
-117.466835,35.65217,4.09,2019-07-06 03:35:05.420000,1.97,-1
-117.73383,35.902832,4.35,2019-07-06 03:36:16.460000,7.27,-1