Accessing CyberShake Seismograms - Revision history

Scottcal at 18:48, 5 May 2023

2023-05-05T18:48:22Z

Scottcal: /* Seismogram Format */

2023-05-05T18:47:55Z

Seismogram Format

Scottcal: /* Seismogram Format */

2023-05-05T18:22:34Z

Seismogram Format

Scottcal at 22:37, 6 June 2022

2022-06-06T22:37:36Z

Scottcal: /* Identifying Seismograms of Interest */

2022-06-06T22:37:26Z

Identifying Seismograms of Interest

Scottcal: /* Seismogram format */

2022-06-04T04:51:40Z

Seismogram format

Scottcal at 04:51, 4 June 2022

2022-06-04T04:51:29Z

Scottcal: /* Data organization */

2022-06-03T21:30:35Z

Data organization

Scottcal at 21:30, 3 June 2022

2022-06-03T21:30:08Z

Scottcal: /* Data organization */

2022-06-03T21:27:38Z

Data organization

@@ Line 74: / Line 74: @@
 Sample Python and C code for reading out the seismogram contents for a seismogram with just 1 rupture variation is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
 <pre>
@@ Line 134: / Line 136: @@
 print("seis_data[Y][1000] = %f" % s.y_data[1000])
 </pre>
 <pre>

@@ Line 73: / Line 73: @@
 However, the rupture variations are not necessarily in order in the file.  So if you are only interested in a subset of the rupture variations, you will have to read the header data to find the rupture variations you are interested in.  A description of the header structure is given in [[CyberShake_output_data_headers#Seismogram_header]].  C and Python examples to read the header information are provided there.
-Sample Python and C code for reading out the seismogram contents is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
+Sample Python and C code for reading out the seismogram contents for a seismogram with just 1 rupture variation is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
 <pre>
-#!/usr/bin/env python
+!/usr/bin/env python3
 class Seismogram:
-−
      def __init__(self, filename):
          self.filename = filename
@@ Line 86: / Line 89: @@
      def parseHeader(self, header_str):
-         header_str = fp_in.read(56)
+         #Find null terminator
-         version = header_str[0:8]
+        split_point = header_str[0:8].find(b'\x00')
-         if version[0:5]!="12.10":
+         self.version = header_str[0:split_point].decode()
-            print "Error: version does not match expected string '12.10', aborting."
+         if self.version!="12.10":
             sys.exit(2)
-         site = header_str[8:16]
+         #Find null terminator
-         source_id = struct.unpack('i', header_str[24:28])
+        split_point = header_str[8:16].find(b'\x00')
-         rupture_id = struct.unpack('i', header_str[28:32])
+        self.site = header_str[8:8+split_point].decode()
-         rup_var_id = struct.unpack('i', header_str[32:36])
+         self.source_id = struct.unpack('i', header_str[24:28])[0]
-         dt = struct.unpack('f', header_str[36:40])
+         self.rupture_id = struct.unpack('i', header_str[28:32])[0]
-         nt = struct.unpack('i', header_str[40:44])
+         self.rup_var_id = struct.unpack('i', header_str[32:36])[0]
-         comps = struct.unpack('i', header_str[44:48])
+         self.dt = struct.unpack('f', header_str[36:40])[0]
-         x_flag = y_flag = z_flag = False
+         self.nt = struct.unpack('i', header_str[40:44])[0]
-         if (comps & 1)==1:
+         self.comps = struct.unpack('i', header_str[44:48])[0]
-             x_flag = True
+         self.x_flag = self.y_flag = self.z_flag = False
-         if (comps & 2)==2:
+         if (self.comps & 1)==1:
-             y_flag = True
+             self.x_flag = True
-         if (comps & 4)==4:
+         if (self.comps & 2)==2:
-             z_flag = True
+             self.y_flag = True
-         det_max_freq = struct.unpack('f', header_str[48:52])[0]
+         if (self.comps & 4)==4:
-         stoch_max_freq = struct.unpack('f', header_str[52:56])[0]
+             self.z_flag = True
+         self.det_max_freq = struct.unpack('f', header_str[48:52])[0]
      def readData(self):
          fp_in = open(self.filename, "rb")
@@ Line 125: / Line 131: @@
 s = Seismogram(seis_file)
 s.readData()
+print("seis_data[X][1000] = %f" % s.x_data[1000])
 </pre>

@@ Line 73: / Line 73: @@
 However, the rupture variations are not necessarily in order in the file.  So if you are only interested in a subset of the rupture variations, you will have to read the header data to find the rupture variations you are interested in.  A description of the header structure is given in [[CyberShake_output_data_headers#Seismogram_header]].  C and Python examples to read the header information are provided there.
-Sample C code for reading out the seismogram contents is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
+Sample Python and C code for reading out the seismogram contents is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
 <pre>

@@ Line 3: / Line 3: @@
 == Identifying Seismograms of Interest ==
-The first step is to figure out which seismograms are of interest.  You'll need to access the database to determine this; instructions are available at [Accessing CyberShake Database Data]. To do this, you'll need to know a) the site and run ID of the CyberShake run, and b) the source id, rupture id, and rupture variation id of the seismogram.  You may have done this already.  If not, you can determine the run ID by using MySQL to query the CyberShake database on focal.usc.edu.  To determine a run ID, you will need to know values for a few parameters, such as the ERF, the velocity model, and the rupture variation generator version.  Here's a sample query:
+The first step is to figure out which seismograms are of interest.  You'll need to access the database to determine this; instructions are available at [[Accessing CyberShake Database Data]]. To do this, you'll need to know a) the site and run ID of the CyberShake run, and b) the source id, rupture id, and rupture variation id of the seismogram.  You may have done this already.  If not, you can determine the run ID by using MySQL to query the CyberShake database on focal.usc.edu.  To determine a run ID, you will need to know values for a few parameters, such as the ERF, the velocity model, and the rupture variation generator version.  Here's a sample query:
   select R.Run_ID from CyberShake_Runs R, CyberShake_Sites S

@@ Line 3: / Line 3: @@
 == Identifying Seismograms of Interest ==
-The first step is to figure out which seismograms are of interest.  To do this, you'll need to know a) the site and run ID of the CyberShake run, and b) the source id, rupture id, and rupture variation id of the seismogram.  You may have done this already.  If not, you can determine the run ID by using MySQL to query the CyberShake database on focal.usc.edu.  To determine a run ID, you will need to know values for a few parameters, such as the ERF, the velocity model, and the rupture variation generator version.  Here's a sample query:
+The first step is to figure out which seismograms are of interest.  You'll need to access the database to determine this; instructions are available at [Accessing CyberShake Database Data]. To do this, you'll need to know a) the site and run ID of the CyberShake run, and b) the source id, rupture id, and rupture variation id of the seismogram.  You may have done this already.  If not, you can determine the run ID by using MySQL to query the CyberShake database on focal.usc.edu.  To determine a run ID, you will need to know values for a few parameters, such as the ERF, the velocity model, and the rupture variation generator version.  Here's a sample query:
   select R.Run_ID from CyberShake_Runs R, CyberShake_Sites S

@@ Line 52: / Line 52: @@
 The data is organized by site name and run ID.  So, for example, there's a directory for site ACTN and then run ID 4296.  Inside this directory are all the seismograms, PSA, RotD, and duration files for site ACTN, run ID 4296.  The filenames are <Seismogram | PeakVals | RotD | Duration>_<site>_<source ID>_<rupture ID>.<grm | bsa | rotd | dur> .  The data for each run of Study 15.12 is approximately 90 GB.
-== Seismogram format ==
+== Seismogram Format ==
 The seismogram file Seismogram_<site>_<sourceID>_<ruptureID>.grm contains the seismograms for all the rupture variations for the given source ID.

@@ Line 50: / Line 50: @@
 === Data organization ===
-The data is organized by site name and run ID.  So, for example, there's a directory for site ACTN and then run ID 4296.  Inside this directory are all the seismograms, PSA, RotD, and duration files for site ACTN, run ID 4296.  The data for each run of Study 15.12 is approximately 90 GB.
+The data is organized by site name and run ID.  So, for example, there's a directory for site ACTN and then run ID 4296.  Inside this directory are all the seismograms, PSA, RotD, and duration files for site ACTN, run ID 4296.  The filenames are <Seismogram | PeakVals | RotD | Duration>_<site>_<source ID>_<rupture ID>.<grm | bsa | rotd | dur> .  The data for each run of Study 15.12 is approximately 90 GB.
-==== Using the Seismogram Retrieval Script ====
+The seismograms are velocity seismograms, with <b>units of cm/s</b>.  The number of timesteps and dt depends on what frequency the run was performed at, as well as whether the run included stochastic high-frequency results, and can be determined by reading the header information.
-Once you know the run ID and the event of interest, you can use a Python script, /home/scec-02/cybershk/utils/retrieve_seismogram.py (and in SVN at https://source.usc.edu/svn/cybershake/import/trunk/Utils/retrieve_seismogram.py), to retrieve the seismogram for you.  It takes as arguments the run ID, source ID, rupture ID, rupture variation ID, output file, and if you want the seismogram header information included or not.  This script automatically searches the directories below, retrieves the specific two-component seismogram of interest, and writes an output file.  It is highly recommended to use this script rather than looking for the file by hand.
+The format is (binary):
-==== Locating Seismogram Files by Hand ====
+  <Rupture Variation 1 56-byte header>
   <Rupture Variation 1, X component - NT 4-byte floats>
   <Rupture Variation 1, Y component - NT 4-byte floats>
-  <Rupture Variation 2 header>
+  <Rupture Variation 2 56-byte header>
   <Rupture Variation 2, X component - NT 4-byte floats>
   <Rupture Variation 2, Y component - NT 4-byte floats>
   ...
-  <Rupture Variation n header>
+  <Rupture Variation n 56-byte header>
   <Rupture Variation n, X component - NT 4-byte floats>
   <Rupture Variation n, Y component - NT 4-byte floats>
-Where this gets tricky is that the rupture variations are not necessarily in order in the file.  So you will have to read the header data to find the rupture variations you are interested in.  A description of the header structure is given in [[CyberShake_output_data_headers#Seismogram_header]].  C and Python examples to read the header information are provided there.
+However, the rupture variations are not necessarily in order in the file.  So if you are only interested in a subset of the rupture variations, you will have to read the header data to find the rupture variations you are interested in.  A description of the header structure is given in [[CyberShake_output_data_headers#Seismogram_header]].  C and Python examples to read the header information are provided there.
 Sample C code for reading out the seismogram contents is below.  This code prints the 1000th timesteps for X and Y components, but obviously you could do whatever is needed with the arrays.
@@ Line 185: / Line 120: @@
 </pre>
-==== Old (Run ID < 1310) Seismogram Format ====
+<!-- === Plotting Seismograms ===
 If you have new-format seismogram files, you can use the script /home/scec-00/cybershk/scripts/plot_N_seis.py to plot the X and Y components of multiple seismograms on the same plot.  The script takes the seismogram files, the legends, a plot title, and an output file as arguments.  If you prefer to plot the spectral content of the seismograms, you can use the script /home/scec-00/cybershk/scripts/plot_N_seis_spectral.py .
@@ Line 202: / Line 128: @@
 === Comparing Seismograms ===
-If you want to compare two seismograms, you can use the script /home/scec-00/cybershk/scripts/compare_seis.py, The script compares two 2-component seismograms point-by-point to see if they exceed a specified tolerance.
+If you want to compare two seismograms, you can use the script /home/scec-00/cybershk/scripts/compare_seis.py, The script compares two 2-component seismograms point-by-point to see if they exceed a specified tolerance.//-->

@@ Line 50: / Line 50: @@
 === Data organization ===
-The data is organized by site name and run ID.  So, for example, there's a directory for site ACTN and then run ID 4296.  Inside this directory are all the seismograms, PSA, RotD, and duration files for site ACTN, run ID 4296.  The data for each site is approximately 90 GB.
+The data is organized by site name and run ID.  So, for example, there's a directory for site ACTN and then run ID 4296.  Inside this directory are all the seismograms, PSA, RotD, and duration files for site ACTN, run ID 4296.  The data for each run of Study 15.12 is approximately 90 GB.
@@ Line 118: / Line 118: @@
   Seismogram found.
 Therefore, all the seismograms for run ID 4749 are in /home/scec-04/tera3d/CyberShake/data/PPFiles/USC/4749 .
 === Reading Seismogram Files ===

@@ Line 1: / Line 1: @@
-We are in the process of migrating seismograms from past CyberShake studies to Globus collections, to simplify access for users.  Currently, only CyberShake Study 15.12 is available through Globus, but if you are interested in accessing seismograms from a different CyberShake study, please email software@scec.org to make a request.
+This page details the process of identifying and accessing CyberShake seismograms.
-Once you've connected to the collection, you can transfer the data anywhere with a Globus endpoint - [https://docs.globus.org/how-to/get-started/#request_a_file_transfer details here].  Many clusters, universities, and HPC centers have Globus endpoints set up already.  If you can't find an endpoint where you need one, or you'd prefer to transfer the data to your local laptop or desktop, you can create your own endpoint using [https://www.globus.org/globus-connect-personal Globus Connect Personal].
+== Identifying Seismograms of Interest ==
 The first step is to figure out which seismograms are of interest.  To do this, you'll need to know a) the site and run ID of the CyberShake run, and b) the source id, rupture id, and rupture variation id of the seismogram.  You may have done this already.  If not, you can determine the run ID by using MySQL to query the CyberShake database on focal.usc.edu.  To determine a run ID, you will need to know values for a few parameters, such as the ERF, the velocity model, and the rupture variation generator version.  Here's a sample query:
@@ Line 49: / Line 26: @@
 To understand the different slips, the easiest thing to do is to plot them.
-=== Finding the seismogram ===
+== Retrieving Seismograms ==
 ==== Using the Seismogram Retrieval Script ====