CyberShake Data Request
CyberShake Data Request Web Site is a first stage in automating CyberShake data requests. The variety of CyberShake data products is growing, so we propose to develop a web site that will help user request CyberShake data. We will use current CyberShake data requests to try out the proposed data request system. The goal of this first system is to generate a well defined data request. For now, we will continue to process the data requests manually, but will look to automating the retrieval if useful.
Contents
Prototype CyberShake Data Request Site
General Workflow
At a SCEC URL, the user is presented with a series of choices that selects some subset of all our CyberShake data products. The user makes all the choices and presses send and the web page generates a data request specification that someone can then processes and post. The user can check on the status of their request while waiting.
Initially, the user can select from an official CyberShake Study (13.4, 2.2, or 1.0) and then select the map from which the results should be retrieved (e.g. RWG CVM-S). The user can then select the sites from which they would like the results, either visually via a map or textually via a table. Should they wish to have results from the whole map, there is the option to "select all" the sites. After this, the user is presented with a list of the possible data products that the system can retrieve. We are developing support for the following data products:
- Maps
- Hazard Curves
- Seismograms
- Peak Amplitudes
- List of Ruptures and Disaggregation
Upon selecting the data product desired, filling out their e-mail and name, the system then proceeds to perform the necessary computations to serve that request. When it is finished, the system notifies the user via e-mail and a download link is provided.
The system was designed with the following use cases in mind.
Use Case 1
User would like the Graves-Pitarka CVM-S hazard map from CyberShake Study 13.4. User can go to the site, select 13.4, Graves-Pitarka CVM-S, all sites, with data product of map. The system will then calculate and return that map.
Use Case 2
User would like the hazard curve for USC from the Graves-Pitarka CVM-S map from CyberShake Study 13.4. User can go to site, select 13.4, Graves-Pitarka CVM-S, site USC only, and then select hazard curve. The system will generate the hazard curve and notify the user.
Use Case 3
User would like to see visual seismograms available for USC from the AWP CVM-H map from CyberShake Study 13.4. User can go to the site, select 13.4, AWP CVM-H, site USC, and then select seismograms as the data product. The system will generate images and return all applicable seismograms in PNG format.
Use Case 4
User would like to get a list of the top 20 sources that contribute to the hazard at USC from the Study 13.4 AWP CVM-S map. User can go to the site, select 13.4 , AWP CVM-S, site USC, and then select rupture list as the data product, sorted by top contributors.
Use Case 5
User would like to get a list of the peak amplitudes for given periods at USC from the Study 13.4 AWP CVM-H map. User can go to the site, select 13.4, AWP CVM-H, site USC, and then select the data product of peak amplitudes.
XML Document
When the front-end steps are completed a XML document is generated. The format of this XML document is:
<datarequest>
<first_name>John</first_name>
<last_name>Doe</last_name>
<email>john.doe@usc.edu</email>
<study>CyberShake 13.4</study>
<map>AWP CVM-S</map>
<sites>
<site>USC</site>
</sites>
<ruptures>none</ruptures>
<product>
<name>Hazard Curves</name>
<format>log-linear</format>
...
</product>
</datarequest>
This is then used by the backend processor, described below, to finalize and fulfill the request.
Backend Workflow
The backend is comprised of three components:
- csdrs.py
- requestprocessor.py
- Separate Python scripts for each data product (e.g. map.py, peakamps.py, etc.)
csdrs.py is an interface Python script allowing for the following functionality:
- List requests (state) - get a list of all requests in the system matching state.
- Submit request (request XML) - submits request XML to be parsed by requestprocessor.py, returns a request ID
- Cancel request (request ID) - halts processing on request ID
- Status of request (request ID) - gets the status of a particular request ID
When a job is submitted, it spawns a new requestprocessor.py and records the id of that thread (for canceling a request). Requestprocessor.py is intended to be a background process and takes an XML document as input.
User Choices CyberShake Data Sets
Initial User Choices of data sets available:
- CyberShake 1.0 Map
- CyberShake 2.2 Map
- CyberShake 13.4 Map
Other User Choices of data sets:
- All sites rupture generator 2.X
- All sites with new rupture generator 3.x
- PBR sites
- 10Hz sites
- CVM-H sites
User Choices Output Formats
Data products output formats (user-define options):
- Maps (POE@IMT or IMT@POE)
- Curves (IMT duration POE)
- Rupture Sets (ERF, Rupture Generator version, Site of Interest)
- Seismograms (site name, simulation id)
- Amplitudes (site name, rupture set, imt)
- ERF Information (erf)
- Velocity Profiles (site, cvm, sgt)
- Single Component Hazard curves (site, rotation angle, imt)
Progress
The CyberShake data request system is presently being worked on and is currently in the pre-alpha stage. Presently, we have the following features:
- Select from CyberShake studies or from a combination of rupture forecasts, SGT generators, rupture generators, velocity models, and frequencies
- Select sites from a visual (on Google Maps) list of sites as well as a standard row-based one
- Filter sites based off of categories (e.g. precarious rocks, broadband sites, etc.)
- Select relevant data products and configure options for them
- Generates well-formed XML document for either human or machine use
- Administrative panel to see requests and update links for download
- Check on status of request via web interface
