Difference between revisions of "CyberShake DAX Generator"
From SCECpedia
Jump to navigationJump to search| Line 9: | Line 9: | ||
The DAX generator package consists of DAX creation classes: | The DAX generator package consists of DAX creation classes: | ||
| + | *CyberShake_Integrated_DAXGen.java: Entry point for generating a full ('integrated') workflow. | ||
| + | *CyberShake_SGT_DAXGen.java: Generates an SGT-only workflow, by calling PreCVM and then CyberShake_AWP_SGT_DAXGen if running AWP-ODC-SGT. | ||
*CyberShake_AWP_SGT_DAXGen.java: Creates workflow with all SGT jobs except PreCVM. This is because, when running AWP-ODC-SGT in CPU mode, we need to figure out the box dimensions in PreCVM before we can determine the number of processors we'll need in each dimension, so after PreCVM an AWP subworkflow is generated. | *CyberShake_AWP_SGT_DAXGen.java: Creates workflow with all SGT jobs except PreCVM. This is because, when running AWP-ODC-SGT in CPU mode, we need to figure out the box dimensions in PreCVM before we can determine the number of processors we'll need in each dimension, so after PreCVM an AWP subworkflow is generated. | ||
| − | |||
| − | |||
*CyberShake_PP_DAXGen.java: Entry point for generating a post-processing workflow. Is also called as part of creating an integrated workflow. | *CyberShake_PP_DAXGen.java: Entry point for generating a post-processing workflow. Is also called as part of creating an integrated workflow. | ||
| − | |||
*CyberShake_Stochastic_DAXGen.java: Entry point for generating a stochastic-only post-processing workflow. | *CyberShake_Stochastic_DAXGen.java: Entry point for generating a stochastic-only post-processing workflow. | ||
*CyberShake_Sub_Stoch_DAXGen.java: Most of the guts of generating a stochastic-only post-processing workflow. Since we need velocity model information as input to some of the stochastic codes, we generate the velocity model information in CyberShake_Stochastic_DAXGen, then pass the file to this class as an input at creation time. That way we can put the velocity in as a command-line parameter, rather than having to have every task read the same velocity file. | *CyberShake_Sub_Stoch_DAXGen.java: Most of the guts of generating a stochastic-only post-processing workflow. Since we need velocity model information as input to some of the stochastic codes, we generate the velocity model information in CyberShake_Stochastic_DAXGen, then pass the file to this class as an input at creation time. That way we can put the velocity in as a command-line parameter, rather than having to have every task read the same velocity file. | ||
| + | *CyberShake_DB_DAXGen.java: Generates a sub-workflow with the database and data product jobs. | ||
and helper classes: | and helper classes: | ||
| Line 27: | Line 27: | ||
*Stochastic_DAXParameters.java: Container class to track all science and technical parameters for a stochastic workflow which impact either which codes are called, or what to pass in the command-line to tasks. | *Stochastic_DAXParameters.java: Container class to track all science and technical parameters for a stochastic workflow which impact either which codes are called, or what to pass in the command-line to tasks. | ||
| + | Below are some diagrams to help explain the relationship between these classes (original file here: ). | ||
| + | |||
| + | {| | ||
| + | | [[File:SGT workflow classes.png|thumb|600px|SGT workflow classes diagram]] | ||
| + | | [[File:PP workflow classes.png|thumb|600px|Post-processing workflow classes diagram]] | ||
| + | |- | ||
| + | | [[File:Stoch workflow classes.png|thumb|600px|Stochastic workflow classes diagram]] | ||
| + | | [[File:Full workflow classes.png|thumb|600px|Full workflow classes diagram]] | ||
| + | |} | ||
== SGT Generation == | == SGT Generation == | ||
Revision as of 18:28, 28 February 2018
This page provides an overview of the CyberShake DAX generator, which includes all the Java classes associated with producing DAXes for any kind of CyberShake workflow.
The source for the DAX generator is available at http://source.usc.edu/svn/cybershake/import/trunk/dax-generator-3/ . All required dependencies are in the lib directory, though the Pegasus JAR may need to be updated if new Pegasus API features are used.
The DAX Generator is fairly complex, as we have retained backwards compatibility, even though some of the options may not have been used in several years.
Java classes
The DAX generator package consists of DAX creation classes:
- CyberShake_Integrated_DAXGen.java: Entry point for generating a full ('integrated') workflow.
- CyberShake_SGT_DAXGen.java: Generates an SGT-only workflow, by calling PreCVM and then CyberShake_AWP_SGT_DAXGen if running AWP-ODC-SGT.
- CyberShake_AWP_SGT_DAXGen.java: Creates workflow with all SGT jobs except PreCVM. This is because, when running AWP-ODC-SGT in CPU mode, we need to figure out the box dimensions in PreCVM before we can determine the number of processors we'll need in each dimension, so after PreCVM an AWP subworkflow is generated.
- CyberShake_PP_DAXGen.java: Entry point for generating a post-processing workflow. Is also called as part of creating an integrated workflow.
- CyberShake_Stochastic_DAXGen.java: Entry point for generating a stochastic-only post-processing workflow.
- CyberShake_Sub_Stoch_DAXGen.java: Most of the guts of generating a stochastic-only post-processing workflow. Since we need velocity model information as input to some of the stochastic codes, we generate the velocity model information in CyberShake_Stochastic_DAXGen, then pass the file to this class as an input at creation time. That way we can put the velocity in as a command-line parameter, rather than having to have every task read the same velocity file.
- CyberShake_DB_DAXGen.java: Generates a sub-workflow with the database and data product jobs.
and helper classes:
- CyberShake_Workflow_Container.java: This was designed for when we had multiple post-processing workflows per run. It's still used to combine a pre PP subworkflow with a PP subworkflow, a DB subworkflow, and a post subworkflow.
- DBConnect.java: Contains code for accessing databases.
- PP_DAXParameters.java: Container class to track all science and technical parameters for a post-processing workflow which impact either which codes are called, or what to pass in the command-line to tasks.
- RunIDQuery.java: Container class to track all the database parameters for a run.
- RuptureVariationDB.java: Supports writing an SQLite database to track mapping of which events go with which subworkflow. Since we've moved to a single subworkflow with a single DirectSynth job, we don't use this.
- SGT_DAXParameters.java: Container class to track all science and technical parameters for an SGT workflow which impact either which codes are called, or what to pass in the command-line to tasks.
- Stochastic_DAXParameters.java: Container class to track all science and technical parameters for a stochastic workflow which impact either which codes are called, or what to pass in the command-line to tasks.
Below are some diagrams to help explain the relationship between these classes (original file here: ).
SGT Generation
Rough call graph:
CyberShake_SGT_DAXGen.main()
CyberShake_SGT_DAXGen.subMain()
CyberShake_SGT_DAXGen.parseCommandLine(): parses command-line arguments.
