BBP Multi-Segment Ruptures - Revision history

Fsilva: /* Approach 1: Running Individual Segments Separately */

2016-09-27T18:40:37Z

Approach 1: Running Individual Segments Separately

Fsilva: /* Approach 2: Combining Segments Into Single Simulation */

2016-09-27T18:38:18Z

Approach 2: Combining Segments Into Single Simulation

Fsilva: /* Approach 1: Running Individual Segments on the Cluster Separately */

2016-09-27T18:37:05Z

Approach 1: Running Individual Segments on the Cluster Separately

Fsilva: /* Approach 2: Running Individual Segments and Combining Results on the Cluster */

2016-09-27T18:34:12Z

Approach 2: Running Individual Segments and Combining Results on the Cluster

Fsilva: /* Approach 2: Running Individual Segments and Combining Results on the Cluster */

2016-09-27T18:22:53Z

Approach 2: Running Individual Segments and Combining Results on the Cluster

Fsilva: /* Running BBP on the Cluster */

2016-09-27T18:22:29Z

Running BBP on the Cluster

Fsilva at 18:22, 27 September 2016

2016-09-27T18:22:06Z

Fsilva: /* Running the BBP for Multi-Segment Ruptures */

2016-09-27T16:41:10Z

Running the BBP for Multi-Segment Ruptures

Fsilva at 18:58, 26 September 2016

2016-09-26T18:58:14Z

Fsilva: /* SRC Description */

2016-09-26T17:56:54Z

SRC Description

← Older revision		Revision as of 18:40, 27 September 2016
Line 118:		Line 118:

	[[File:Multi-segment-single.png\|400px]]		[[File:Multi-segment-single.png\|400px]]
		+
		+	Pros/Cons:
		+
		+	* Working today
		+	* Results for each segment available (but requires extra storage)
		+	* Can be reprocessed with different merging parameters (timing)
		+	* Requires an extra wrapper around the platform to run each segment separately and then combine results

	==== Approach 2: Combining Segments Into Single Simulation ====		==== Approach 2: Combining Segments Into Single Simulation ====

← Older revision		Revision as of 18:38, 27 September 2016
Line 124:		Line 124:

	[[File:Multi-segment-single-SRFV2.png\|400px]]		[[File:Multi-segment-single-SRFV2.png\|400px]]
		+
		+	Pros/Cons:
		+
		+	* Not suitable for all methods
		+	* Faster processing as individual segments are not calculated separately
		+	* Results for individual segments not available

	=== Running BBP on the Cluster ===		=== Running BBP on the Cluster ===

@@ Line 129: / Line 129: @@
 ==== Approach 1: Running Individual Segments on the Cluster Separately ====
-When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:
+When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked R times (once for each realization), combining the data for all N segments into a final simulation. The processing diagram is shown below:
 [[File:Multi-segment-cluster-separate.png|400px]]
-In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.
+Pros/Cons:
 ==== Approach 2: Running Individual Segments and Combining Results on the Cluster ====

@@ Line 141: / Line 141: @@
 [[File:Multi-segment-cluster-together.png|350px]]
-This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).
+Pros/Cons:
 == SRF Plots ==

@@ Line 139: / Line 139: @@
 A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.
-[[File:Multi-segment-cluster-together.png|400px]]
+[[File:Multi-segment-cluster-together.png|350px]]
 This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).

@@ Line 107: / Line 107: @@
 == Running the BBP for Multi-Segment Ruptures ==
-==== Running Individual Simulations ====
+=== Running BBP on a Single Computer ===
 At this moment, the validation packages can specify multiple SRC files. For examples:
@@ Line 113: / Line 115: @@
   P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src
-Currently, when the Platform finds a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user then runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:
+Currently, when the Platform finds a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:
 [[File:Multi-segment-single.png|400px]]
 It may also be possible for some methods to generate a single, combined rupture SRF file that can be used by the wave propagation codes directly. In this case, a single simulation is done and the post processing steps are executed only once. The workflow for this method is shown below:
-[[File:Multi-segment-single-SRFV2.png|150px]]
+[[File:Multi-segment-single-SRFV2.png|400px]]
-==== Running Multiple Realizations on the Cluster ====
+==== Running Individual Segments on the Cluster Separately ====
-When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. This process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:
+When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:
 [[File:Multi-segment-cluster-separate.png|400px]]
 In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.
 A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.

← Older revision		Revision as of 18:58, 26 September 2016
Line 104:		Line 104:

	For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.		For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.
		+
		+	== Running the BBP for Multi-Segment Ruptures ==
		+
		+	==== Running Individual Simulations ====
		+
		+	At this moment, the validation packages can specify multiple SRC files. For examples:
		+
		+	P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src
		+
		+	When the Platform encounters a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user then runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:
		+
		+	[[File:Multi-segment-single.png\|400px]]
		+
		+	==== Running Multiple Realizations on the Cluster ====
		+
		+	When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. This process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:
		+
		+	[[File:Multi-segment-cluster-separate.png\|400px]]
		+
		+	In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.
		+
		+	A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.
		+
		+	[[File:Multi-segment-cluster-together.png\|400px]]
		+
		+	This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).

	== SRF Plots ==		== SRF Plots ==

@@ Line 70: / Line 70: @@
   RUPTURE_DELAY = 1.00
   MOMENT_FRACTION = 0.26
 * Segment 3
@@ Line 90: / Line 91: @@
   RUPTURE_DELAY = 2.00
   MOMENT_FRACTION = 0.36
 As shown above, the magnitude value remains the same through all segments, but there are additional parameters added to the standard SRC file. The moment-scaling and
 time delay information is included in the rupture description so that the final summation does not need to know about any of this. Although this approach is currently GP method specific, the idea is that other models will hopefully be able to use the same (or similar) approach.
-There are 3 additional parameters in each SRC file:
+There are 4 additional parameters in each SRC file:
 * MAX_FAULT_LENGTH = <float>
 * RUPTURE_DELAY = <float>
 * MOMENT_FRACTION = <float>
-For now these parameters will need to be supplied by hand by an "expert".  Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things.
+For now these parameters will need to be supplied by hand by an "expert".  Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.
 == SRF Plots ==