BBP Multi-Segment Ruptures - Revision history

Fsilva: /* Approach 1: Running Individual Segments Separately */

2016-09-27T18:40:37Z

Approach 1: Running Individual Segments Separately

← Older revision		Revision as of 18:40, 27 September 2016
Line 118:		Line 118:

	[[File:Multi-segment-single.png\|400px]]		[[File:Multi-segment-single.png\|400px]]

			Pros/Cons:

			* Working today
			* Results for each segment available (but requires extra storage)
			* Can be reprocessed with different merging parameters (timing)
			* Requires an extra wrapper around the platform to run each segment separately and then combine results

	==== Approach 2: Combining Segments Into Single Simulation ====		==== Approach 2: Combining Segments Into Single Simulation ====

Fsilva: /* Approach 2: Combining Segments Into Single Simulation */

2016-09-27T18:38:18Z

Approach 2: Combining Segments Into Single Simulation

← Older revision		Revision as of 18:38, 27 September 2016
Line 124:		Line 124:

	[[File:Multi-segment-single-SRFV2.png\|400px]]		[[File:Multi-segment-single-SRFV2.png\|400px]]

			Pros/Cons:

			* Not suitable for all methods
			* Faster processing as individual segments are not calculated separately
			* Results for individual segments not available

	=== Running BBP on the Cluster ===		=== Running BBP on the Cluster ===

Fsilva: /* Approach 1: Running Individual Segments on the Cluster Separately */

2016-09-27T18:37:05Z

Approach 1: Running Individual Segments on the Cluster Separately

← Older revision		Revision as of 18:37, 27 September 2016
Line 129:		Line 129:
	==== Approach 1: Running Individual Segments on the Cluster Separately ====		==== Approach 1: Running Individual Segments on the Cluster Separately ====

	When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked ~~one time~~ for each realization, combining the data for all N segments into a final simulation~~. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node~~. The processing diagram is shown below:		When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked R times (once for each realization), combining the data for all N segments into a final simulation. The processing diagram is shown below:

	[[File:Multi-segment-cluster-separate.png\|400px]]		[[File:Multi-segment-cluster-separate.png\|400px]]

	~~In this approach, the separate~~ simulations ~~(time series, plots) are~~ stored separately ~~in the user directory and~~ can be reprocessed ~~later~~ using a different ~~approach/~~parameters if needed.		Pros/Cons:

			* Individual segment simulations stored separately
			** Extra storage required
			** Results can be reprocessed using different timing parameters if needed
			* Time consuming as merging is done outside cluster

	==== Approach 2: Running Individual Segments and Combining Results on the Cluster ====		==== Approach 2: Running Individual Segments and Combining Results on the Cluster ====

Fsilva: /* Approach 2: Running Individual Segments and Combining Results on the Cluster */

2016-09-27T18:34:12Z

Approach 2: Running Individual Segments and Combining Results on the Cluster

← Older revision		Revision as of 18:34, 27 September 2016
Line 141:		Line 141:
	[[File:Multi-segment-cluster-together.png\|350px]]		[[File:Multi-segment-cluster-together.png\|350px]]

	~~This approach is much faster and simpler~~ to manage (fewer jobs to submit to the cluster ~~and no need~~ to ~~run~~ the ~~post-processing computations separately for each realization~~) ~~but it does~~ not save ~~the separate computations~~ (~~and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster~~).		Pros/Cons:

			* Simpler to manage (fewer jobs to submit to the cluster)
			* Faster to execute (everything runs on the cluster)
			* Does not save intermediate results (individual segments' time series/plots)
			* Less disk space required

	== SRF Plots ==		== SRF Plots ==

Fsilva: /* Approach 2: Running Individual Segments and Combining Results on the Cluster */

2016-09-27T18:22:53Z

Approach 2: Running Individual Segments and Combining Results on the Cluster

← Older revision		Revision as of 18:22, 27 September 2016
Line 139:		Line 139:
	A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.		A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.

	[[File:Multi-segment-cluster-together.png\|~~400px~~]]		[[File:Multi-segment-cluster-together.png\|350px]]

	This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).		This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).

Fsilva: /* Running BBP on the Cluster */

2016-09-27T18:22:29Z

Running BBP on the Cluster

← Older revision		Revision as of 18:22, 27 September 2016
Line 127:		Line 127:
	=== Running BBP on the Cluster ===		=== Running BBP on the Cluster ===

	==== Running Individual Segments on the Cluster Separately ====		==== Approach 1: Running Individual Segments on the Cluster Separately ====

	When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:		When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:
Line 135:		Line 135:
	In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.		In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.

	==== Running Individual Segments and Combining Results on the Cluster ====		==== Approach 2: Running Individual Segments and Combining Results on the Cluster ====

	A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.		A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.

Fsilva at 18:22, 27 September 2016

2016-09-27T18:22:06Z

← Older revision		Revision as of 18:22, 27 September 2016
Line 107:		Line 107:
	== Running the BBP for Multi-Segment Ruptures ==		== Running the BBP for Multi-Segment Ruptures ==

	==== Running Individual ~~Simulations~~ ====		=== Running BBP on a Single Computer ===

			==== Approach 1: Running Individual Segments Separately ====

	At this moment, the validation packages can specify multiple SRC files. For examples:		At this moment, the validation packages can specify multiple SRC files. For examples:
Line 113:		Line 115:
	P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src		P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src

	Currently, when the Platform finds a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user ~~then~~ runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:		Currently, when the Platform finds a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:

	[[File:Multi-segment-single.png\|400px]]		[[File:Multi-segment-single.png\|400px]]

			==== Approach 2: Combining Segments Into Single Simulation ====

	It may also be possible for some methods to generate a single, combined rupture SRF file that can be used by the wave propagation codes directly. In this case, a single simulation is done and the post processing steps are executed only once. The workflow for this method is shown below:		It may also be possible for some methods to generate a single, combined rupture SRF file that can be used by the wave propagation codes directly. In this case, a single simulation is done and the post processing steps are executed only once. The workflow for this method is shown below:

	[[File:Multi-segment-single-SRFV2.png\|~~150px~~]]		[[File:Multi-segment-single-SRFV2.png\|400px]]

			=== Running BBP on the Cluster ===

	==== Running ~~Multiple Realizations~~ on the Cluster ====		==== Running Individual Segments on the Cluster Separately ====

	When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. ~~This~~ process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:		When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. The merging process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:

	[[File:Multi-segment-cluster-separate.png\|400px]]		[[File:Multi-segment-cluster-separate.png\|400px]]

	In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.		In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.

			==== Running Individual Segments and Combining Results on the Cluster ====

	A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.		A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.

Fsilva: /* Running the BBP for Multi-Segment Ruptures */

2016-09-27T16:41:10Z

Running the BBP for Multi-Segment Ruptures

← Older revision		Revision as of 16:41, 27 September 2016
Line 113:		Line 113:
	P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src		P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src

	~~When~~ the Platform ~~encounters~~ a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user then runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:		Currently, when the Platform finds a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user then runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:

	[[File:Multi-segment-single.png\|400px]]		[[File:Multi-segment-single.png\|400px]]

			It may also be possible for some methods to generate a single, combined rupture SRF file that can be used by the wave propagation codes directly. In this case, a single simulation is done and the post processing steps are executed only once. The workflow for this method is shown below:

			[[File:Multi-segment-single-SRFV2.png\|150px]]

	==== Running Multiple Realizations on the Cluster ====		==== Running Multiple Realizations on the Cluster ====

Fsilva at 18:58, 26 September 2016

2016-09-26T18:58:14Z

← Older revision		Revision as of 18:58, 26 September 2016
Line 104:		Line 104:

	For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.		For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.

			== Running the BBP for Multi-Segment Ruptures ==

			==== Running Individual Simulations ====

			At this moment, the validation packages can specify multiple SRC files. For examples:

			P_CODEBASE_GP_source = gp/landers_v16_08_1_seg1.src, gp/landers_v16_08_1_seg2.src, gp/landers_v16_08_1_seg3.src

			When the Platform encounters a validation package like this, it will prompt the user to select which segment s/he wants to run. After all segments are completed (separately), the user then runs the "merge_multisegment_validation.py" to add the time series from each segment together into a new BBP simulation and then run the entire BBP post-processing codes to calculate the BBP outputs based on the combined seismograms. The simple processing workflow is shown below:

			[[File:Multi-segment-single.png\|400px]]

			==== Running Multiple Realizations on the Cluster ====

			When running multiple realizations on the cluster, simulations can still be run one segment at a time. In this case, the user submits N jobs to the cluster (one for each segment). The data products of each segment is saved and copied back to the user directory. Then, the merge_multisegment_validation tool is invoked one time for each realization, combining the data for all N segments into a final simulation. This process is done one realization at a time and can be time consuming as it is not running on the cluster but in a single node. The processing diagram is shown below:

			[[File:Multi-segment-cluster-separate.png\|400px]]

			In this approach, the separate simulations (time series, plots) are stored separately in the user directory and can be reprocessed later using a different approach/parameters if needed.

			A different approach is to calculate all segments on the cluster serially in a single job and then merge the simulations at each computation node. This allows for faster data processing as the computations are distributed across a large number of nodes (just like the simulations are). After post-processing is completed, the combined results are copied back to the user directory.

			[[File:Multi-segment-cluster-together.png\|400px]]

			This approach is much faster and simpler to manage (fewer jobs to submit to the cluster and no need to run the post-processing computations separately for each realization) but it does not save the separate computations (and if it did, further reprocessing would fall into the previous scenario where it had to be done outside of the cluster).

	== SRF Plots ==		== SRF Plots ==

Fsilva: /* SRC Description */

2016-09-26T17:56:54Z

SRC Description

← Older revision		Revision as of 17:56, 26 September 2016
Line 70:		Line 70:
	RUPTURE_DELAY = 1.00		RUPTURE_DELAY = 1.00
	MOMENT_FRACTION = 0.26		MOMENT_FRACTION = 0.26
			COMMON_SEED = 865558645

	* Segment 3		* Segment 3
Line 90:		Line 91:
	RUPTURE_DELAY = 2.00		RUPTURE_DELAY = 2.00
	MOMENT_FRACTION = 0.36		MOMENT_FRACTION = 0.36
			COMMON_SEED = 865558645

	As shown above, the magnitude value remains the same through all segments, but there are additional parameters added to the standard SRC file. The moment-scaling and		As shown above, the magnitude value remains the same through all segments, but there are additional parameters added to the standard SRC file. The moment-scaling and
	time delay information is included in the rupture description so that the final summation does not need to know about any of this. Although this approach is currently GP method specific, the idea is that other models will hopefully be able to use the same (or similar) approach.		time delay information is included in the rupture description so that the final summation does not need to know about any of this. Although this approach is currently GP method specific, the idea is that other models will hopefully be able to use the same (or similar) approach.

	There are 3 additional parameters in each SRC file:		There are 4 additional parameters in each SRC file:

	* MAX_FAULT_LENGTH = <float>		* MAX_FAULT_LENGTH = <float>
	* RUPTURE_DELAY = <float>		* RUPTURE_DELAY = <float>
	* MOMENT_FRACTION = <float>		* MOMENT_FRACTION = <float>
			* COMMON_SEED = <integer>

	For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things.		For now these parameters will need to be supplied by hand by an "expert". Once we make sure the computation process is all worked out, we can worry about how to automate/optimize things. The COMMON_SEED parameter is needed so that certain parameters can be calculated consistently across all segments. Right now segments number 2 and 3 have their COMMON_SEED set to segment's 1 SEED value. If this parameter is missing, the SEED value is used for all calculations where random numbers are needed.

	== SRF Plots ==		== SRF Plots ==