CyberShake Workflow Framework - Revision history

Scottcal: /* Typical invocation */

2018-10-04T18:39:53Z

Typical invocation

Scottcal: /* X509 certificates */

2018-10-04T18:23:49Z

X509 certificates

Scottcal: /* X509 certificates */

2018-10-04T17:52:36Z

X509 certificates

Scottcal: /* X509 certificates */

2018-10-04T17:49:32Z

X509 certificates

Scottcal: /* Typical invocation */

2018-10-04T17:38:45Z

Typical invocation

Scottcal at 17:35, 4 October 2018

2018-10-04T17:35:07Z

Scottcal: /* Run */

2018-10-04T17:26:23Z

Run

Scottcal: /* Integrated workflow */

2018-10-04T17:12:28Z

Integrated workflow

Scottcal: /* Run */

2018-10-03T21:15:32Z

Run

Scottcal: /* bluewaters properties file */

2018-05-09T18:40:28Z

bluewaters properties file

@@ Line 1,037: / Line 1,037: @@
 == Typical invocation ==
-Below are typical invocations for a few common use cases.
+Below are typical invocations for a few common use cases.  Note that if you're running using an RSQSim ERF, you should include -bbox in the SGT arguments.
 === SGT followed by PP workflow ===

@@ Line 1,069: / Line 1,069: @@
 # Run myproxy-logon -t 300:00, or some similar variant.
 # Transfer the generated certificate back to your submit host (shock.usc.edu)
-# If you are working with a new system, make sure that the correct CA certificates authorizing the entities signing the certificate are installed.  Typically these are installed in /etc/grid-security/certificates; you can usually override these with your own certificates in ~/.globus/certificates, particularly useful on systems for which we don't have write access to /etc/grid-security/certificates.  You want to make sure you have a .0 and .signing_policy for your certificates; the .crl_url and .r0 are more trouble than they're worth.
+# If you are working with a new system, make sure that the correct certificate authority (CA) certificates authorizing the entities signing the certificate are installed.  Typically these are installed in /etc/grid-security/certificates; you can usually override these with your own certificates in ~/.globus/certificates, particularly useful on systems for which we don't have write access to /etc/grid-security/certificates.  You want to make sure you have a .0 and .signing_policy for your certificates; the .crl_url and .r0 are more trouble than they're worth.
 Your proxy must be valid in order to be useful.  You can check validity by running the command:

@@ Line 1,069: / Line 1,069: @@
 # Run myproxy-logon -t 300:00, or some similar variant.
 # Transfer the generated certificate back to your submit host (shock.usc.edu)
-# Make sure that the correct CA certificates authorizing the entities signing the certificate are installed.  Typically these are installed in /etc/grid-security/certificates; you can usually override these with your own certificates in ~/.globus/certificates, particularly useful on systems for which we don't have write access to /etc/grid-security/certificates.
+# If you are working with a new system, make sure that the correct CA certificates authorizing the entities signing the certificate are installed.  Typically these are installed in /etc/grid-security/certificates; you can usually override these with your own certificates in ~/.globus/certificates, particularly useful on systems for which we don't have write access to /etc/grid-security/certificates.  You want to make sure you have a .0 and .signing_policy for your certificates; the .crl_url and .r0 are more trouble than they're worth.
 Your proxy must be valid in order to be useful.  You can check validity by running the command:

← Older revision		Revision as of 17:49, 4 October 2018
Line 1,062:		Line 1,062:

	== X509 certificates ==		== X509 certificates ==
		+
		+	When using GRAM, GridFTP, or Globus Online, X509 proxies are used to authenticate. The idea is that once a user logs into a resource, the user can obtain a 'proxy', which the user can present instead of a password or SSH key to authenticate themselves to a remote system. These proxies are generated from certificates; the key difference is that the proxies are only valid for a limited period of time, so even if they're compromised they do not lead to permanent access to the system. Since X509 certificates are built on trust chains, any system which authenticates using your X509 proxy must ultimately trust the entity which signed your proxy.
		+
		+	In practice, this usually means doing some variation of the following:
		+	# Log into the system you want to obtain a proxy for. You may need to log into a separate server to get a proxy (for example, on Titan, you must log into one of the data transfer nodes [dtn].)
		+	# Run myproxy-logon -t 300:00, or some similar variant.
		+	# Transfer the generated certificate back to your submit host (shock.usc.edu)
		+	# Make sure that the correct CA certificates authorizing the entities signing the certificate are installed. Typically these are installed in /etc/grid-security/certificates; you can usually override these with your own certificates in ~/.globus/certificates, particularly useful on systems for which we don't have write access to /etc/grid-security/certificates.
		+
		+	Your proxy must be valid in order to be useful. You can check validity by running the command:
		+	grid-proxy-info -f <proxy path>
		+
		+	Check the 'timeleft' field to make sure it is not 00:00:00. For CyberShake production runs, we have a cronjob which checks certificates every hour and emails Scott if any have <24 hours left. Note that on Titan the longest duration proxy they will issue is 72 hours. On Stampede2 and Blue Waters, it's about 11 days. If suddenly jobs stop working, or file transfers fail, checking certificate validity is a good place to start.

	== Pegasus-MPI-Cluster ==		== Pegasus-MPI-Cluster ==

← Older revision		Revision as of 17:38, 4 October 2018
Line 1,052:		Line 1,052:

	=== Integrated workflow ===		=== Integrated workflow ===
		+
		+	This is for a site with our currently most frequently used parameters (velocity model CVM-S5, ERF ID 36, rupture variation scenario ID 6 for uniform spacing, SGT ID 8 for AWP GPU, frequency 1.0 Hz, filter frequency 2.0 Hz, moment.usc.edu as the run server, separate MD5 sums so we don't exceed bin 5 queue times on Titan, DirectSynth for the post-processing, calculate durations, non-blocking MD5 sum checks in the post-processing, calculate RotD50). We specify the SGT and PP site, but it's not required.
		+
		+	<pre>
		+	./create_full_wf.sh -s <site> -v vsi -e 36 -r 6 -g 8 -f 1.000000 -q 2.000000 --server moment.usc.edu --sgtargs -ss <remote system> --ppargs -ps <remote system> -ds -du -nb -r
		+	./plan_full.sh <site> <run ID> <remote system>
		+	./run_full.sh <site> <run ID>
		+	</pre>

	== X509 certificates ==		== X509 certificates ==

← Older revision		Revision as of 17:35, 4 October 2018
Line 1,034:		Line 1,034:

	<b>Output files:</b> None.		<b>Output files:</b> None.
		+
		+	== Typical invocation ==
		+
		+	Below are typical invocations for a few common use cases.
		+
		+	=== SGT followed by PP workflow ===
		+
		+	This is for a site with our currently most frequently used parameters (velocity model CVM-S5, ERF ID 36, rupture variation scenario ID 6 for uniform spacing, SGT ID 8 for AWP GPU, frequency 1.0 Hz, filter frequency 2.0 Hz, moment.usc.edu as the run server, separate MD5 sums so we don't exceed bin 5 queue times on Titan, DirectSynth for the post-processing, calculate durations, non-blocking MD5 sum checks in the post-processing, calculate RotD50, use moment.usc.edu as the run server)
		+
		+	<pre>./create_sgt_dax.sh -s <site> -v vsi -e 36 -r 6 -g 8 -f 1.000000 -q 2.000000 --server moment.usc.edu --sgtargs -sm
		+	./plan_sgt.sh <site> <run ID> <remote system>
		+	./run_sgt.sh <site> <run ID>
		+	./create_pp_wf.sh -s <site> -v vsi -e 36 -r 6 -g 8 -f 1.0 -q 2.0 -ds -du -nb -r --server moment.usc.edu
		+	./plan_pp.sh <site> <run ID> <remote system> <user> <email>
		+	./run_pp.sh <site> <run_id>
		+	</pre>
		+
		+	=== Integrated workflow ===

	== X509 certificates ==		== X509 certificates ==

@@ Line 913: / Line 913: @@
 == Run ==
-The final step of workflow preparation is to run the workflow.  This takes the planned workflow, containing Condor DAG and submit scripts, and submits it to Condor to manage the real-time execution.  After running the workflow, jobs will appear in the Condor queue and can be monitored (see [[CyberShake Workflow Monitoring]] for details).
+The final step of workflow preparation is to run the workflow.  This takes the planned workflow, containing Condor DAG and submit scripts, and submits it to Condor to manage the real-time execution.  After running the workflow, jobs will appear in the Condor queue and can be monitored (see [[CyberShake Workflow Monitoring]] for details).  This can be either the initial submission of a workflow, or a restart if there has been a failure.
-=== X509 certificates ===
+=== SGT Workflow ===
 == Pegasus-MPI-Cluster ==

← Older revision		Revision as of 17:12, 4 October 2018
Line 882:		Line 882:

	==== Integrated workflow ====		==== Integrated workflow ====
		+
		+	<b>Purpose:</b> To plan a full CyberShake workflow.
		+
		+	<b>Detailed description:</b> plan_full.sh takes command-line arguments with the DAX to plan, runs pegasus-plan, and updates the record in the database to reflect that planning has occurred. The TC catalog is also recorded. Some changes are made to the properties files as well: for example, pegasus.exitcode.scope is set to none in the top-level properties file, but all in the subdax files, and a different user certificate is selected depending on the execution system.
		+
		+	<b>Needs to be changed if:</b>
		+	#Different dynamic configuration options need to be set.
		+
		+	<b>Source code location:</b> http://source.usc.edu/svn/cybershake/import/trunk/runs/plan_full.sh
		+
		+	<b>Author:</b> Scott Callaghan
		+
		+	<b>Dependencies:</b> [[CyberShake_Workflow_Framework#RunManager\|Run Manager]]
		+
		+	<b>Executable chain:</b>
		+	plan_full.sh
		+
		+	<b>Compile instructions:</b>None
		+
		+	<b>Usage:</b>
		+	<pre>
		+	Usage: ./plan_full.sh <integrated directory> \| <site> <run_id> <remote_site>
		+	Example: ./plan_full.sh USC 2494 stampede
		+	</pre>
		+
		+	<b>Input files:</b> None specified, but requires DAX files in directory.
		+
		+	<b>Output files:</b> DAG files.

	== Run ==		== Run ==

← Older revision		Revision as of 21:15, 3 October 2018
Line 884:		Line 884:

	== Run ==		== Run ==
		+
		+	The final step of workflow preparation is to run the workflow. This takes the planned workflow, containing Condor DAG and submit scripts, and submits it to Condor to manage the real-time execution. After running the workflow, jobs will appear in the Condor queue and can be monitored (see [[CyberShake Workflow Monitoring]] for details).

	=== X509 certificates ===		=== X509 certificates ===

@@ Line 776: / Line 776: @@
 ==== bluewaters properties file ====
 *X509 proxy (Scott's on Blue Waters), how the cache file is used with the RC, other sites
-pegasus.local.env=X509_USER_PROXY=/tmp/x509up_u33527
+ pegasus.local.env=X509_USER_PROXY=/tmp/x509up_u33527
-pegasus.catalog.replica.cache.asrc=true
+ pegasus.catalog.replica.cache.asrc=true
-pegasus.transfer.inter.thirdparty.sites = hpc,mercury,shock,sdsc,abe,ranger,stampede,bluewaters
+ pegasus.transfer.inter.thirdparty.sites = hpc,mercury,shock,sdsc,abe,ranger,stampede,bluewaters
 <b>Needs to be changed if:</b>