CVM-T User Guide - Revision history

Patrices at 17:55, 3 November 2010

2010-11-03T17:55:44Z

← Older revision		Revision as of 17:55, 3 November 2010
Line 3:		Line 3:
	=== Overview ===		=== Overview ===

	~~TBD~~		The SCEC CVM-Toolkit (CVM-T) is a software system that provides one or more community velocity models (CVMs), a mesh extraction tool for generated 3D velocity meshes from these CVMs in a variety of formats, a testing framework for evaluating the performance of CVMs, and visualization tools for plotting slices from either a CVM or a mesh. Each of these components is largely independent of the others (an exception being cvm2mesh which has dependencies on CVM-H), so the user is able to choose which components he wishes to use. The following sections of this user guide describe how to install and operate each tool.


Line 353:		Line 353:
	=== Installing a New CVM ===		=== Installing a New CVM ===

	~~TBD~~		Installing a new CVM into the test framework is somewhat involved. The general steps needed to perform this are:
			* Install copy of CVM on the local host. If using CruiseControl, update CC to build the CVM and run the unit tests
			* Edit ${CVMTEST_HOME}/remote/AcceptTest.py on the local host to change the default CVM used in the testing
			* Edit cvm2mesh to support the new CVM
			* Edit ${CVMTEST_HOME}/config/python/ATFConfig.py on the remote host to add the new CVM tag and properties
			* Create PBS job submit scripts in ${CVMTEST_HOME}/pbs/kraken for building the CVM, cvm2mesh, mesh extraction, AWP sim, GoF calculation, and plotting

Patrices at 01:12, 3 November 2010

2010-11-03T01:12:41Z

← Older revision		Revision as of 01:12, 3 November 2010
Line 330:		Line 330:
	* Create the following new directories:		* Create the following new directories:
	${CVMTEST_HOME}/awp/${NEW_EVENT_TAG}/${CVM} (one for each CVM defined in ATFConfig.py)		${CVMTEST_HOME}/awp/${NEW_EVENT_TAG}/${CVM} (one for each CVM defined in ATFConfig.py)
	${~~CVMTESET_HOME~~}/config/topo/${NEW_EVENT_TAG}		${CVMTEST_HOME}/config/topo/${NEW_EVENT_TAG}
	${~~CVMTESET_HOME~~}/config/cvm2mesh/${NEW_EVENT_TAG}		${CVMTEST_HOME}/config/cvm2mesh/${NEW_EVENT_TAG}
	${~~CVMTESET_HOME~~}/config/stations/${NEW_EVENT_TAG}		${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}

	where ${NEW_EVENT_TAG} is the new event's tag		where ${NEW_EVENT_TAG} is the new event's tag

Patrices at 00:54, 3 November 2010

2010-11-03T00:54:25Z

← Older revision		Revision as of 00:54, 3 November 2010
Line 344:		Line 344:
	% ./ExtractDEM.py ${NEW_EVENT_TAG}		% ./ExtractDEM.py ${NEW_EVENT_TAG}
	* Retrieve event hypocenter from SCEC DC and save in ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/${NEW_EVENT_TAG}_event.txt		* Retrieve event hypocenter from SCEC DC and save in ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/${NEW_EVENT_TAG}_event.txt
	* Retrieve complete CISN station list from SCEC DC (or copy existing list from ${CVMTEST_HOME}/config/stations/chino_uf/cisn_stations.txt) to ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/cisn_stations.txt		* Retrieve complete CISN station list from SCEC DC http://www.data.scec.org/stationinfo.html (or copy existing list from ${CVMTEST_HOME}/config/stations/chino_uf/cisn_stations.txt) to ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/cisn_stations.txt
	* Define list of cities within the meshing region and save to ${CVMTEST_HOME}/config/station/${NEW_EVENT_TAG}/socal_cities.txt		* Define list of cities within the meshing region and save to ${CVMTEST_HOME}/config/station/${NEW_EVENT_TAG}/socal_cities.txt
	* Generate a new event station list for ${NEW_EVENT_TAG} with the CISN2XY.py script:		* Generate a new event station list for ${NEW_EVENT_TAG} with the CISN2XY.py script:

Patrices at 00:17, 3 November 2010

2010-11-03T00:17:08Z

← Older revision		Revision as of 00:17, 3 November 2010
Line 14:		Line 14:
	*SCEC Broadband Platform https://source.usc.edu/svn/broadband		*SCEC Broadband Platform https://source.usc.edu/svn/broadband
	*AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/		*AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/
	*Python 2.6+ with numpy, matplotlib, and ~~basemap~~ modules on local/remote hosts		*Python 2.6+ with numpy, matplotlib, basemap, and pyproj modules on local/remote hosts
	*USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php		*USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php
	*CruiseControl		*CruiseControl
Line 230:		Line 230:
	* SCEC Broadband Platform https://source.usc.edu/svn/broadband on remote host		* SCEC Broadband Platform https://source.usc.edu/svn/broadband on remote host
	* AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/ on remote host		* AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/ on remote host
	* Python 2.6+ with numpy, matplotlib, and ~~basemap~~ modules on local/remote hosts		* Python 2.6+ with numpy, matplotlib, basemap, and pyproj modules on local/remote hosts
	* USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php on remote host		* USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php on remote host
	* CruiseControl [Optional] on local host		* CruiseControl [Optional] on local host
Line 335:		Line 335:

	where ${NEW_EVENT_TAG} is the new event's tag		where ${NEW_EVENT_TAG} is the new event's tag
			* Download and unpack DEMs covering the entire meshing region for the new scenario into ${CVMTEST_HOME}/dem. Download 1/3 arcsec in GridFloat format. The USGS seamless server places limits on how much elevation data you may request, therefore limit requests to boxes of 1 deg lat/lon.
			* Add the new DEM directories to the list of DEMs searched by ${CVMTEST_HOME}/topo/ExtractDEM.py. For example, add appropriate lines such as the following in main():

	* Download and install DEMs covering the entire meshing region for the new scenario into ${CVMTEST_HOME}/dem. ~~Download 1/3 arcsec in GridFloat format~~. ~~The USGS seamless server places limits on how much elevation data you may request, therefore limit requests to boxes of 1 deg lat~~/~~lon~~.		demlist.append('../../dem/NED_28288029/ned_28288029')
	* Add the new DEM directories to the list of DEMs searched by ExtractDEM.~~py. For example, add appropriate lines such as the following in main(~~):

	~~demlist~~.~~append~~('../../~~dem~~/~~NED_28288029~~/~~ned_28288029')~~		* Generate a topography file for the event with ${CVMTEST_HOME}/tools/topo/ExtractDEM.py:
			% cd ${CVMTEST_HOME}/tools/topo
			% ./ExtractDEM.py ${NEW_EVENT_TAG}
			* Retrieve event hypocenter from SCEC DC and save in ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/${NEW_EVENT_TAG}_event.txt
			* Retrieve complete CISN station list from SCEC DC (or copy existing list from ${CVMTEST_HOME}/config/stations/chino_uf/cisn_stations.txt) to ${CVMTEST_HOME}/config/stations/${NEW_EVENT_TAG}/cisn_stations.txt
			* Define list of cities within the meshing region and save to ${CVMTEST_HOME}/config/station/${NEW_EVENT_TAG}/socal_cities.txt
			* Generate a new event station list for ${NEW_EVENT_TAG} with the CISN2XY.py script:
			% cd ${CVMTEST_HOME}/tools/stations
			% ./CISN2XY.py ${NEW_EVENT_TAG}

	* Generate a topography file for the event with ${CVMTEST_HOME}/topo/ExtractDEM.py
	* Generate a station list (TBD)

	=== Installing a New CVM ===		=== Installing a New CVM ===

Patrices at 22:55, 2 November 2010

2010-11-02T22:55:52Z

← Older revision		Revision as of 22:55, 2 November 2010
Line 1:		Line 1:
			== CVM-T ==

			=== Overview ===

			TBD


			=== Requirements/Dependencies ===

			The CVM-T system has many dependencies on third-party software packages. Each CVM-T component has its own specific requirements which are listed in that component's user guide. This document does not describe in detail how to install these dependencies, as many are complex enough to warrant their own guide. Instead, this document refers the reader to the URL where the software distribution and documentation can be found. In general, the CVM-T system relies on these packages:

			*SCEC CVM-H https://source.usc.edu/svn/cvmh
			*SCEC CVM-4 http://earth.usc.edu/~gely/coseis/www/index.html
			*SCEC Broadband Platform https://source.usc.edu/svn/broadband
			*AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/
			*Python 2.6+ with numpy, matplotlib, and basemap modules on local/remote hosts
			*USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php
			*CruiseControl
			*Globus Toolkit (gsissh, GridFTP)


	== CVM-H ==		== CVM-H ==

Line 224:		Line 245:
	==== Local Install ====		==== Local Install ====

	*Check out cvmtest distribution from https://source.usc.edu/svn/cvmtest/trunk or unpack the pre-packaged tar file.		*Check out cvmtest distribution from https://source.usc.edu/svn/cvmtest/trunk or unpack the pre-packaged tar file into ${CVMTEST_HOME}, a directory of your choosing.
			*Check out a local copy of CVM-H from https://source.usc.edu/svn/cvmtest into ${CVMTEST_HOME}/cvmh
	*If cvmtest will be run under the auspices of a CruiseControl installation:		*If cvmtest will be run under the auspices of a CruiseControl installation:
	**~~Check out a local copy of CVM-H from https://source.usc.edu/svn/cvmtest and configure~~ CC to execute the update, build, and unit test targets in the ./trunk/build.xml ANT file according to your software configuration management policies.		**Configure CC to execute the update, build, and unit test targets in the ${CVMTEST_HOME}/cvmh/trunk/build.xml ANT file according to your software configuration management policies.
	**Configure CC to invoke the GoF acceptance tests,./remote/AcceptTest.py, at the desired date and time.		**Configure CC to invoke the GoF acceptance tests, ${CVMTEST_HOME}/remote/AcceptTest.py, at the desired date and time.
	*~~Create a TeraGrid grid proxy using your user certificate~~		**Ensure the owner of the CC processes and the owner of the cvmtest/cvmh checkouts are the same userid.


Line 235:		Line 257:
	Currently, cvmtest is configured to run GoF acceptance tests on NICS Kraken. The following steps outline the setup required on Kraken.		Currently, cvmtest is configured to run GoF acceptance tests on NICS Kraken. The following steps outline the setup required on Kraken.

	*Check out cvmtest distribution from https://source.usc.edu/svn/cvmtest/trunk or unpack the pre-packaged tar file.		*Check out cvmtest distribution from https://source.usc.edu/svn/cvmtest/trunk or unpack the pre-packaged tar file into ${CVMTEST_HOME}, a directory of your choosing on the /lustre scratch filesystem.

	*Customize ${CVMTEST_HOME}/config/python/ATFConfig.py for system specific paths. The installation base directory needs to be configured as well as the location of the SCEC Broadband GoF codes:		*Customize ${CVMTEST_HOME}/config/python/ATFConfig.py for system specific paths. The installation base directory needs to be configured as well as the location of the SCEC Broadband GoF codes:

	INSTALL_BASE_PATH='/lustre/scratch/patricks/build'		INSTALL_BASE_PATH='/lustre/scratch/patricks/build'
	GOF_PATH = '/lustre/scratch/patricks/broadband/src/urs/bin'		GOF_PATH = '/lustre/scratch/patricks/broadband/src/urs/bin'

			* Copy AWP-ODC executable pmcl3d in all ${CVMTEST_HOME}/awp/${EVENT}/${CVM} directories, where EVENT and CVM are all event and CVM combinations defined in ${CVMTEST_HOME}/config/python/ATFConfig.py


Line 246:		Line 271:
	==== Running from CruiseControl ====		==== Running from CruiseControl ====

	* Configure CC to execute ${CVMTEST_HOME}/remote/AcceptTest.py at the desired date and time		*Create a valid Teragrid x.509 proxy certificate for the owner of cvmtest/cvmh/CC. Note that the owner of the grid user certificate does not need to be the same as the owner of cvmtest/cvmh/CC. In other words, cvmtest can piggy-back on a different userid's certificate.
			*Configure CC to execute ${CVMTEST_HOME}/remote/AcceptTest.py at the desired date and time

			The test results and plots in ${CVMTEST_HOME}/web_results.tar may then be transferred to a webhost and unpacked for public release.


	==== Running manually from Local Host ====		==== Running manually from Local Host ====

			*Create a valid Teragrid x.509 proxy certificate for the owner of cvmtest/cvmh/CC. Note that the owner of the grid user certificate does not need to be the same as the owner of cvmtest/cvmh/CC. In other words, cvmtest can piggy-back on a different userid's certificate.
	* Execute the following script from the shell prompt:		* Execute the following script from the shell prompt:

	% cd ${CVMTEST_HOME}/remote		% cd ${CVMTEST_HOME}/remote
	% ./AcceptTest.py		% ./AcceptTest.py

			The test results and plots in ${CVMTEST_HOME}/web_results.tar may then be transferred to a webhost and unpacked for public release.


	==== Running manually on NICS Kraken ====		==== Running manually on NICS Kraken ====

	Since the workflow management provided in AcceptTest.py is not being used, the user ~~most~~ execute these steps sequentially ~~and~~ monitor the job progress himself.		Since the workflow management provided in AcceptTest.py is not being used, the user must login to Kraken and execute these steps sequentially while monitor the job progress himself.

	* Build CVM-H		* Build CVM-H
Line 296:		Line 327:
	=== Installing a New Earthquake Scenario ===		=== Installing a New Earthquake Scenario ===

	~~TBD~~		* Edit ${CVMTEST_HOME}/config/python/ATFConfig.py to add your new event, event tag, event description, and mesh dimensions.
			* Create the following new directories:
			${CVMTEST_HOME}/awp/${NEW_EVENT_TAG}/${CVM} (one for each CVM defined in ATFConfig.py)
			${CVMTESET_HOME}/config/topo/${NEW_EVENT_TAG}
			${CVMTESET_HOME}/config/cvm2mesh/${NEW_EVENT_TAG}
			${CVMTESET_HOME}/config/stations/${NEW_EVENT_TAG}

			where ${NEW_EVENT_TAG} is the new event's tag

			* Download and install DEMs covering the entire meshing region for the new scenario into ${CVMTEST_HOME}/dem. Download 1/3 arcsec in GridFloat format. The USGS seamless server places limits on how much elevation data you may request, therefore limit requests to boxes of 1 deg lat/lon.
			* Add the new DEM directories to the list of DEMs searched by ExtractDEM.py. For example, add appropriate lines such as the following in main():

			demlist.append('../../dem/NED_28288029/ned_28288029')

			* Generate a topography file for the event with ${CVMTEST_HOME}/topo/ExtractDEM.py
			* Generate a station list (TBD)

	=== Installing a New CVM ===		=== Installing a New CVM ===

Patrices at 20:19, 2 November 2010

2010-11-02T20:19:00Z

@@ Line 207: / Line 207: @@
 === Requirements ===
-* SCEC Broadband Platform https://source.usc.edu/svn/broadband
+* SCEC Broadband Platform https://source.usc.edu/svn/broadband on remote host
-* AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/
+* AWP-ODC Finite Difference Codes http://hpgeoc.sdsc.edu/AWPODC/ on remote host
-* Python 2.6+ with numpy, matplotlib, and basemap modules
+* Python 2.6+ with numpy, matplotlib, and basemap modules on local/remote hosts
-* USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php
+* USGS DEMs [Optional] from the USGS National Map Seamless Server http://seamless.usgs.gov/index.php on remote host
 === Installation ===
-* Customize ./config/python/ATFConfig.py for system specific paths. Specifically, the installation base directory needs to be configured as well as the location of the SCEC Broadband GoF codes:
+Installation of the automated test framework has two components, an install on a local machine (from which you will launch the GoF acceptance tests) and an install on a remote grid resource (such as NICS Kraken, where the tests will execute). If you wish to have the tests managed by CruiseControl, your local machine must have the CC package installed.
   INSTALL_BASE_PATH='/lustre/scratch/patricks/build'
@@ Line 222: / Line 243: @@
 === Running the Existing Chino Hills scenario with CVM-H ===
 * Build CVM-H
 * Build cvm2mesh linking in the previously build CVM-H
 * Extract Chino Hills 100m mesh
 * Execute AWP-ODC simulation for event=chino_uf and cvm=cvmh
 * Extract synthetic seimsograms from AWP and perform GoF calculation
 * Plot results
 * Package results
@@ Line 286: / Line 348: @@
 An equivalent script for CVM-H is already present in the official CVM-H distribution.
 *Update CVM-H/CVM-4 installation paths in Slice.py
   # CVM Constants
   cvmh_dir = '/home/rcf-104/patrices/cvmh/trunk/bin'

Patrices at 23:37, 1 November 2010

2010-11-01T23:37:39Z

@@ Line 214: / Line 214: @@
 === Installation ===
 TBD
-=== Running the Existing Chino Hills scenario with CVM-H ===
 TBD
 === Known Issues ===

Patrices at 21:30, 1 November 2010

2010-11-01T21:30:03Z

← Older revision		Revision as of 21:30, 1 November 2010
Line 24:		Line 24:


	=== Querying for SoCal Material Properties ===		=== Querying for SoCal Material Properties on the Command Line ===

			Two utilities are provided for querying the community velocity model, vx and vx_lite. The vx utility is the original interfance to Harvard's model. It is described in the PDF linked in the previous section. The vx_lite utility is a new SCEC-developed interface which supports a number of enhancements, including query by depth, extension of the coverage region with a SCEC 1D model, and replacement of the original GTL with a Vs30-derived GTL.

	The command line format for vx_lite is as follows:		The command line format for vx_lite is as follows:
Line 42:		Line 44:
	Note: Options -d and -v are mutually exclusive. Specifying neither option instructs vx_lite to interpret the z coordinate as elevation offset from free surface.		Note: Options -d and -v are mutually exclusive. Specifying neither option instructs vx_lite to interpret the z coordinate as elevation offset from free surface.

	The query data is printed to stdout, and this may be redirected to a file. ~~For the~~ sample test points (./bin/test_data/test.dat) ~~shown below~~:		The query data is printed to stdout, and this may be redirected to a file.


			CVM-H defines a set of sample test points (given in ./bin/test_data/test.dat):

	-125 35 -7777		-125 35 -7777
Line 52:		Line 57:
	376592 3773379 -17700		376592 3773379 -17700
	408669 3766189 -3000		408669 3766189 -3000


	These may be submitted to vx_lite with this command:		These may be submitted to vx_lite with this command:
Line 57:		Line 63:
	% ./vx_lite -s -v < ./test_data/test.dat		% ./vx_lite -s -v < ./test_data/test.dat

	The following annotated ~~output will be produced:~~
			The material properties at the sample points will be printed to stdout. You can expect to see the following output, annotated here with column headers. The three important fields are the last three, containing vp, vs, and rho.

	X Y Z utmX utmY elevX elevY topo mtop base moho hr/lr/cm cellX cellY cellZ tag vp vs rho		X Y Z utmX utmY elevX elevY topo mtop base moho hr/lr/cm cellX cellY cellZ tag vp vs rho
Line 68:		Line 75:
	376592.000000 3773379.000000 -17700.00 376592.00 3773379.00 376625.00 3773375.00 99.38 100.00 -2374.53 -28165.35 cm 380000.00 3770000.00 -18000.00 2.00 6533.31 3776.40 2841.47		376592.000000 3773379.000000 -17700.00 376592.00 3773379.00 376625.00 3773375.00 99.38 100.00 -2374.53 -28165.35 cm 380000.00 3770000.00 -18000.00 2.00 6533.31 3776.40 2841.47
	408669.000000 3766189.000000 -3000.00 408669.00 3766189.00 408625.00 3766125.00 93.89 50.00 -2820.45 -29799.86 hr 408552.25 3766250.00 -3000.00 2.00 4997.06 2889.03 2534.30		408669.000000 3766189.000000 -3000.00 408669.00 3766189.00 408625.00 3766125.00 93.89 50.00 -2820.45 -29799.86 hr 408552.25 3766250.00 -3000.00 2.00 4997.06 2889.03 2534.30


			=== Querying for SoCal Material Properties in a C Program ===

			SCEC has developed a C API to the Harvard model. The header containing the datatype and function definitions is located in ./src/vx_sub.h. By including this header in your source, and the libvxapi.a library at link time, your program can directly query CVM-H. See the cvm2mesh mesh extraction tool below for an example implementation that uses this API.

Patrices at 21:14, 1 November 2010

2010-11-01T21:14:12Z

← Older revision		Revision as of 21:14, 1 November 2010
Line 15:		Line 15:
	The following executable utilities are provided:		The following executable utilities are provided:
	;vx		;vx
	: Original interface to Harvard's model. Accepts lon/lat/elev points from stdin and prints material properties to stdout. The original user guide can be found here ~~(TBD)~~.		: Original interface to Harvard's model. Accepts lon/lat/elev points from stdin and prints material properties to stdout. The original user guide can be found here [[File:cvmh_manual.pdf]].

	;vx_lite		;vx_lite

Patrices at 20:10, 1 November 2010

2010-11-01T20:10:34Z

← Older revision		Revision as of 20:10, 1 November 2010
Line 42:		Line 42:
	Note: Options -d and -v are mutually exclusive. Specifying neither option instructs vx_lite to interpret the z coordinate as elevation offset from free surface.		Note: Options -d and -v are mutually exclusive. Specifying neither option instructs vx_lite to interpret the z coordinate as elevation offset from free surface.

	The query data is printed to stdout, and this may be redirected to a file. ~~The six~~ sample ~~point~~ test shown below:		The query data is printed to stdout, and this may be redirected to a file. For the sample test points (./bin/test_data/test.dat) shown below:

	% .~~/vx_lite~~ -s -~~v <~~ .~~/test_data/test~~.~~data~~		-125 35 -7777
			-118.56 32.55 -2450
			360061 3750229 -1400
			-118.52 34.12 -1400
			-116.40 32.34 -1000
			376592 3773379 -1770
			376592 3773379 -17700
			408669 3766189 -3000

	~~Will generate~~ this annotated output:		These may be submitted to vx_lite with this command:

			% ./vx_lite -s -v < ./test_data/test.dat

			The following annotated output will be produced:

	X Y Z utmX utmY elevX elevY topo mtop base moho hr/lr/cm cellX cellY cellZ tag vp vs rho		X Y Z utmX utmY elevX elevY topo mtop base moho hr/lr/cm cellX cellY cellZ tag vp vs rho