UCVM 13.9.0 User Guide
Contents
Overview
Released on September 8th, 2013, UCVM 13.9.0 represents the second major release of the Unified Community Velocity Model (UCVM) framework. UCVM is a collection of software utilities that are designed to make querying velocity models, building meshes, and visualizing velocity models easier to do through a uniform software interface. UCVM has been used extensively to generate meshes and e-trees that are then used for 3D wave propagation simulations within California.
The full feature list is as follows:
- Seamlessly combine two or more models into a composite model for querying
- Optionally include a California statewide geotechnical layer and interpolate it with the crustal velocity models
- Extract a 3D mesh or CVM Etree (octree) of material properties from any combination of models
- Standard California statewide elevation and Vs30 data map is provided
- Miscellaneous tools are provided for creating 2D etree maps, and optimizing etrees
- Numerically smooth discontinuities at the interfaces between different regional models
- Add support for future velocity models with the extendable interface
Currently, we support CVM-S4, CVM-H 11.9.1, CVM-S4.23, CenCal 0.8.0, and the Hadley-Kanamori as part of our automated installation package. Other models, such as SCEC CVM-NCI, Magistrale WFCVM, Graves Cape Mendocino, Lin-Thurber Statewide, and Tape SoCal are supported, however they will require a manual installation. We have also tested and include support for three major high-performance computing resources: NICS Kraken, TACC Stampede, and NCSA Blue Waters. It should work with other HPC machines as well.
The API itself is written entirely in C. We will show how to query UCVM through both C and Fortran in this user guide.
Finally, UCVM requires either Linux or OS X, GCC 4.3+, Python 2.5+, and an active internet connection to download and retrieve the models. For parallel mesh or e-tree extraction, the MPI libraries are also required. If you are installing UCVM on OS X, you must have the OS X Developer Tools (Xcode and Command Line Tools) and you must also have gfortran installed.
Download
| Platform | File | Download | Mirror |
|---|---|---|---|
| Linux | SCEC UCVM 13.9.0 Official Release (391 Mb) | ucvm-13.9.0.tar.gz | N/A |
| Linux | SCEC UCVM 13.9.0 md5 checksum (< 1Kb) | ucvm-13.9.0.tar.gz.md5 | N/A |
Installation Instructions
Easy Method
If you are installing UCVM on Linux or OS X and only need CVM-S4, CVM-H 11.9.1, CVM-S4.23, and/or CenCal, we strongly recommend following the easy installation method. Simply download UCVM 13.9.0 and run the following commands:
tar zxvf ucvm-13.9.0.tar.gz
cd ./UCVM
./ucvm_setup.py
You will then be asked a series of questions:
It looks like you are installing UCVM for the first time.
Where would you like UCVM to be installed?
(Default: /your/home/dir)>
Enter path or blank to use default:
Hit enter to use the default path or type your own. On high-performance computing resources you must change this path to be in the scratch or work directory so that UCVM can be seen by the compute nodes.
You will then be asked which models you'd like to download, such as:
Would you like to download and install CVM-S4?
Enter yes or no:
Enter "yes" (without quotation marks) or "no" if you would like to download this model or not.
After answering a few of these questions, UCVM will then begin the process of downloading and building the models you have requested. When the process is completed, you will be advised on how to test to verify that your installation is working and also any modifications that may be needed to your ~/.bash_profile.
Custom Method
Please see this page on how to install UCVM with models other than CVM-S4, CVM-H, CVM-S4.23, and CenCal.
Configuration Files
UCVM Main Configuration Format (conf/ucvm.conf)
The main UCVM configuration file, conf/ucvm.conf, specifies the velocity models that have been installed and registered with UCVM. An example configuration file is as follows:
# UCVM config file # UCVM model path # # Change to reflect UCVM installation directory. # ucvm_interface=map_etree ucvm_mappath=/path/to/ucvm-13.9.0/model/ucvm/ucvm.e # Pre-defined models # # SCEC CVM-S cvms_modelpath=/path/to/ucvm-13.9.0/model/cvms4/src # # SCEC CVM-H cvmh_modelpath=/path/to/ucvm-13.9.0/model/cvmh1191/model # # USGS Bay Area high-rez and extended etrees cencal_modelpath=/path/to/ucvm-13.9.0/model/cencal/USGSBayAreaVM-08.3.0.etree cencal_extmodelpath=/path/to/ucvm-13.9.0/model/cencal/USGSBayAreaVMExt-08.3.0.etree # # SCEC CVM-SI cvmsi_modelpath=/path/to/ucvm-13.9.0/model/cvms423/model/i23 # # SCEC CVM-NCI cvmnci_modelpath=/path/to/ucvm-13.9.0/cvmnci/model/i2 # # Wasatch Front CVM wfcvm_modelpath=/path/to/ucvm-13.9.0/wfcvm/src # # Lin-Thurber Statewide cvmlt_modelpath=/path/to/ucvm-13.9.0/cvmlt/model # # Cape Mendocino RG cmrg_modelpath=/path/to/ucvm-13.9.0/model/cmrg.conf # # Tape SoCal Model tape_modelpath=/path/to/ucvm-13.9.0/cvm-tape/model/m16 # # 1D 1d_modelpath=/path/to/ucvm-13.9.0/model/1d/1d.conf # # 1D GTL 1dgtl_modelpath=/path/to/ucvm-13.9.0/model/1d/1d.conf # User-defined models # # Change to reflect paths to user model files # # Current supported interfaces: model_etree, model_patch # #user1_interface=model_etree #user1_modelpath=/home/username/model_etree/user1.etree #user2_interface=model_patch #user2_modelpath=/home/username/model_patch/user2.conf # User-defined maps # # Change to reflect paths to user map files # # Current supported interfaces: map_etree # # Optional Yong-Wald vs30 map yong_interface=map_etree yong_mappath=/path/to/ucvm-13.9.0/model/ucvm/ucvm_yong_wald.e # Model flags # cvmh_param=USE_1D_BKG,True cvmh_param=USE_GTL,True
The first two lines specify the location and type of the DEM and Vs30 data:
ucvm_interface=map_etree
ucvm_mappath=/path/to/ucvm-13.9.0/model/ucvm/ucvm.e
The file, ucvm.e, contains USGS NED 1 arcsec DEM and Wills-Wald Vs30 data. An alternate file, ucvm_yong_wald.e, is also available. It contains USGS NED 1 arcsec DEM and Yong-Wald Vs30 data.
The next lines specify the models that are available and the location of their model (data) directories. For example, the following code tells UCVM to look for the CVM-S4 data in /path/to/ucvm-13.9.0/model/cvms4/src.
cvms_modelpath=/path/to/ucvm-13.9.0/model/cvms4/src
Every model that is installed must have an entry in this file. UCVM also has the ability to use an etree as a community velocity model. To use and query an etree, you need to add the following lines to ucvm.conf:
myetree_interface=model_etree myetree_modelpath=/path/to/my/custom/model.etree
Then, you can reference your etree similar to any model, like so:
./ucvm_query -f ../conf/ucvm.conf -m myetree
CVM-H also has two user-configurable flags, USE_1D_BKG and USE_GTL, which when set to true tell UCVM to use the 1D background model and use the Ely GTL, respectively.
ucvm2etree Configuration File Format
UCVM includes the ability to generate either a SCEC or CMU formatted etree from any registered velocity model. In order to generate an etree, a configuration file must be provided that give the parameters of extraction. An example configuration file is as follows:
# Domain corners coordinates (clockwise, degrees):
proj=geo-bilinear
lon_0=-119.288842
lat_0=34.120549
lon_1=-118.354016
lat_1=35.061096
lon_2=-116.846030
lat_2=34.025873
lon_3=-117.780976
lat_3=33.096503
# Domain dimensions (meters):
x-size=180000.0000
y-size=135000.0000
z-size=61875.0000
# Blocks partition parameters:
nx=32
ny=24
# Max freq, points per wavelength, Vs min
max_freq=0.5
ppwl=4.0
vs_min=200.0
# Max allowed size of octants in meters
max_octsize=10000.0
# Etree parameters and info
title=ChinoHills_0.5Hz_200ms
author=D_Gill
date=05/2011
outputfile=./cmu_cvmh_chino_0.5hz_200ms.e
format=etree
# UCVM parameters
ucvmstr=cvms
ucvm_interp_zrange=0.0,350.0
ucvmconf=../../conf/kraken/ucvm.conf
# Scratch
scratch=/lustre/scratch/scecdave/scratch
#
# Buffering parameters used by MPI version only
#
# Etree buffer size in MB
buf_etree_cache=128
# Max octants to buffer for flat file during extraction
buf_extract_mem_max_oct=4194304
# Max octants to save in flat file before reporting full during extraction
buf_extract_ffile_max_oct=16000000
# Max octants to read from input flat file during sorting
buf_sort_ffile_max_oct=20000000
# Minimum number of octants between reports during merging
buf_merge_report_min_oct=10000000
# MPI send/recv octant buffer size during merging
buf_merge_sendrecv_buf_oct=4096
# Etree read/write octant buffer size during merging
buf_merge_io_buf_oct=4194304
The file first specifies the four corners of the etree volume, in latitude, longitude format. The bounding box must be either square or rectangular.
proj=geo-bilinear
lon_0=-119.288842
lat_0=34.120549
lon_1=-118.354016
lat_1=35.061096
lon_2=-116.846030
lat_2=34.025873
lon_3=-117.780976
lat_3=33.096503
The etree volume length, width, and height must also be specified in meters.
x-size=180000.0000
y-size=135000.0000
z-size=61875.0000
We must also specify the number of partitions in the X and Y directions. Note that nx * ny must be equal to the number of cores. We must also specify the maximum frequency, points per wavelength, and Vs minimum as well. The maximum frequency and PPWL are terms that decide on the discretization of the mesh. Specifically, the points per wavelength is the quantity used to piece-wise discretize the length of the traveling waves. It will basically define how many "elements" you need to represent a wave cycle of a certain length accurately enough up to a user defined or numerical method tolerance. This in turn is associated to the dominant frequency of the traveling wavefield and the medium through which it is traveling, that is, the seismic velocities of the material. Therefore, higher frequencies and low velocities mean shorter wavelengths. As a result, the maximum frequency and the minimum velocity to be considered in a simulation as it defines the minimum size of a grid or a mesh model.
In building the etree respresentation of a velocity model we use the rule:
e = Vs / ( f_max * P )
where e is the element containing any given point with shear wave velocity, Vs, f_max is the maximum frequency and P is the number of points per wavelength.
After defining this, we must also define the maximum size, in meters, that any one "cell" or octant can have.
# Blocks partition parameters:
nx=32
ny=24
# Max freq, points per wavelength, Vs min
max_freq=0.5
ppwl=4.0
vs_min=200.0
# Max allowed size of octants in meters
max_octsize=10000.0
We must also specify some metadata for our etree. For example, we must specify the etree's title, author, date it was made, output location (file location), and format=etree (this is a constant).
# Etree parameters and info
title=ChinoHills_0.5Hz_200ms
author=D_Gill
date=09/2013
outputfile=./cmu_cvmh_chino_0.5hz_200ms.e
format=etree
In order to extract an etree, the CVM from which we're extracting must be provided (ucvmstr=cvms). If we're using a Vs30 GTL, we can specify the interpolation range, and we also need to specify the location of the ucvm.conf file we're using. Also, we need to specify the location of a scratch or temporary directory where we can write the temporary data files.
# UCVM parameters
ucvmstr=cvms
ucvm_interp_zrange=0.0,350.0
ucvmconf=../../conf/kraken/ucvm.conf
# Scratch
scratch=/lustre/scratch/scecdave/scratch
There are also a number of buffer values in the configuration file that must be accurately set for extraction, sorting, and merging of the MPI version of ucvm2etree to take place. Please see below for a list of those values and a description of the purpose of each one.
buf_etree_cache
Units: MB
Recommended Value: 128 usually works well, you can increase or decrease as need be
For Process: ucvm2etree-extract-MPI
Description: Used in the merging process during the etree_open call. According to the documentation of etree_open, this is more precisely defined as the "internal buffer space being allocated in megabytes". The value defined represents the memory buffer for etree traversing, finding, writing, etc.
buf_extract_mem_max_oct
Units: Number of octants
Recommended Value: 2097152
For Process: ucvm2etree-extract-MPI
Description: Used in the extraction process as the number of bytes to buffer before writing to disk. It reserves, in memory, a placeholder for this number of octants. The size of each octant is 52 bytes and as such, the actual memory allocation is 52 * this value.
buf_extract_ffile_max_oct
Units: Number of octants
Recommended Value: Varies. It must be greater than total output file size divided by the number of processes.
For Process: ucvm2etree-extract-MPI
Description: This value is very critical! This defines the maximum output file size that each worker process can write. If this is not large enough, the worker will halt and return a "worker is full message". The best way to set this is to guess how large your extraction file might be (a relatively detailed Chino Hills e-tree, for example, is 250GB, and divide that number by 52 * the number or processes you have. So if I was building that mesh, on 32 processes, this value would need to be at least 162 million. However, this value also ties in with the number below, so please read it carefully as well.
buf_sort_ffile_max_oct
Units: Number of octants
Recommended Value: Must be at least equal to buf_extract_ffile_max_oct
For Process: ucvm2etree-sort-MPI
Description: Used in the sorting process to determine the max number of octants to read into memory. Therefore, for each process running on the server, there must be number of processes * 52 * buf_sort_ffile_max_oct bytes of memory available. So in our example above, if we were running those 32 processes on 16 dual core machines, we would need 162 million * 52 * 2 cores bytes of RAM or about 16 GB. All sorting is done in memory, not using the file on the disk.
buf_merge_report_min_oct
Units: Number of octants
Recommended Value: Personal preference - default is 10000000.
For Process: ucvm2etree-merge-MPI
Description:. This defines the number of octants that must be processed between progress reports in the merge process. This progress reports take the form of "Appended [at least buf_merge_report_min_oct] in N.NNs seconds". If you would like more frequent updates, set this value to be smaller.
buf_merge_sendrecv_buf_oct
Units: Bytes
Recommended Value: 4096
For Process: ucvm2etree-merge-MPI
Description: The number of bytes to merge per loop during the merge process. It is recommended to keep this value as is.
buf_merge_io_buf_oct
Units: Number of octants
Recommended Value: 4194304
For Process: ucvm2etree-merge-MPI
Description: Number of octants to read in and store in memory for merging. After being merged, the memory is then flushed to disk and the process repeats. Since this is a value in octants, it will require the machine to have at least buf_merge_io_buf_oct * 52 bytes of memory.
#
# Buffering parameters used by MPI version only
#
# Etree buffer size in MB
buf_etree_cache=128
# Max octants to buffer for flat file during extraction
buf_extract_mem_max_oct=4194304
# Max octants to save in flat file before reporting full during extraction
buf_extract_ffile_max_oct=16000000
# Max octants to read from input flat file during sorting
buf_sort_ffile_max_oct=20000000
# Minimum number of octants between reports during merging
buf_merge_report_min_oct=10000000
# MPI send/recv octant buffer size during merging
buf_merge_sendrecv_buf_oct=4096
# Etree read/write octant buffer size during merging
buf_merge_io_buf_oct=4194304
ucvm2mesh Configuration File Format
Extracting a mesh, much like extracting an etree, also requires a separate configuration file that specifies the parameters of the mesh that you would like to extract.
# List of CVMs to query
ucvmlist=cvmh
# UCVM conf file
ucvmconf=../conf/kraken/ucvm.conf
# Gridding cell centered or vertex
gridtype=CENTER
# Spacing of cells
spacing=2000.0
# Projection
proj=+proj=utm +datum=WGS84 +zone=11
rot=-40.0
x0=-122.3
y0=34.7835
z0=0.0
# Number of cells along each dim
nx=384
ny=248
nz=25
# Partitioning of grid among processors
px=2
py=2
pz=5
# Vs/Vp minimum
vp_min=0
vs_min=0
# Mesh and grid files, format
meshfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.media
gridfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.grid
# Location of scratch dir
scratch=/lustre/scratch/user/scratch
Fundamentally, we must specify the simulation box bottom-left corner and rotation, which CVM to use, and the the grid type and spacing. We must also specify the location of the UCVM configuration file.
# List of CVMs to query
ucvmlist=cvmh
# UCVM conf file
ucvmconf=../conf/kraken/ucvm.conf
# Gridding cell centered or vertex
gridtype=CENTER
# Spacing of cells
spacing=2000.0
# Projection
proj=+proj=utm +datum=WGS84 +zone=11
rot=-40.0
x0=-122.3
y0=34.7835
z0=0.0
We also need to specify the length, width, and depth of our simulation volume. The length of the volume is defined as nx * spacing, the width is ny * spacing, and the depth is nz * spacing. For the MPI version of ucvm2mesh, we need to specify the number of processors to dedicate to each axis. The total number of processors required is px * py * pz. For this example, we would need 20 cores dedicated to this task.
# Number of cells along each dim
nx=384
ny=248
nz=25
# Partitioning of grid among processors
px=2
py=2
pz=5
Finally, we must also specify a floor for Vs and Vp. We must also specify the location for our mesh file and the grid file. As well, we need to specify a directory (scratch directory) to use for temporary files.
# Vs/Vp minimum
vp_min=0
vs_min=0
# Mesh and grid files, format
meshfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.media
gridfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.grid
# Location of scratch dir
scratch=/lustre/scratch/user/scratch
Single Core Command Reference
basin_query
The command basin_query allows you to retrieve the depth at which vs-threshold is first crossed. By default, vs-threshold is set to be 1000m/s, but that can easily be changed with the "-v" flag.
Example usage:
./basin_query ../conf/ucvm.conf -m cvms -v 2500
ecoalesce
The command ecoalesce helps compact an e-tree file that conforms either to CMU or SCEC standards. It does this by replacing eight adjacent octants with identical material properties at level N with a single octant containing the same material properties at level N-1. Usually, this command is run with ecompact as well.
Example usage:
./ecoalesce chino_hills.etree compacted_chino_hills.etree
ecompact
The command ecompact helps compact an e-tree file that conforms either to CMU or SCEC standards. It does this by removing empty space in the Etree data structure. Usually, this command is run with ecoalesce as well.
Example usage:
./ecompact chino_hills.etree compacted_chino_hills.etree
grd_query
The command grd_query queries data from a set of ArcGIS grid files in GridFloat format.
grd2etree
The command grd2etree extracts a SCEC-formatted Etree map from a set of DEM and Vs30 grid files in ArcGIS Gridfloat format.
Example usage:
./grd2etree -f ./grd2float_sample.conf
mesh-check
The command mesh-check does a basic quality assurance check of a mesh file. It checks to make sure that each record in the file is of the correct size. Furthermore, it checks to make sure that each value is not NaN, infinity, or negative.
Example usage:
./mesh-check new_mesh.mesh IJK-12
mesh-op
The command mesh-op subtracts a mesh from another mesh and outputs the difference.
Example usage:
./mesh-op diff ./inmesh1 ./inmesh2 IJK-12 ./outmesh
mesh-strip-ijk
The command mesh-strip-ijk converts an IJK-20 or IJK-32 mesh to an IJK-12 formatted mesh.
Example usage:
./mesh-strip-ijk ijk20_mesh IJK-20 output_mesh
ucvm_query
This is the command-line tool for querying CVMs. Any set of crustal and GTL velocity models may be selected and queried in order of preference. Points may be queried by (lon,lat,dep) or (lon,lat,elev) and the coordinate conversions for a particular model are handled transparently. A configuration file needs to be passed to ucvm_query. Typically, this would be /path/to/ucvm-13.9.0/conf/ucvm.conf.
Example usage:
./ucvm_query -f ../conf/ucvm.conf -m cvms < ../tests/cvms_input.txt
Using Geo Depth coordinates as default mode.
-118.0000 34.0000 0.000 280.896 390.000 cvms 696.491 213.000 1974.976 none 0.000 0.000 0.000 crust 696.491 213.000 1974.976
ucvm2etree
The command ucvm2etree builds an e-tree from the specifications in a given configuration file, config.
Note that this is the serial version of the command, meaning that it will only run on a single process. As such, building a large e-tree can be very slow. For large e-trees, we strongly recommend using ucvm2etree-extract-MPI, ucvm2etree-sort-MPI, and ucvm2etree-merge-MPI.
Example usage:
./ucvm2etree -f ./ucvm2etree_example.conf
ucvm2mesh
The command ucvm2mesh generates a mesh in either IJK-12, IJK-20, IJK-32, or SORD format. It does so by reading in the specifications in a given configuration file, config.
This mesh can then be used in forward 3D wave propagation simulation software such as AWP-ODC.
Example usage:
./ucvm2mesh -f ./ucvm2mesh_example.conf
MPI Command Reference
ucvm2etree-extract-MPI
Notice: This command is intended to be run as a MPI job (e.g. mpirun ./ucvm-extract-MPI). Please do not attempt to run it as a regular process.
The command ucvm2etree-extract-MPI extracts components of an e-tree from the specifications in a given configuration file, config.
Specifically, it divides the etree region into C columns for extraction. This is an embarrassingly parallel operation. A dispatcher (rank 0) farms out each column to a worker in a pool of N cores for extraction. Each worker queries UCVM for the points in its column and writes a flat-file formatted etree. After program execution, there are N sub-etree files, each locally unsorted. The extractor must be run on 2^Y + 1 cores where Y>0 and (2^Y) < C. The output flat file format is a list of octants (24 byte addr, 16 byte key, 12 byte payload) in arbitrary Z-order.
Since the number of points in a column depends on the minimum Vs values within that column, some columns will have high octant counts and others will have very low octant counts. Having sub-etrees that vary greatly in size is not optimal for the sorting operations that follow, so ucvm2etree-extract-MPI implements a simple octant balancing mechanism. When a worker has extracted more than X octants (the default 16M octants), it reports to the dispatcher that it cannot extract any more columns and terminates. This strategy approximately balances the sub-etrees so that they may be loaded into memory by ucvm2etree-sort-MPI. In the case of very large extractions where the dispatcher reports that all workers have reported they are full yet columns remain to be extracted, increase the job size by a factor of 2 until there is room for all the columns.
You would typically run this command, followed by ucvm2etree-sort-MPI and ucvm2etree-merge-MPI.
mpirun -np 769 ucvm2etree-extract-MPI -f ./ucvm2etree_example.conf
ucvm2etree-sort-MPI
Notice: This command is intended to be run as a MPI job (e.g. mpirun ./ucvm-sort-MPI). Please do not attempt to run it as a regular process.
The command ucvm2etree-sort-MPI sorts the extracted components of an e-tree from the ucvm2etree-extract-MPI command. It does so by reading in the specifications in a given configuration file, config.
Specifically, it sorts the sub-etrees produced by ucvm2etree-extract-MPI so that each file is in local pre-order (Z-order). Again, the is an embarrassingly parallel operation. Each rank in the job reads in one of the sub-etrees produced by the previous program, sorts the octants in Z-order, and writes the sorted octants to a new sub-etree. The sorter must be run on 2^Y cores where Y>0. The worker pool must be large enough to allow each worker to load all the octants from its assigned file into memory. By default, this octant limit is 20M octants. If a rank reports that the size of the sub-etree exceeds memory capacity, the 20M buffer size constant may be increased if memory allows, or alternatively, cvm-bycols-extract-MPI may be rerun with a larger job size to reduce the number of octants per file.
You would typically run this command after ucvm2etree-extract-MPI and before ucvm2etree-merge-MPI.
mpirun -np 768 ucvm2etree-sort-MPI -f ./ucvm2etree_example.conf
ucvm2etree-merge-MPI
Notice: This command is intended to be run as a MPI job (e.g. mpirun ./ucvm-merge-MPI). Please do not attempt to run it as a regular process.
The command ucvm2etree-merge-MPI merges the sorted components of an e-tree from the ucvm2etree-sort-MPI command. It does so by reading in the specifications in a given configuration file, config.
Specifically, it merges N locally sorted etrees in flat file format into a final, compacted etree. This is essentially a merge sort on the keys from the addresses read from the local files. The cores at the lowest level of the merge tree each read in octants from two flat files in pre-order, merge sort the two sets of addresses, then pass the locally sorted list of addresses to a parent node for additional merging. This proceeds until the points rise to rank 1 which has a completely sorted list of etree addresses. Rank 0 takes this sorted list and performs a transactional append on the final Etree.
The merger must be run on 2^N cores. The program reads in input files that are in flat file format. It can output a merged Etree in either Etree format or flat file format. Although, due to space considerations, it strips the output flat file format to a pre-order list ot octants(16 byte key, 12 byte pay-load). The missing addr field is redundant and can be regenerated from the key field.
You would typically run this command after ucvm2etree-extract-MPI and ucvm2etree-merge-MPI.
Example usage:
mpirun -np 768 ucvm2etree-merge-MPI -f ./ucvm2etree_example.conf
ucvm2mesh-mpi
Notice: ucvm2mesh-mpi is meant to be run as a MPI job, not as a standalone executable.
The command ucvm2mesh-mpi generates a mesh in either IJK-12, IJK-20, IJK-32, or SORD format. Unlike its serial version, ucvm2mesh this command can use multiple cores to generate the mesh. It does so by reading in the specifications in a given configuration file.
This mesh can then be used in forward 3D wave propagation simulation software such as AWP-ODC.
mpirun -np 768 ucvm2mesh-mpi -f ./ucvm2mesh_example.conf
Troubleshooting
If you see an error similar to the following while running either the tests or the UCVM programs:
error while loading shared libraries: libsomelib.so: cannot open shared object file: No such file or directory
This indicates that UCVM was linked against one or more shared libraries and the dynamic library loader cannot find the actual .so library at run-time. The solution is to update your LD_LIBRARY_PATH to include the directory containing the library mentioned in the error message. For example, the following command adds a new search directory to LD_LIBRARY_PATH in a csh shell:
$ setenv LD_LIBRARY_PATH /home/USER/opt/somepkg/lib:${LD_LIBRARY_PATH}
By default, the ucvm_setup.py script uses static libraries. UCVM is only compiled dynamically if you pass ucvm_setup.py the "-d" option or if your system does not support static linking.
Proj. 4 Error: major axis or radius = 0 or not given
On systems with home filesystems that are not viewable to compute nodes (such as NICS Kraken), you may encounter errors with Proj.4 when trying to run any component of UCVM on compute nodes. This is due to the fact that Proj.4 actually relies on a file called proj_defs.dat which is located in the ${MAKE_INSTALL_LOCATION}/share/proj directory. So for example, suppose you had configured Proj.4 with ./configure --prefix=/not_viewable_to_compute_nodes/proj-4.7.0, Proj. 4 would then search for proj_defs.dat in /not_viewable_to_compute_nodes/proj-4.7.0/share/proj/proj_defs.dat. This will cause UCVM to throw the error "Proj.4 Error: major axis or radius = 0 or not given" and your job will fail.
The only way it seems to solve this issue is to actually make sure your --prefix directory is actually visible to the compute nodes and do a make install there. Documentation suggests that you can set the PROJ_LIB environment variable, however this seems to not work correctly without modifications to the Proj.4 source code.
API Examples
Please note that the examples shown below are basic examples of how the API works. More extensive documentation can be found in the UCVM 13.9.0 Developer Guide.
C
Calling UCVM from C code is relatively trivial. Minimally, you need the UCVM library as well as "ucvm.h".
Using UCVM involves three fundamental steps: 1) Initializing UCVM through the ucvm_init function, 2) adding appropriate models through the ucvm_add_model function, and 3) retrieving material properties through the ucvm_query function. These functions are shown in the example below.
Example.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <getopt.h>
#include <sys/time.h>
#include "ucvm.h"
int main(int argc, char **argv)
{
int nn = 1;
ucvm_point_t pnts;
ucvm_data_t data;
char cmb_label[UCVM_MAX_LABEL_LEN];
printf("Init\n");
if (ucvm_init("../conf/test/ucvm.conf") != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Init failed\n");
return(1);
}
printf("Query Mode\n");
if (ucvm_setparam(UCVM_PARAM_QUERY_MODE,
UCVM_COORD_GEO_DEPTH) != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Failed to set z mode\n");
return(1);
}
printf("Add Crustal Model 1D\n");
if (ucvm_add_model(UCVM_MODEL_1D) != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Retrieval of 1D failed\n");
return(1);
}
printf("Add GTL Model Ely\n");
if (ucvm_add_model(UCVM_MODEL_ELYGTL) != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Retrieval of Ely GTL failed\n");
return(1);
}
/* Change GTL interpolation function from default (linear)
to Ely interpolation */
if (ucvm_assoc_ifunc(UCVM_MODEL_ELYGTL,
UCVM_IFUNC_ELY) != UCVM_CODE_SUCCESS) {
fprintf(stderr,
"Failed to associate interpolation function with Ely GTL\n");
return(1);
}
/* Change interpolation z range from 0,0 to 0,350 */
if (ucvm_setparam(UCVM_PARAM_IFUNC_ZRANGE, 0.0,
350.0) != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Failed to set interpolation range\n");
return(1);
}
printf("Create point\n");
pnts.coord[0] = -118.0;
pnts.coord[1] = 34.0;
pnts.coord[2] = 2000.0;
printf("Query Model\n");
if (ucvm_query(nn, &pnts, &data) != UCVM_CODE_SUCCESS) {
fprintf(stderr, "Query UCVM failed\n");
return(1);
}
/* Get cmb data label */
ucvm_ifunc_label(data.cmb.source,
cmb_label, UCVM_MAX_LABEL_LEN);
printf("Results:\n");
printf("\tsource=%s, vp=%lf, vs=%lf, rho=%lf\n",
cmb_label, data.cmb.vp, data.cmb.vs, data.cmb.rho);
return(0);
}
Fortran
Calling UCVM using Fortran is a relatively trivial procedure. After you have installed UCVM as per this user guide, you must include the UCVM library, the Proj.4 library, the e-tree library, as well as any velocity model libraries that you have compiled into UCVM. For CVM-H, please note that there are actually two libraries required: lvxapi and lgeo. Because the default convention for calling C programs from Fortran automatically appends an underscore to the end of the function name, you must turn that off via a flag called "fno-underscoring". This will make the Fortran compiler try and find foo() instead of foo_().
As an example, suppose we have a Fortran file, example.f, that calls UCVM. We have compiled UCVM with CVM-S and CVM-H. The code to compile example.f would be as follows:
gfortran example.f -o ./example -L/path/to/ucvm-13.9.0/lib -L./path/to/ucvm-13.9.0/model/cvms4/lib -L/path/to/ucvm-13.9.0/model/cvmh1191/lib -L/path/to/ucvm-13.9.0/lib/proj-4/lib -L/path/to/ucvm-13.9.0/lib/euclid3/libsrc -lucvm -lcvms -lvxapi -lgeo -lproj -letree -fno-underscoring
The basic structure of how to call UCVM within Fortran is outlined in the example below.
Example.f
program example
c UCVM Configuration Location
CHARACTER(LEN=80) ucvmconf
c Model Name
CHARACTER(LEN=4) model
c Number of points we're passing to ucvm_query
INTEGER pts
c The UCVM point data structure.
c coord(1) is longitude
c coord(2) is latitutde
c coord(3) is depth
TYPE :: ucvm_point_t
REAL*8 coord(3)
END TYPE ucvm_point_t
c Generic property structure
c Source is where it comes from
c vp is P-wave velocity in m/s
c vs is S-wave velocity in m/s
c rho is density in kg/m^3
TYPE :: ucvm_prop_t
INTEGER source
REAL*8 vp
REAL*8 vs
REAL*8 rho
END TYPE ucvm_prop_t
c Returned data structure
TYPE :: ucvm_data_t
REAL*8 surf
REAL*8 vs30
REAL*8 depth
INTEGER domain
REAL*8 shift_cr
REAL*8 shift_gtl
type(ucvm_prop_t) crust
type(ucvm_prop_t) gtl
type(ucvm_prop_t) cmb
END TYPE ucvm_data_t
c For our example we'll query five points
type(ucvm_point_t) point(5)
c And we'll get back five sets of material properties
type(ucvm_data_t) returnData(5)
c Number of points is 5.
pts = 5
c We'll start at -118, 34 at 0 depth and go down by 1000m
c each step
do 10 i = 1, 5
point(i)%coord(1) = -118
point(i)%coord(2) = 34
point(i)%coord(3) = (i - 1) * 1000
10 continue
c Where is our configuration file?
ucvmconf = "/home/scec-01/davidgil/ucvm.conf" // CHAR(0)
c What model are we querying?
model = "cvms"
c Initialize UCVM
call ucvm_init(ucvmconf)
c Add the model to UCVM
call ucvm_add_model(model)
c Query the model. Note that the number of points is passed
c by value, not reference.
call ucvm_query(%VAL(pts), point, returnData)
print *, model, " results for lon -118, lat 34"
c Print out the results.
do 20 i = 1, 5
print *, "Depth ", (i - 1) * 1000
print *, "Vs ", returnData(i)%crust%vs
print *, "Vp ", returnData(i)%crust%vp
print *, "Rho ", returnData(i)%crust%rho
20 continue
c Close UCVM now that we've queried the points
call ucvm_finalize()
end
GCC Fortran 4.3+ is required for this example to work.
History of UCVM
- 13.9.0 (Sep 8, 2013): Improved installation, basin_query utility, OS X compliance
- 12.2.0 (Feb 9, 2012): Initial release
Acknowldgements and Contact Info
Support for the development and maintenance of the UCVM framework has been provided by the Southern California Earthquake Center (SCEC). SCEC is funded by NSF Cooperative Agreement EAR-0106924 and USGS Cooperative Agreement 02HQAG0008.
Contributions to this manual were made by: David Gill, Patrick Small, and Philip Maechling.
Please email software@scec.org for help on downloading and using UCVM, and for any suggestions for the delivery of the code or for this manual.
Please reference at least Small et al (2011) if you use this software framework; other references should be considered, depending on the purpose.
References
- Ely, G., T. H. Jordan, P. Small, P. J. Maechling (2010), A Vs30-derived Near-surface Seismic Velocity Model Abstract S51A-1907, presented at 2010 Fall Meeting, AGU, San Francisco, Calif., 13-17 Dec. [Ely2010-AGU.pdf]
- Graves, R. (1994), Rupture History and Strong Motion Modeling of the 1992 Cape Mendocino Earthquake, USGS External Grant Report
- Lin, G., C. H. Thurber, H. Zhang, E. Hauksson, P. Shearer, F. Waldhauser, T. M. Brocher, and J. Hardebeck (2010), A California statewide three-dimensional seismic velocity model from both absolute and differential Times, Bull. Seism. Soc. Am., 100, in press. supplemental
- Small, P., P. Maechling, T. Jordan, G. Ely, and R. Taborda (2011), SCEC UCVM - Unified California Velocity Model, in 2011 Southern California Earthquake Center Annual Meeting, Proceedings and Abstracts, vol. TBD, p. TBD.
- Taborda R., López J., O'Hallaron D., Tu T. and Bielak J. (2007), A review of the current approach to CVM-Etrees, SCEC Annual Meeting, Palm Springs, CA, USA, September 8–12. [1]
- Wald, D. J., and T. I. Allen (2007), Topographic slope as a proxy for seismic site conditions and amplification, Bull. Seism. Soc. Am., 97 (5), 1379-1395, doi:10.1785/0120060267.
- Wills, C. J., and K. B. Clahan (2006), Developing a map of geologically defined site-condition categories for California, Bull. Seism. Soc. Am., 96 (4A), 1483-1501, doi:10.1785/0120050179.
- Yong, A., Hough, S.E., Iwahashi, J., and A. Braverman (2012), A terrain-based site conditions map of California with implications for the contiguous United States, Bull. Seism. Soc. Am., Vol. 102, No. 1, pp. 114–128, February 2012, doi: 10.1785/0120100262.
