Difference between revisions of "UCVM User Guide"
(Redirected page to UCVM 14.3.0 User Guide) |
|||
(66 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | #REDIRECT [[UCVM 14.3.0 User Guide]] | ||
+ | |||
= Overview = | = Overview = | ||
− | The Unified | + | The Unified Community Velocity Model Framework [[UCVM]] is a software package which provides a standard interface to several regional community velocity models. Any differences in coordinate projections within each model are hidden so that the researcher queries any of the models by longitude, latitude, and Z (where Z is depth from surface or elevation relative MSL). UCVM also provides several other unique capabilities and products, including: |
− | The software is MPI-capable, and has been shown to scale to 3300+ cores on NICS Kraken in a mesh generator. It allows application programs to tailor which set of velocity models are linked in to minimize memory footprint. It is also compatible with grid resources that have data stage-in policies for jobs, and light-weight kernels that support static libraries only. This package has been tested on NCCS Jaguar, NICS Kraken, and USC HPCC. | + | * Seamlessly combine two or more models into a composite model for querying |
+ | * Optionally include a California statewide geotechnical layer and interpolate it with the crustal velocity models | ||
+ | * Extract a 3D mesh or CVM Etree (octree) of material properties from any combination of models | ||
+ | * Standard California statewide elevation and Vs30 data map is provided | ||
+ | * Miscellaneous tools are provided for creating 2D etree maps, and optimizing etrees | ||
+ | * Numerically smooth discontinuities at the interfaces between different regional models | ||
+ | * Add support for future velocity models with the extendable interface | ||
+ | |||
+ | |||
+ | Current list of supported velocity models: SCEC CVM-H, SCEC CVM-S, SCEC CVM-SI, SCEC CVM-NCI, Magistrale WFCVM, USGS CenCalVM, Graves Cape Mendocino, Lin-Thurber Statewide, Tape SoCal. | ||
+ | |||
+ | The API itself is written in the C language. The software is MPI-capable, and has been shown to scale to 3300+ cores on NICS Kraken in a mesh generator. It allows application programs to tailor which set of velocity models are linked in to minimize memory footprint. It is also compatible with grid resources that have data stage-in policies for jobs, and light-weight kernels that support static libraries only. This package has been tested on NCCS Jaguar, NICS Kraken, and USC HPCC. | ||
Line 9: | Line 22: | ||
The system requirements are as follows: | The system requirements are as follows: | ||
− | # UNIX operating system (Linux | + | # UNIX operating system (Linux) |
# GNU gcc/gfortran compilers (MPI wrappers such as mpicc are OK) | # GNU gcc/gfortran compilers (MPI wrappers such as mpicc are OK) | ||
# tar for opening the compressed files | # tar for opening the compressed files | ||
− | # | + | # [http://www.cs.cmu.edu/~euclid/ Euclid Etree library] (recommend [http://hypocenter.usc.edu/research/ucvm/euclid3-1.3.tar.gz euclid3-1.3.tar.gz]) |
− | # Proj.4 | + | # [http://trac.osgeo.org/proj/ Proj.4 Projection Library] (recommend [http://download.osgeo.org/proj/proj-4.7.0.tar.gz proj-4.7.0.tar.gz]) |
+ | |||
+ | |||
+ | Optional dependencies include any of the following standard velocity models and packages: | ||
+ | # Standard community velocity models: SCEC CVM-H, SCEC CVM-S, SCEC CVM-SI, SCEC CVM-NCI, Magistrale WFCVM, USGS CenCalVM, Graves Cape Mendocino, Lin-Thurber Statewide, Tape SoCal | ||
+ | # NetCDF (network Common Data Form) library [http://www.unidata.ucar.edu/downloads/netcdf/index.jsp NetCDF] | ||
= Installation = | = Installation = | ||
Line 22: | Line 40: | ||
#Start at SCEC website: [http://scec.usc.edu/scecpedia/UCVM http://scec.usc.edu/scecpedia/UCVM] | #Start at SCEC website: [http://scec.usc.edu/scecpedia/UCVM http://scec.usc.edu/scecpedia/UCVM] | ||
#Navigate to the Downloads section of the [[UCVM]] web page. | #Navigate to the Downloads section of the [[UCVM]] web page. | ||
− | #Click the download link to download the latest source code distribution file and any listed dependencies. This is typically posted as a tar and gzipped file (tgz format). The source code distribution file is large ( | + | #Click the download link to download the latest source code distribution file and any listed dependencies. This is typically posted as a tar and gzipped file (tgz format). The source code distribution file is large (350 Mb), so the download make take awhile. |
== CheckSum Test == | == CheckSum Test == | ||
− | + | Verify that the provided md5 checksum file matches the computed md5 for your downloaded tarball: | |
+ | |||
+ | <pre> | ||
+ | $ md5sum -c ucvm-12.2.0.tar.gz.md5 | ||
+ | </pre> | ||
Line 44: | Line 66: | ||
For the Euclid Etree library: | For the Euclid Etree library: | ||
− | + | <pre> | |
− | + | $ tar zxvf euclid3-1.3.tar.gz | |
− | + | $ cd ./euclid3-1.3/libsrc | |
+ | $ make all | ||
+ | </pre> | ||
− | If installing the Etree library on a Lustre filesystem (NICS Kraken | + | If installing the Etree library on a Lustre filesystem on a Cray system (NICS Kraken, NCCS Jaguar), you must enable the Cray IOBUF module prior to compiling the library. Also, |
+ | edit the ./libsrc/Makefile to add the compile flags "-DUSE_IOBUF -DUSE_IOBUF_LOCAL_MACROS". | ||
− | For the Proj.4 library: | + | For the Proj.4 library (version 4.7.0): |
− | + | <pre> | |
− | + | $ tar xvf proj-4.7.0.tar | |
− | + | $ cd proj-4.7.0 | |
− | + | $ ./configure --prefix=<your desired install directory> | |
+ | $ make; make install | ||
+ | </pre> | ||
Be sure to note where you installed these two packages. You will need this information later in the installation process. | Be sure to note where you installed these two packages. You will need this information later in the installation process. | ||
Line 62: | Line 89: | ||
For the SCEC CVM-H model: | For the SCEC CVM-H model: | ||
<pre> | <pre> | ||
− | + | $ tar xvf cvmh-11.9.1.tar.gz | |
− | + | $ cd cvmh-11.9.1 | |
− | + | $ ./configure --prefix=<your desired_install directory> | |
− | + | $ make; make install | |
</pre> | </pre> | ||
− | |||
For the SCEC CVM-S model: | For the SCEC CVM-S model: | ||
<pre> | <pre> | ||
− | + | $ tar xvf cvms-11.11.0.tar.gz | |
− | + | $ cd cvms/src | |
− | + | $ make all | |
</pre> | </pre> | ||
− | |||
Again, be sure to note where you installed the community velocity models. | Again, be sure to note where you installed the community velocity models. | ||
Line 89: | Line 114: | ||
UCVM_INSTALL_DIR=<desired UCVM install path> | UCVM_INSTALL_DIR=<desired UCVM install path> | ||
</pre> | </pre> | ||
− | |||
In a bash shell, for example, the ETREE_DIR environment variable can be declared with the command: | In a bash shell, for example, the ETREE_DIR environment variable can be declared with the command: | ||
− | + | <pre> | |
− | + | $ ETREE_DIR=/opt/etree;export ETREE_DIR | |
+ | </pre> | ||
In a csh shell: | In a csh shell: | ||
− | + | <pre> | |
− | + | $ setenv ETREE_DIR /opt/etree | |
+ | </pre> | ||
Now unpack the UCVM software distribution: | Now unpack the UCVM software distribution: | ||
− | + | <pre> | |
+ | $ tar zxvf ucvm-12.2.0.tar.gz | ||
+ | </pre> | ||
The build configuration differs slightly depending on where you wish to run UCVM. On a standard Linux system with gcc, run the following | The build configuration differs slightly depending on where you wish to run UCVM. On a standard Linux system with gcc, run the following | ||
− | commands to | + | commands to configure the software with CVM-H and CVM-S enabled: |
− | |||
<pre> | <pre> | ||
− | + | $ cd ucvm-12.2.0 | |
− | + | $ ./configure --prefix=${UCVM_INSTALL_DIR} --with-etree-include-path="${ETREE_DIR}/libsrc" | |
--with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" | --with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" | ||
--with-proj4-lib-path="${PROJ4_DIR}/lib" --enable-model-cvmh --enable-model-cvms | --with-proj4-lib-path="${PROJ4_DIR}/lib" --enable-model-cvmh --enable-model-cvms | ||
--with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" | --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" | ||
− | --with-gctpc-lib-path="${CVMH_DIR} | + | --with-gctpc-lib-path="${CVMH_DIR}/lib" --with-cvms-include-path="${CVMS_DIR}/include" |
--with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/model" --with-cvms-model-path="${CVMS_DIR}/src" | --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/model" --with-cvms-model-path="${CVMS_DIR}/src" | ||
− | |||
</pre> | </pre> | ||
Line 119: | Line 145: | ||
Note that unless you force static linking with the "--enable-static" command line option, you must update your LD_LIBRARY_PATH environment variable with the paths to the Etree, Proj.4, and model shared libraries. | Note that unless you force static linking with the "--enable-static" command line option, you must update your LD_LIBRARY_PATH environment variable with the paths to the Etree, Proj.4, and model shared libraries. | ||
− | |||
The configure command is quite long, so an example shell script with this command is located in ./scripts/autoconf/basic_install.sh. It assumes that you previously defined the environment variables described in this installation guide. The script can be executed with the following shell commands: | The configure command is quite long, so an example shell script with this command is located in ./scripts/autoconf/basic_install.sh. It assumes that you previously defined the environment variables described in this installation guide. The script can be executed with the following shell commands: | ||
+ | <pre> | ||
+ | $ cd scripts/autoconf | ||
+ | $ ./basic_install.sh | ||
+ | </pre> | ||
+ | Compile the software by executing the following command in the main directory: | ||
<pre> | <pre> | ||
− | + | $ make | |
− | |||
</pre> | </pre> | ||
+ | Check that the package was built correctly by executing the following command in the main directory: | ||
+ | <pre> | ||
+ | $ make check | ||
+ | </pre> | ||
− | + | Verify that all unit tests and acceptance tests pass. Depending on your system and your UCVM configuration, testing may take up to 15-30 minutes. | |
− | + | Install the software to the desired target location by executing the following command in the main directory: | |
+ | <pre> | ||
+ | $ make install | ||
+ | </pre> | ||
− | + | A full list of supported standard velocity models and options can be listed with the command: | |
+ | <pre> | ||
+ | $ ./configure --help | ||
+ | </pre> | ||
− | + | == Updating the Configuration == | |
− | + | The UCVM installer will automatically create a configuration file in ${UCVM_INSTALL_DIR}/conf/ucvm.conf with entries for all enabled velocity models and some standard settings. However, you may wish to edit this file to toggle model flags or to add new user models in the future. For example, the CVM-H model has a number of flags that may be set in the UCVM configuration file. Please see [[UCVM_User_Guide#Framework_Configuration]] for more details. | |
− | |||
− | |||
− | |||
− | |||
− | |||
== Advanced Install == | == Advanced Install == | ||
− | Additional options must be passed to the ./configure script on a more exotic Linux system, such as a host that requires static linking (NICS Kraken), or has a Lustre high-performace filesystem ( | + | Additional options must be passed to the ./configure script on a more exotic Linux system, such as a host that requires static linking (NICS Kraken), or has a Lustre high-performace filesystem (NCCS Jaguar, NICS Kraken). The following example shows how to enable static linking and Cray IOBUF buffering: |
− | + | <pre> | |
− | + | $ module add iobuf | |
− | + | $ ./configure --prefix=${UCVM_INSTALL_DIR} --enable-static --enable-iobuf --enable-model-cvmh --enable-model-cvms | |
− | |||
--with-etree-include-path="${ETREE_DIR}/libsrc" --with-etree-lib-path="${ETREE_DIR}/libsrc" | --with-etree-include-path="${ETREE_DIR}/libsrc" --with-etree-lib-path="${ETREE_DIR}/libsrc" | ||
--with-proj4-include-path="${PROJ4_DIR}/include" --with-proj4-lib-path="${PROJ4_DIR}/lib" | --with-proj4-include-path="${PROJ4_DIR}/include" --with-proj4-lib-path="${PROJ4_DIR}/lib" | ||
--with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" | --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" | ||
− | --with-gctpc-lib-path="${CVMH_DIR} | + | --with-gctpc-lib-path="${CVMH_DIR}/lib" --with-cvms-include-path="${CVMS_DIR}/include" |
− | --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/ | + | --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/model" --with-cvms-model-path="${CVMS_DIR}/src" CC=cc |
− | + | $ make; make check; make install | |
+ | </pre> | ||
+ | |||
+ | <p>It is generally advisable to compile all components of UCVM using the same compilation system. NICS Kraken, for example, uses the Cray compiler wrappers (cc not gcc). As such, on this system, you must use CC to compile the Euclid library.</p> | ||
A few notes: | A few notes: | ||
* Some systems require applications to be linked statically (example above). | * Some systems require applications to be linked statically (example above). | ||
− | * Lustre filesystems require linking with IOBUF module (NICS Kraken example above). | + | * Lustre filesystems may require linking with Cray IOBUF module (NICS Kraken example above). |
* The configure script needs to be told where to find the library file(s), the header file(s), and the model files for each velocity model that you wish to enable. The configure script will check that the libraries and headers are present, and it will setup UCVM with that model enabled and properly configured. | * The configure script needs to be told where to find the library file(s), the header file(s), and the model files for each velocity model that you wish to enable. The configure script will check that the libraries and headers are present, and it will setup UCVM with that model enabled and properly configured. | ||
* The script will attempt to auto-detect a GNU compliant compiler. If an MPI wrapper is found, the MPI version of UCVM is compiled, otherwise the serial version is built. The compiler may be overridden with the option 'CC="mpicc"' for example. | * The script will attempt to auto-detect a GNU compliant compiler. If an MPI wrapper is found, the MPI version of UCVM is compiled, otherwise the serial version is built. The compiler may be overridden with the option 'CC="mpicc"' for example. | ||
Line 170: | Line 206: | ||
<pre> | <pre> | ||
--enable-static enable static linking | --enable-static enable static linking | ||
+ | --enable-netcdf enable netCDF module | ||
--enable-iobuf enable IOBUF module | --enable-iobuf enable IOBUF module | ||
--enable-model-cvmh enable model SCEC CVM-H | --enable-model-cvmh enable model SCEC CVM-H | ||
Line 175: | Line 212: | ||
--enable-model-cencal enable model USGS CenCalVM | --enable-model-cencal enable model USGS CenCalVM | ||
--enable-model-cvmsi enable model SCEC CVM-SI | --enable-model-cvmsi enable model SCEC CVM-SI | ||
+ | --enable-model-cvmnci enable model SCEC CVM-NCI | ||
--enable-model-wfcvm enable model WFCVM | --enable-model-wfcvm enable model WFCVM | ||
+ | --enable-model-cvmlt enable model Lin-Thurber | ||
+ | --enable-model-cmrg enable model CMRG | ||
+ | --enable-model-tape enable model Tape | ||
--with-etree-include-path | --with-etree-include-path | ||
location of the Etree headers | location of the Etree headers | ||
--with-etree-lib-path location of the Etree libraries | --with-etree-lib-path location of the Etree libraries | ||
− | |||
--with-proj4-include-path | --with-proj4-include-path | ||
location of the Proj.4 headers | location of the Proj.4 headers | ||
--with-proj4-lib-path location of the Proj.4 libraries | --with-proj4-lib-path location of the Proj.4 libraries | ||
− | + | --with-netcdf-include-path | |
+ | location of the netCDF headers | ||
+ | --with-netcdf-lib-path location of the netCDF libraries | ||
--with-cencal-include-path | --with-cencal-include-path | ||
location of the USGS CenCalVM headers | location of the USGS CenCalVM headers | ||
Line 192: | Line 234: | ||
--with-cencal-extmodel-path | --with-cencal-extmodel-path | ||
location of the USGS CenCalVM extended etree | location of the USGS CenCalVM extended etree | ||
− | |||
--with-cvmh-include-path | --with-cvmh-include-path | ||
location of the SCEC CVM-H headers | location of the SCEC CVM-H headers | ||
--with-cvmh-lib-path location of the SCEC CVM-H libraries | --with-cvmh-lib-path location of the SCEC CVM-H libraries | ||
--with-cvmh-model-path location of the SCEC CVM-H model files | --with-cvmh-model-path location of the SCEC CVM-H model files | ||
− | --with-gctpc-lib-path location of the CVM-H GCTPC libraries | + | --with-gctpc-lib-path location of the CVM-H GCTPC libraries (CVM-H 11.2.0) |
− | |||
--with-cvms-include-path | --with-cvms-include-path | ||
location of the SCEC CVM-S headers | location of the SCEC CVM-S headers | ||
--with-cvms-lib-path location of the SCEC CVM-S libraries | --with-cvms-lib-path location of the SCEC CVM-S libraries | ||
--with-cvms-model-path location of the SCEC CVM-S model files | --with-cvms-model-path location of the SCEC CVM-S model files | ||
− | |||
--with-cvmsi-include-path | --with-cvmsi-include-path | ||
location of the SCEC CVM-SI headers | location of the SCEC CVM-SI headers | ||
--with-cvmsi-lib-path location of the SCEC CVM-SI libraries | --with-cvmsi-lib-path location of the SCEC CVM-SI libraries | ||
--with-cvmsi-model-path location of the SCEC CVM-SI model files | --with-cvmsi-model-path location of the SCEC CVM-SI model files | ||
− | + | --with-cvmnci-include-path | |
+ | location of the SCEC CVM-NCI headers | ||
+ | --with-cvmnci-lib-path location of the SCEC CVM-NCI libraries | ||
+ | --with-cvmnci-model-path | ||
+ | location of the SCEC CVM-NCI model files | ||
--with-wfcvm-include-path | --with-wfcvm-include-path | ||
location of the WFCVM headers | location of the WFCVM headers | ||
--with-wfcvm-lib-path location of the WFCVM libraries | --with-wfcvm-lib-path location of the WFCVM libraries | ||
--with-wfcvm-model-path location of the WFCVM model files | --with-wfcvm-model-path location of the WFCVM model files | ||
+ | --with-cvmlt-include-path | ||
+ | location of the CVMLT headers | ||
+ | --with-cvmlt-lib-path location of the CVMLT libraries | ||
+ | --with-cvmlt-model-path location of the CVMLT model files | ||
+ | --with-cmrg-include-path | ||
+ | location of the CMRG headers | ||
+ | --with-cmrg-lib-path location of the CMRG libraries | ||
+ | --with-cmrg-model-path location of the CMRG model files | ||
+ | --with-tape-include-path | ||
+ | location of the TAPE headers | ||
+ | --with-tape-lib-path location of the TAPE libraries | ||
+ | --with-tape-model-path location of the TAPE model files | ||
</pre> | </pre> | ||
+ | == Unit and Acceptance Tests == | ||
− | + | Both the unit tests and acceptance tests may be executed with the following command from the main directory: | |
− | + | <pre> | |
− | + | $ make check | |
− | + | </pre> | |
All tests should result in a PASS. | All tests should result in a PASS. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== Trouble-shooting == | == Trouble-shooting == | ||
Line 242: | Line 290: | ||
This indicates that UCVM was linked against one or more shared libraries and the dynamic library loader cannot find the actual .so library at run-time. The solution is to update your LD_LIBRARY_PATH to include the directory containing the library mentioned in the error message. For example, the following command adds a new search directory to LD_LIBRARY_PATH in a csh shell: | This indicates that UCVM was linked against one or more shared libraries and the dynamic library loader cannot find the actual .so library at run-time. The solution is to update your LD_LIBRARY_PATH to include the directory containing the library mentioned in the error message. For example, the following command adds a new search directory to LD_LIBRARY_PATH in a csh shell: | ||
+ | <pre> | ||
+ | $ setenv LD_LIBRARY_PATH /home/USER/opt/somepkg/lib:${LD_LIBRARY_PATH} | ||
+ | </pre> | ||
− | + | Alternatively, if you do not want to work with shared libraries, UCVM can be compiled statically with the "--enable-static" configuration option. | |
+ | |||
+ | === Proj. 4 Error: major axis or radius = 0 or not given === | ||
+ | |||
+ | On systems with home filesystems that are not viewable to compute nodes (such as NICS Kraken), you may encounter errors with Proj.4 when trying to run any component of UCVM on compute nodes. This is due to the fact that Proj.4 actually relies on a file called proj_defs.dat which is located in the ${MAKE_INSTALL_LOCATION}/share/proj directory. So for example, suppose you had configured Proj.4 with ./configure --prefix=/not_viewable_to_compute_nodes/proj-4.7.0, Proj. 4 would then search for proj_defs.dat in /not_viewable_to_compute_nodes/proj-4.7.0/share/proj/proj_defs.dat. This will cause UCVM to throw the error "Proj.4 Error: major axis or radius = 0 or not given" and your job will fail. | ||
+ | |||
+ | The only way it seems to solve this issue is to actually make sure your --prefix directory is actually visible to the compute nodes and do a make install there. Documentation suggests that you can set the PROJ_LIB environment variable, however this seems to not work correctly without modifications to the Proj.4 source code. | ||
+ | |||
+ | === Building Against CVM-H 11.2.0 === | ||
+ | |||
+ | UCVM is designed to work with CVM-H 11.9.1 by default, but it can optionally be built with CVM-H 11.2.0. Use this modified configure command to configure the package with CVM-H 11.2.0 and CVM-S: | ||
− | + | <pre> | |
+ | $ cd ucvm-12.2.0 | ||
+ | $ ./configure --prefix=${UCVM_INSTALL_DIR} --with-etree-include-path="${ETREE_DIR}/libsrc" | ||
+ | --with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" | ||
+ | --with-proj4-lib-path="${PROJ4_DIR}/lib" --enable-model-cvmh --enable-model-cvms | ||
+ | --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" | ||
+ | --with-gctpc-lib-path="${CVMH_DIR}/gctpc/lib" --with-cvms-include-path="${CVMS_DIR}/include" | ||
+ | --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/bin" --with-cvms-model-path="${CVMS_DIR}/src" | ||
+ | CFLAGS="-D_UCVM_MODEL_CVMH_11_2_0" | ||
+ | </pre> | ||
+ | Note that the CVM-H model and gctpc paths have changed, and there is now a CFLAGS argument. After configuring, you may compile, check, and install normally. | ||
= Framework Configuration = | = Framework Configuration = | ||
Line 261: | Line 332: | ||
# Change to reflect UCVM installation directory. | # Change to reflect UCVM installation directory. | ||
# | # | ||
− | + | ucvm_interface=map_etree | |
+ | ucvm_mappath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/ucvm/ucvm.e | ||
Line 270: | Line 342: | ||
# | # | ||
# SCEC CVM-H | # SCEC CVM-H | ||
− | cvmh_modelpath=/home/scec-00/USER/opt/aftershock/cvmh/ | + | cvmh_modelpath=/home/scec-00/USER/opt/aftershock/cvmh-11.9.1/model |
# | # | ||
# USGS Bay Area high-rez and extended etrees | # USGS Bay Area high-rez and extended etrees | ||
Line 277: | Line 349: | ||
# | # | ||
# SCEC CVM-SI | # SCEC CVM-SI | ||
− | cvmsi_modelpath=/home/scec-00/USER/opt/aftershock/cvmsi/model | + | cvmsi_modelpath=/home/scec-00/USER/opt/aftershock/cvmsi/model/i8 |
+ | # | ||
+ | # SCEC CVM-NCI | ||
+ | cvmnci_modelpath=/home/scec-00/USER/opt/aftershock/cvmnci/model/i2 | ||
# | # | ||
# Wasatch Front CVM | # Wasatch Front CVM | ||
Line 283: | Line 358: | ||
# | # | ||
# Lin-Thurber Statewide | # Lin-Thurber Statewide | ||
− | + | cvmlt_modelpath=/home/scec-00/USER/opt/aftershock/cvmlt/model | |
# | # | ||
# Cape Mendocino RG | # Cape Mendocino RG | ||
− | cmrg_modelpath=/home/scec-00/USER/opt/aftershock/ucvm-11. | + | cmrg_modelpath=/home/scec-00/USER/opt/aftershock/cvm-cmrg/model/cmrg.conf |
+ | # | ||
+ | # Tape SoCal Model | ||
+ | tape_modelpath=/home/scec-00/USER/opt/aftershock/cvm-tape/model/m16 | ||
+ | # | ||
+ | # 1D | ||
+ | 1d_modelpath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/1d/1d.conf | ||
+ | # | ||
+ | # 1D GTL | ||
+ | 1dgtl_modelpath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/1d/1d.conf | ||
Line 309: | Line 393: | ||
# Optional Yong-Wald vs30 map | # Optional Yong-Wald vs30 map | ||
yong_interface=map_etree | yong_interface=map_etree | ||
− | yong_mappath=/home/scec-00/USER/opt/aftershock/ucvm-11. | + | yong_mappath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/ucvm/ucvm_yong_wald.e |
# Model flags | # Model flags | ||
# | # | ||
− | + | cvmh_param=USE_1D_BKG,True | |
− | + | cvmh_param=USE_GTL,True | |
</pre> | </pre> | ||
Line 328: | Line 412: | ||
! Description | ! Description | ||
|- | |- | ||
− | | | + | | ucvm_interface |
− | | ${UCVM_INSTALL_DIR}/model/ucvm | + | | map_etree |
− | | | + | | The interface code needed to parse the UCVM map |
+ | |- | ||
+ | | ucvm_mappath | ||
+ | | ${UCVM_INSTALL_DIR}/model/ucvm/ucvm.e | ||
+ | | String identifying location of the UCVM map Etree | ||
|- | |- | ||
| *_modelpath | | *_modelpath | ||
Line 379: | Line 467: | ||
| cencal | | cencal | ||
| USGS Bay Area CenCalVM | | USGS Bay Area CenCalVM | ||
+ | | External, installed separately and linked at install | ||
+ | | 1 | ||
+ | |- | ||
+ | | cvmnci | ||
+ | | SCEC CVM-NCI | ||
| External, installed separately and linked at install | | External, installed separately and linked at install | ||
| 1 | | 1 | ||
Line 397: | Line 490: | ||
| 1 | | 1 | ||
|- | |- | ||
− | | | + | | lt |
| Lin-Thurber California Statewide | | Lin-Thurber California Statewide | ||
− | | | + | | External, installed separately and linked at install |
| 1 | | 1 | ||
|- | |- | ||
| cmrg | | cmrg | ||
| Cape Mendocino Rob Graves | | Cape Mendocino Rob Graves | ||
− | | | + | | External, installed separately and linked at install |
+ | | 1 | ||
+ | |- | ||
+ | | tape | ||
+ | | Carl Tape SoCal | ||
+ | | External, installed separately and linked at install | ||
| 1 | | 1 | ||
|- | |- | ||
Line 411: | Line 509: | ||
| Internal, included in UCVM, user specifies etree path in ./conf/ucvm.conf | | Internal, included in UCVM, user specifies etree path in ./conf/ucvm.conf | ||
| 1 | | 1 | ||
− | |||
|} | |} | ||
Line 445: | Line 542: | ||
|- | |- | ||
| elygtl | | elygtl | ||
− | | Ely Vs30-derived GTL | + | | Ely Vs30-derived GTL (Ely et al., 2010) |
|- | |- | ||
| 1dgtl | | 1dgtl | ||
| Generic 1D, identical to 1d crustal model | | Generic 1D, identical to 1d crustal model | ||
|} | |} | ||
+ | |||
+ | |||
+ | Interpolation functions are used to smooth GTL material properties with the underlying crustal model material properties. This smoothing is performed over an interpolation zone along the Z axis. Interpolation functions can be assigned on a per-GTL basis. Two predefined interpolation functions are provided to the user: | ||
+ | |||
+ | {| class="wikitable" border="1" | ||
+ | |- | ||
+ | ! Label | ||
+ | ! Interpolation Function | ||
+ | |- | ||
+ | | linear | ||
+ | | Linear interpolation | ||
+ | |- | ||
+ | | ely | ||
+ | | Ely interpolation relation (Ely et al., 2010) | ||
+ | |} | ||
+ | |||
+ | |||
+ | If the user enables a GTL model but does not specify an interpolation function, linear interpolation is used by default. If no interpolation zone is specified, a depth range of 0 m - 350m is used. | ||
Line 480: | Line 595: | ||
| map_etree | | map_etree | ||
| SCEC Map Etree | | SCEC Map Etree | ||
+ | |} | ||
+ | |||
+ | |||
+ | == Supported Model Flags == | ||
+ | |||
+ | Underlying models may be configured in one of two ways: with a key/value string in the UCVM configuration file, or by passing configuration parameters with the <code>ucvm_setparam()</code>function. The following model flags are supported: | ||
+ | |||
+ | {| class="wikitable" border="1" | ||
+ | |- | ||
+ | ! Parameter | ||
+ | ! Value | ||
+ | ! Models | ||
+ | |- | ||
+ | | USE_1D_BKG | ||
+ | | True/False (default False) | ||
+ | | CVM-H | ||
+ | |- | ||
+ | | USE_GTL | ||
+ | | True/False (default True) | ||
+ | | CVM-H | ||
|} | |} | ||
Line 520: | Line 655: | ||
− | For other models that either are represented by tables, or have their own native API, UCVM may be extended to query from these models. However, this | + | For other models that either are represented by tables, or have their own native API, UCVM may be extended to query from these models. However, this requires code modifications to the core UCVM library located in ./src/ucvm and recompilation. The general steps are: |
* Define new model label UCVM_MODEL_MODELNAME in ucvm_dtypes.h | * Define new model label UCVM_MODEL_MODELNAME in ucvm_dtypes.h | ||
Line 527: | Line 662: | ||
** Add ucvm_model_modelname.h include at top | ** Add ucvm_model_modelname.h include at top | ||
** Add model lookup to ucvm_add_model() function | ** Add model lookup to ucvm_add_model() function | ||
+ | ** Add model to ucvm_get_resources() function | ||
* Modify ./src/ucvm/Makefile.am: | * Modify ./src/ucvm/Makefile.am: | ||
** Add ucvm_model_modelname.o to list of libucvm.a dependencies | ** Add ucvm_model_modelname.o to list of libucvm.a dependencies | ||
− | * | + | * Modify ./configure.ac: |
− | * | + | ** Define "enable-modelname", "with-modelname-lib-path", and "with-modelname-incl-path" configure options |
− | + | ** Add library and header checks for your new model | |
− | + | * Modify installer to customize path to your new model in ucvm.conf: | |
+ | ** Add your new model to ./conf/Makefile.am | ||
+ | ** Add config entry for your new model to ./conf/ucvm_template.conf | ||
+ | * Regenerate the Makefiles with ./scripts/autoconf/reconf.sh | ||
+ | * Reconfigure and recompile UCVM, making sure to pass your new model flags to the configure utility | ||
Line 544: | Line 684: | ||
User-defined maps can be created with the ./bin/grd2etree utility. | User-defined maps can be created with the ./bin/grd2etree utility. | ||
− | |||
= Framework Description = | = Framework Description = | ||
Line 553: | Line 692: | ||
UCVM combines multiple regional velocity models, along with a DEM map and a Vs30 map, into one composite model for the purposes of querying surface elevation, Vs30, Vp, Vs, and density. Individual model setup and query details are abstracted under a uniform interface, and the application sees only the composite model. This simplifies application code and makes it easier for those programs to support new velocity models in the future. | UCVM combines multiple regional velocity models, along with a DEM map and a Vs30 map, into one composite model for the purposes of querying surface elevation, Vs30, Vp, Vs, and density. Individual model setup and query details are abstracted under a uniform interface, and the application sees only the composite model. This simplifies application code and makes it easier for those programs to support new velocity models in the future. | ||
+ | |||
+ | [[File:UCVM_Tiling_Concept.png|256px|thumb|right|UCVM model tiling method.]] | ||
Velocity models are combined into the composite by tiling them on top of one another. When UCVM is initialized, the user selects an ordered list of models to query for data. Query points are then submitted to each velocity model in that list, one model at a time in list order. The first model to return valid velocity data for the point is considered to have fullfilled that data request and subsequent models are not queried. Generally, no smoothing is performed at the interfaces between models (an exception is interpolation between a GTL and crustal model as described below). For each query point, the following data is returned: surface elevation, Vs30, Vp, Vs, and density. The figure at right illustrates how this tiling is performed. | Velocity models are combined into the composite by tiling them on top of one another. When UCVM is initialized, the user selects an ordered list of models to query for data. Query points are then submitted to each velocity model in that list, one model at a time in list order. The first model to return valid velocity data for the point is considered to have fullfilled that data request and subsequent models are not queried. Generally, no smoothing is performed at the interfaces between models (an exception is interpolation between a GTL and crustal model as described below). For each query point, the following data is returned: surface elevation, Vs30, Vp, Vs, and density. The figure at right illustrates how this tiling is performed. | ||
− | |||
− | |||
Regional models may include: CVM-H, CVM-H, USGS Bay Area, 1D, or other user-defined models. Most models and maps have a maximum extent, outside of which no data is available. The exception is the generic 1D model as it has infinite extents. Applications must check the returned velocity and density values to ensure they are valid. | Regional models may include: CVM-H, CVM-H, USGS Bay Area, 1D, or other user-defined models. Most models and maps have a maximum extent, outside of which no data is available. The exception is the generic 1D model as it has infinite extents. Applications must check the returned velocity and density values to ensure they are valid. | ||
Line 565: | Line 704: | ||
The API distinguishes between two types of regional velocity models: GTL models, and crustal models. GTL models are shallow near-surface velocity models called geotechnical layers. Crustal models are deeper velocity models such as tomographic models. When a user selects to query from one or more crustal models only, the API executes the previously described behavior. However, if one or more GTL models are selected, additional processing is performed. | The API distinguishes between two types of regional velocity models: GTL models, and crustal models. GTL models are shallow near-surface velocity models called geotechnical layers. Crustal models are deeper velocity models such as tomographic models. When a user selects to query from one or more crustal models only, the API executes the previously described behavior. However, if one or more GTL models are selected, additional processing is performed. | ||
− | When a GTL model is given in the ordered list of models, the API creates a default interpolation zone along the z-axis over which interpolation between this GTL and the underlying crustal models is performed. It also assigns a default (linear) interpolation function to smooth the velocities of the GTL and underlying crustal models. Both the interpolation zone and interpolation function can be changed by the user. Querying for a point in this operating mode is now slightly different from the previous crustal-only case. As before, all query points are submitted to the crustal models in the ordered list of models (skipping any GTLs). However, the z coordinate of the points may be shifted down to the lower edge of the interpolation zone if the point falls within it. The models are queried with this possibly modified 3D coordinate and the velocities/density values are saved. The same list of query points is then submitted to the GTL models in the ordered list of models (skipping any crustals) and the z coordinate of those points within the interpolation zone may be shifted up to the top edge. The models are queried with this possibly modified 3D coordinate and the velocity/density values are saved. The list of query points is then traversed one final time. Those points that fall above the interpolation zone will have their velocities/densities set to those of the GTL as processed by the interpolation function assigned to that GTL. Those points that fall within the interpolation zone and have valid GTL/crustal values will have their GTL/crustal values smoothed according to the assigned interpolation function. Points that fall below the interpolation zone and have valid crustal values will have their velocities/densities set to the crustal values. Note that points with invalid/incompatible velocities in either the GTL or crustal dataset cannot be smoothed and the interpolation results will be marked as invalid. | + | When a GTL model is given in the ordered list of models, the API creates a default interpolation zone along the z-axis over which interpolation between this GTL and the underlying crustal models is performed. It also assigns a default (linear) interpolation function to smooth the velocities of the GTL and underlying crustal models. Both the interpolation zone and interpolation function can be changed by the user. |
+ | |||
+ | Querying for a point in this operating mode is now slightly different from the previous crustal-only case. As before, all query points are submitted to the crustal models in the ordered list of models (skipping any GTLs). However, the z coordinate of the points may be shifted down to the lower edge of the interpolation zone if the point falls within it. The crustal models are queried with this possibly modified 3D coordinate and the velocities/density values are saved. The same list of query points is then submitted to the GTL models in the ordered list of models (skipping any crustals) and the z coordinate of those points within the interpolation zone may be shifted up to the top edge. The GTL models are queried with this possibly modified 3D coordinate and the velocity/density values are saved. | ||
+ | |||
+ | The list of query points is then traversed one final time. Those points that fall above the interpolation zone will have their velocities/densities set to those of the GTL as processed by the interpolation function assigned to that GTL. Those points that fall within the interpolation zone and have valid GTL/crustal values will have their GTL/crustal values smoothed according to the assigned interpolation function. Points that fall below the interpolation zone and have valid crustal values will have their velocities/densities set to the crustal values. Note that points with invalid/incompatible velocities in either the GTL or crustal dataset cannot be smoothed and the interpolation results will be marked as invalid. | ||
Line 606: | Line 749: | ||
- If running interactively, type Cntl-D to end input coord list. | - If running interactively, type Cntl-D to end input coord list. | ||
− | Version: | + | Version: 12.2.0 |
</pre> | </pre> | ||
Line 620: | Line 763: | ||
* The Etree data structure is efficient and the library provides disk caching and buffering, resulting in very fast query performance. | * The Etree data structure is efficient and the library provides disk caching and buffering, resulting in very fast query performance. | ||
− | + | Two versions of this utility are provided, a serial version and an MPI version. | |
+ | |||
+ | |||
+ | ==== Serial ucvm2etree ==== | ||
+ | |||
+ | The serial ucvm2etree is suitable for small to medium sized etrees, and is executed as follows: | ||
<pre> | <pre> | ||
Usage: ucvm2etree [-h] -f config | Usage: ucvm2etree [-h] -f config | ||
Line 628: | Line 776: | ||
-h: Help message | -h: Help message | ||
− | Version: | + | Version: 12.2.0 |
</pre> | </pre> | ||
+ | |||
The following is an example ucvm2etree configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/ucvm2etree_example.conf: | The following is an example ucvm2etree configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/ucvm2etree_example.conf: | ||
<pre> | <pre> | ||
− | # | + | # Domain corners coordinates (clockwise, degrees): |
+ | proj=geo-bilinear | ||
+ | lon_0=-119.288842 | ||
+ | lat_0=34.120549 | ||
+ | |||
+ | lon_1=-118.354016 | ||
+ | lat_1=35.061096 | ||
+ | |||
+ | lon_2=-116.846030 | ||
+ | lat_2=34.025873 | ||
− | + | lon_3=-117.780976 | |
− | + | lat_3=33.096503 | |
− | |||
− | |||
− | |||
# Domain dimensions (meters): | # Domain dimensions (meters): | ||
− | x-size= | + | x-size=180000.0000 |
− | y-size= | + | y-size=135000.0000 |
− | z-size= | + | z-size=61875.0000 |
# Blocks partition parameters: | # Blocks partition parameters: | ||
nx=32 | nx=32 | ||
− | ny= | + | ny=24 |
− | # Max freq | + | # Max freq, points per wavelength, Vs min |
max_freq=0.5 | max_freq=0.5 | ||
ppwl=4.0 | ppwl=4.0 | ||
+ | vs_min=200.0 | ||
− | # | + | # Max allowed size of octants in meters |
− | + | max_octsize=10000.0 | |
# Etree parameters and info | # Etree parameters and info | ||
− | title= | + | title=ChinoHills_0.5Hz_200ms |
author=P_Small | author=P_Small | ||
date=05/2011 | date=05/2011 | ||
− | outputfile=./ | + | outputfile=./cmu_cvmh_chino_0.5hz_200ms.e |
format=etree | format=etree | ||
− | # | + | # UCVM parameters |
− | + | #ucvmstr=cvms | |
− | ucvmconf=../../conf/ | + | ucvmstr=cvmh |
+ | ucvm_interp_zrange=0.0,350.0 | ||
+ | ucvmconf=../../conf/kraken/ucvm.conf | ||
− | # | + | # Scratch |
− | + | scratch=/lustre/scratch/patricks/scratch | |
− | # | + | # |
− | + | # Buffering parameters used by MPI version only | |
+ | # | ||
+ | # Etree buffer size in MB | ||
+ | buf_etree_cache=128 | ||
+ | # Max octants to buffer for flat file during extraction | ||
+ | buf_extract_mem_max_oct=4194304 | ||
+ | # Max octants to save in flat file before reporting full during extraction | ||
+ | buf_extract_ffile_max_oct=16000000 | ||
+ | # Max octants to read from input flat file during sorting | ||
+ | buf_sort_ffile_max_oct=20000000 | ||
+ | # Minimum number of octants between reports during merging | ||
+ | buf_merge_report_min_oct=10000000 | ||
+ | # MPI send/recv octant buffer size during merging | ||
+ | buf_merge_sendrecv_buf_oct=4096 | ||
+ | # Etree read/write octant buffer size during merging | ||
+ | buf_merge_io_buf_oct=4194304 | ||
</pre> | </pre> | ||
+ | |||
+ | |||
+ | ==== Parallel ucvm2etree ==== | ||
+ | |||
+ | The MPI version is useful for large and very large etrees. It involves three programs: ucvm2etree-extract-MPI, ucvm2etree-sort-MPI, and ucvm2etree-merge-MPI. All three take the same configuration file as the one used by the serial version. The ucvmetree-extract-MPI utility generates the set of points that are to be used in the etree, extracts those points from the specified CVMs, and saves them in a set of N files (where N is the process count). The ucvm2etree-sort-MPI utility sorts the points in each flat file by octant key. The third utility, ucvm2etree-merge-MPI, merges these locally sorted points into the final etree file. | ||
Import the new CVM Etree into UCVM by adding these lines to ${UCVM_INSTALL_DIR}/conf/ucvm.conf: | Import the new CVM Etree into UCVM by adding these lines to ${UCVM_INSTALL_DIR}/conf/ucvm.conf: | ||
Line 687: | Line 865: | ||
</pre> | </pre> | ||
+ | The following sections describe these three programs in more detail: | ||
+ | |||
+ | ===== ucvm2etree-extract-MPI ===== | ||
+ | |||
+ | Divides the etree region into C columns for extraction. This is an embarrassingly parallel operation. A dispatcher (rank 0) farms out each column to a worker in a pool of N cores for extraction. Each worker queries UCVM for the points in its column and writes a flat-file formatted etree. After program execution, there are N sub-etree files, each locally unsorted. The extractor must be run on 2^Y + 1 cores where Y>0 and (2^Y) < C. The output flat file format is a list of octants(24 byte addr, 16 byte key, 12 byte payload) in arbitrary Z-order. | ||
+ | |||
+ | Since the number of points in a column depends on the minimum Vs values within that column, some columns will have high octant counts and others will have very low octant counts. Having sub-etrees that vary greatly in size is not optimal for the sorting operations that follow, so ucvm2etree-extract-MPI implements a simple octant balancing mechanism. When a worker has extracted more than X octants (the default 16M octants), it reports to the dispatcher that it cannot extract any more columns and terminates. This strategy approximately balances the sub-etrees so that they may be loaded into memory by ucvm2etree-sort-MPI. In the case of very large extractions where the dispatcher reports that all workers have reported they are full yet columns remain to be extracted, increase the job size by a factor of 2 until there is room for all the columns. | ||
+ | |||
+ | |||
+ | ===== ucvm2etree-sort-MPI ===== | ||
+ | |||
+ | Sorts the sub-etrees produced by ucvm2etree-extract-MPI so that each file is in local pre-order (Z-order). Again, the is an embarrassingly parallel operation. Each rank in the job reads in one of the sub-etrees produced by the previous program, sorts the octants in Z-order, and writes the sorted octants to a new sub-etree. The sorter must be run on 2^Y cores where Y>0. The worker pool must be large enough to allow each worker to load all the octants from its assigned file into memory. By default, this octant limit is 20M octants. If a rank reports that the size of the sub-etree exceeds memory capacity, the 20M buffer size constant may be increased if memory allows, or alternatively, cvmbycols-extract-MPI may be rerun with a larger job size to reduce the number of octants per file. | ||
+ | |||
+ | |||
+ | ===== ucvm2etree-merge-MPI ===== | ||
+ | |||
+ | Merges N locally sorted etrees in flat file format into a final, compacted etree. This is essentially a merge sort on the keys from the addresses read from the local files. The cores at the lowest level of the merge tree each read in octants from two flat files in pre-order, merge sort the two sets of addresses, then pass the locally sorted list of addresses to a parent node for additional merging. This proceeds until the points rise to rank 1 which has a completely sorted list of etree addresses. Rank 0 takes this sorted list and performs a transactional append on the final Etree. The merger must be run on 2^N cores. | ||
+ | |||
+ | The program reads in input files that are in flat file format. In can output a merged Etree in either Etree format or flat file format. Although, due to space considerations, it strips the output flat file format to a pre-order list ot octants(16 byte key, 12 byte payload). The missing addr field is redundant and can be regenerated from the key field. | ||
+ | |||
+ | |||
+ | ===== MPI Version Buffer Values ===== | ||
+ | |||
+ | There are a number of buffer values in the ucvm.conf file that must be accurately set for extraction, sorting, and merging to take place. Please see below for a list of those values and a description of the purpose of each one. | ||
+ | |||
+ | '''buf_etree_cache''' | ||
+ | |||
+ | Units: MB<br /> | ||
+ | Recommended Value: 128 usually works well, you can increase or decrease as need be<br /> | ||
+ | For Process: ucvm2etree-extract-MPI<br /> | ||
+ | Description: Used in the merging process during the etree_open call. According to the documentation of etree_open, this is more precisely defined as the "internal buffer space being allocated in megabytes". The value defined represents the memory buffer for etree traversing, finding, writing, etc. | ||
+ | |||
+ | '''buf_extract_mem_max_oct''' | ||
+ | |||
+ | Units: Number of octants<br /> | ||
+ | Recommended Value: 2097152<br /> | ||
+ | For Process: ucvm2etree-extract-MPI<br /> | ||
+ | Description: Used in the extraction process as the number of bytes to buffer before writing to disk. It reserves, in memory, a placeholder for this number of octants. The size of each octant is 52 bytes and as such, the actual memory allocation is 52 * this value. | ||
+ | |||
+ | '''buf_extract_ffile_max_oct''' | ||
+ | |||
+ | Units: Number of octants<br /> | ||
+ | Recommended Value: Varies. It must be greater than total output file size divided by the number of processes.<br /> | ||
+ | For Process: ucvm2etree-extract-MPI<br /> | ||
+ | Description: This value is ''very critical''! This defines the maximum output file size that each worker process can write. If this is not large enough, the worker will halt and return a "worker is full message". The best way to set this is to guess how large your extraction file might be (a relatively detailed Chino Hills e-tree, for example, is 250GB, and divide that number by 52 * the number or processes you have. So if I was building that mesh, on 32 processes, this value would need to be at least 162 million. However, this value also ties in with the number below, so '''please read it carefully as well'''. | ||
+ | |||
+ | '''buf_sort_ffile_max_oct''' | ||
+ | |||
+ | Units: Number of octants<br /> | ||
+ | Recommended Value: Must be at least equal to buf_extract_ffile_max_oct<br /> | ||
+ | For Process: ucvm2etree-sort-MPI<br /> | ||
+ | Description: Used in the sorting process to determine the max number of octants to read into memory. Therefore, for each process running on the server, there must be number of processes * 52 * buf_sort_ffile_max_oct bytes of memory available. So in our example above, if we were running those 32 processes on 16 dual core machines, we would need 162 million * 52 * 2 cores bytes of RAM or about 16 GB. All sorting is done in memory, not using the file on the disk. | ||
+ | |||
+ | '''buf_merge_report_min_oct''' | ||
+ | |||
+ | Units: Number of octants<br /> | ||
+ | Recommended Value: Personal preference - default is 10000000.<br /> | ||
+ | For Process: ucvm2etree-merge-MPI<br /> | ||
+ | Description:. This defines the number of octants that must be processed between progress reports in the merge process. This progress reports take the form of "Appended [at least buf_merge_report_min_oct] in N.NNs seconds". If you would like more frequent updates, set this value to be smaller. | ||
+ | |||
+ | '''buf_merge_sendrecv_buf_oct''' | ||
+ | |||
+ | Units: Bytes<br /> | ||
+ | Recommended Value: 4096<br /> | ||
+ | For Process: ucvm2etree-merge-MPI<br /> | ||
+ | Description: The number of bytes to merge per loop during the merge process. It is recommended to keep this value as is. | ||
+ | |||
+ | '''buf_merge_io_buf_oct''' | ||
+ | |||
+ | Units: Number of octants<br /> | ||
+ | Recommended Value: 4194304<br /> | ||
+ | For Process: ucvm2etree-merge-MPI<br /> | ||
+ | Description: Number of octants to read in and store in memory for merging. After being merged, the memory is then flushed to disk and the process repeats. Since this is a value in octants, it will require the machine to have at least buf_merge_io_buf_oct * 52 bytes of memory. | ||
+ | <br /> | ||
+ | <br /> | ||
+ | |||
+ | ==== Etree Summary ==== | ||
+ | |||
+ | In order to understand how ucvm2etree works, some knowledge of the Etree data structure is required. The fundamental unit of measure in an Etree is the tick. This is a distance expressed in meters, and is derived from the length of the longest side of the geographic region: | ||
+ | <pre> | ||
+ | tick size = (max(len x, len y, len z) / 2^31) m | ||
+ | </pre> | ||
+ | In essence, the geographic region is divided up into a set of 2^31 ticks. | ||
+ | |||
+ | The funamental unit of data in an Etree is the octant. An octant contains an address (location code) and a payload (material properties). Octants are organized into a tree structure called an octree, where each node has up to eight children. Each level in this octree represents an octant size (alternatively: edge size, or resolution). Moving from level N to level N+1 in the octree increases resolution by 2x. Moving from level N to level N-1 reduces resolution by 2x. The resolution for a level can be determined with: | ||
+ | |||
+ | <pre> | ||
+ | res from level N = (max_len / 2^N) m | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | The number of ticks in an octant of level N is: | ||
+ | |||
+ | <pre> | ||
+ | ticks for level N = 2^N | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | And thus the octant size in level N is: | ||
+ | |||
+ | <pre> | ||
+ | octant size = (2^N * (max(len x, len y, len z) / 2^31)) m | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | Given a particular resolution, the appropriate level to represent it in the etree can be determined with: | ||
+ | |||
+ | <pre> | ||
+ | level N from res = (int)(log(max_len/res)/log(2)) + 1 (rounded up to next integer) | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | These relations are important in understanding the algorithm described in the following sections. | ||
+ | |||
+ | |||
+ | ===== Algorithm Description ===== | ||
+ | |||
+ | This program uses several algorithmic approaches to increase extraction speed and minimize the amount of disk space required to store the Etree. These are described in the following sections. | ||
+ | |||
+ | |||
+ | ====== Gridding of Geographic Coordinates ====== | ||
+ | |||
+ | Gridding of the geographic bounding box is performed either using bilinear interpolation of lat/long corners, or by a UCVM projection specification (proj4 projection string, rot angle, lat0, lon0). Grid points are located at cell center. | ||
+ | |||
+ | |||
+ | ====== Optimized Etree Resolution Based on Frequency, PPWL, and Minimum Vs ====== | ||
+ | |||
+ | One of the strengths of the Etree (octree) representation is that resolution can be adjusted so that a particular region is sampled at a lower/higher interval than surrounding areas. You are able to achieve high resolution in just those places where it is needed, and the rest of the region can be sampled at a much lower rez thereby saving considerable disk space. | ||
+ | |||
+ | With this flexibility comes the difficulty of determining what resolution is needed at all points within a meshing region to support Computational Seismology simulations. ucvm2etree uses an algorithm to deduce this resolution based off of three input parameters: maximum frequency to support, the number of points per wavelength that is desired, and the minimum Vs to support. The following relation is used to derive the resolution for a region: | ||
+ | |||
+ | <pre> | ||
+ | res = vs_min / (ppwl * max_freq) | ||
+ | |||
+ | where: | ||
+ | res: resolution for the region with vs_min, in meters | ||
+ | vs_min: Minimum Vs found in region, in meters/sec | ||
+ | ppwl: desired points per wavelength | ||
+ | max_freq: maximum frequency to support, in Hz | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | Using the above relation, the 3D space is decomposed into small columns, and the resolution in each column is tailored to the vs values found at each depth. The algorithm can be summarized as follows: | ||
+ | * Divide region in set of x,y columns, where the length of each column equals its width. | ||
+ | * For each column: | ||
+ | ** Starting at depth = 0m, extract a 2D grid for the column at the resolution needed to support the max frequency, pounts/wavelength, and minimum Vs, with a depth that is one-half the current octant edge size (that is, Z axis is cell center as well). | ||
+ | *** If the local minimum Vs is in this 2D grid allows for a lower resolution, and the current z value is divisible by the lower resolution, requery the CVM at the lower resolution. Otherwise, if the local min Vs in this grid needs a higher resolution, increase the resolution to what is required (up to the maximum resolution dictated by the configured global minimum Vs) and requery the CVM. | ||
+ | *** Write this layer to a flat file (a single file when run serially, or a file local to this core when run in parallel) | ||
+ | *** Step the depth down by the resolution of this 2D grid and loop to the next layer in the column. | ||
+ | ** Loop to the next column | ||
+ | |||
+ | |||
+ | The ucvm2etree software enforces a minimum and maximum resolution for octants. Given a certain set of input parameters, these resolutions can be derived as follows: | ||
+ | <pre> | ||
+ | If vs_min = 200 m/s, max_freq = 4.0 Hz, and ppwl = 8, max_dim_length = 180000.0 m, max_cellsize = 500.0 m, then | ||
+ | |||
+ | Maximum Resolution: | ||
+ | res = 200.0 / (8.0 * 4.0) = 6.25 m (minimum resolution needed to support Vs=200 m/s) | ||
+ | N = (log(180000.0/6.25)/log(2)) + 1 = 15 | ||
+ | max_res = 180000.0 / 2^15 ticks ~= 5.49 m | ||
+ | |||
+ | Therefore, maximum resolution (minimum edge size) for these parameters is 5.49 m. | ||
+ | |||
+ | Minimum Resolution: | ||
+ | res = 500.0 m (the maximum cell size to support) | ||
+ | N = (log(180000.0/500.0)/log(2)) + 1 = 9 | ||
+ | min_res = 180000.0 / 2^9 ticks ~= 351.56 m | ||
+ | |||
+ | Therefore, minimum resolution (maximum edge size) for these parameters is 351.56 m. | ||
+ | </pre> | ||
+ | * Minimum resolution represents a floor on the possible resolution of an etree. This is protection against very large octants from being formed by high Vs values. | ||
+ | |||
+ | |||
+ | ====== High-performance Etree merging and compaction ====== | ||
+ | |||
+ | Generation of an Etree in a parallel environment is challenging since the Euclid etree interface only allows serial write access to the etree file. Points may only be inserted, or they may be appended in key order, by a single writer. This makes etee construction and compaction time consuming. To circumvent this issue, a high performance etree merge/compaction algorithm has been developed. | ||
+ | |||
+ | Simply put, it is essentially a merge sort performed on 2^N small flat files that contain all of the points needed for the final etree. The flat files contain material properties previously extracted from UCVM. Once these small, local flat files exist on disk, an MPI job of 2^N cores is then able to sort these octants by their keys in O(n*log n) time. Once all the octants are sorted, a single core may then open the final etree/flat file for writing in append mode, and simply stream the sorted octants to disk efficiently. | ||
+ | |||
+ | Conceptually, the cores in this 2^N MPI job are organized into a tree-like structure with each node containing two children. The bottom layer of this tree has nodes that each do a pre-order traversal on two flat files. They sort the combined octants read from each etree in pre-order, and then pass them up to a parent node. The nodes from the top layer to the second to bottom layer each receive locally sorted lists of octants from their two children and once again sort the combined list of octants in pre-order. This newly sorted list is passed upward in this manner until the root node of the tree receives two locally sorted halves of the entire set of etree points. It then sorts them, and appends the fully sorted stream of octants to the final file. | ||
+ | |||
+ | |||
+ | === ucvm2mesh === | ||
+ | |||
+ | This utility creates a 3D regular mesh extracted from a set of CVMs. The CVM Etree may then be used in a wave propagation simulation such as AWP-ODC. It can produce meshes in a number of formats | ||
+ | |||
+ | * AWP 3 value: float Vp, float Vs, float Rho | ||
+ | * AWP 5 value: float Vp, float Vs, float Rho, float Qp, float Qs | ||
+ | * AWP 8 vlaue: int i, int j, int k, float Vp, float Vs, float Rho, float Qp, float Qs | ||
+ | * SORD: One file each for Vp, Vs, Rho (double precision) | ||
+ | * netCDF (with optional netCDF package installed, and the mesh2netcdf utility) | ||
+ | |||
+ | |||
+ | Two versions of this utility are provided, a serial version and an MPI version. | ||
+ | |||
+ | |||
+ | ==== Serial ucvm2mesh ==== | ||
+ | |||
+ | The serial ucvm2etree is suitable for small to medium sized meshes, and is executed as follows: | ||
+ | <pre> | ||
+ | Usage: ucvm2mesh [-h] -f config | ||
+ | |||
+ | Flags: | ||
+ | -f: Configuration file | ||
+ | -h: Help message | ||
+ | |||
+ | Version: 12.2.0 | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | The following is an example ucvm2mesh configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/ucvm2mesh_example.conf: | ||
+ | |||
+ | <pre> | ||
+ | # List of CVMs to query | ||
+ | ucvmlist=cvmh | ||
+ | |||
+ | # UCVM conf file | ||
+ | ucvmconf=../conf/kraken/ucvm.conf | ||
+ | |||
+ | # Gridding cell centered or vertex | ||
+ | gridtype=CENTER | ||
+ | |||
+ | # Spacing of cells | ||
+ | spacing=2000.0 | ||
+ | |||
+ | # Projection | ||
+ | proj=+proj=utm +datum=WGS84 +zone=11 | ||
+ | rot=-40.0 | ||
+ | x0=-122.3 | ||
+ | y0=34.7835 | ||
+ | z0=0.0 | ||
+ | |||
+ | # Number of cells along each dim | ||
+ | nx=384 | ||
+ | ny=248 | ||
+ | nz=25 | ||
+ | |||
+ | # Partitioning of grid among processors | ||
+ | px=2 | ||
+ | py=2 | ||
+ | pz=5 | ||
+ | |||
+ | # Vs/Vp minimum | ||
+ | vp_min=0 | ||
+ | vs_min=0 | ||
+ | |||
+ | # Mesh and grid files, format | ||
+ | meshfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.media | ||
+ | gridfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.grid | ||
+ | meshtype=IJK-12 | ||
+ | |||
+ | # Location of scratch dir | ||
+ | scratch=/lustre/scratch/user/scratch | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | ==== Parallel ucvm2mesh ==== | ||
+ | |||
+ | The MPI version, ucvm2mesh-mpi, is useful for large and very large meshes. It accepts the same configuration file as the serial version. It provides an additional command-line option for stage-out of the mesh files from the locations specified in the conf file to a final directory location. This is useful on HPC systems that use temporary scratch space. | ||
+ | <pre> | ||
+ | Usage: ucvm2mesh-mpi [-h] [-o dir] -f config | ||
+ | |||
+ | Flags: | ||
+ | -f: Configuration file | ||
+ | -o: Stage-out directory | ||
+ | -h: Help message | ||
+ | |||
+ | Version: 12.2.0 | ||
+ | </pre> | ||
+ | |||
+ | ==== Performance Enhancements for ucvm2mesh on Lustre Filesystems ==== | ||
+ | |||
+ | When running ucvm2mesh on Lustre filesystems (and other systems with similar structures), you may experience some performance issues that result in less than optimal performance. By default, Lustre uses one OST or disk per file. As such, running ucvm2mesh-mpi with a significant number of cores results in all those processes attempting to seek and write to the same file on the same OST/disk. This can result in a disk bottleneck that gives very poor performance, where it may take up to 10 minutes to write a few hundred MBs. | ||
+ | |||
+ | This can be fixed by increasing the number of OSTs/disks to which the output directory and files are written. This is called striping. Generally, you will want to set the stripes for the output directory to be either the recommended amount for large files or the maximum amount allowed. The command to set the number of stripes on Kraken is lfs setstripe [dir] --count [count]. Count can be a fixed number or it can be -1 for the maximum supported by the operating system. The Kraken documentation suggests that 80 is a reasonable amount for very large files. So, for example, suppose we were generating a large mesh with our stage out directory being /lustre/scratch/username/meshes/outstripe. We would first make that directory and then run: | ||
+ | |||
+ | lfs setstripe /lustre/scratch/username/meshes/outstripe --count 80 | ||
+ | |||
+ | Then we would have our configuration file output to that directory: | ||
+ | |||
+ | meshfile=/lustre/scratch/username/meshes/outstripe/USC_cvms_awp.media | ||
+ | gridfile=/lustre/scratch/username/meshes/outstripe/USC_cvms_awp.media.grid | ||
+ | |||
+ | When we qsub the job, it will automatically take advantage of striping and distribute the load across multiple targets. | ||
+ | |||
+ | It is also important to remember that more cores is not always better. Additional cores can result in increased write requests that may cause performance decreases. It is important to find the optimal balance between the number of processes writing to the file and the number of stripes. For exampe, for a large 600km x 400km x 40km meshes, a core count of 600 and stripe count of 80 on NICS Kraken will result in reasonably good performance. | ||
=== grd_query === | === grd_query === | ||
Line 711: | Line 1,175: | ||
- If running interactively, type Cntl-D to end input coord list. | - If running interactively, type Cntl-D to end input coord list. | ||
− | Version: | + | Version: 12.2.0 |
</pre> | </pre> | ||
Line 726: | Line 1,190: | ||
-h: Help message | -h: Help message | ||
− | Version: | + | Version: 12.2.0 |
</pre> | </pre> | ||
Line 785: | Line 1,249: | ||
-h: Help message | -h: Help message | ||
− | Version: | + | Version: 12.2.0 |
</pre> | </pre> | ||
Line 899: | Line 1,363: | ||
/* Query underlying models */ | /* Query underlying models */ | ||
int ucvm_query(int n, ucvm_point_t *pnt, ucvm_data_t *data); | int ucvm_query(int n, ucvm_point_t *pnt, ucvm_data_t *data); | ||
+ | |||
+ | /* Get info on installed models */ | ||
+ | int ucvm_install_info(char *info, int len); | ||
#endif | #endif | ||
Line 906: | Line 1,373: | ||
* Initialization (<code>ucvm_init()</code>) | * Initialization (<code>ucvm_init()</code>) | ||
* Add models in order of preference using <code>ucvm_add_model()</code>. These may be either crustal or GTL models. See below for specific details. | * Add models in order of preference using <code>ucvm_add_model()</code>. These may be either crustal or GTL models. See below for specific details. | ||
+ | * Set any models-specific flags using <code>ucvm_setparam</code>. These will override any flags specified in ucvm.conf. For example, to enable the 1D background model with CVM-H through the API, execute the <code>ucvm_setparam()</code> function as follows: | ||
+ | <pre> | ||
+ | ucvm_setparam(UCVM_PARAM_MODEL_CONF, UCVM_MODEL_CVMH, "USE_1D_BKG", "True"); | ||
+ | </pre> | ||
* If GTLs have been added: | * If GTLs have been added: | ||
** Associate an interpolation function with each using <code>ucvm_assoc_ifunc()</code>. Otherwise, linear interpolation will be used. | ** Associate an interpolation function with each using <code>ucvm_assoc_ifunc()</code>. Otherwise, linear interpolation will be used. | ||
Line 932: | Line 1,403: | ||
</pre> | </pre> | ||
* The programmer may also define their own models and integrate them into UCVM. A user model interface is defined by creating the init(), finalize(), getversion(), getlabel(), setparam(), and query() functions for the model interface, and populating the <code>ucvm_model_t</code> structure with these function pointers and some additional record keeping information. Configuration information for a specific model instance of that interface is placed in a <code>ucvm_modelconf_t</code> structure. Once a model is fully described with a <code>ucvm_model_t</code> interface and a <code>ucvm_modelconf_t</code> configuration, it can be registered for use with the <code>ucvm_add_user_model()</code> function and queried with the <code>ucvm_query()</code> function. | * The programmer may also define their own models and integrate them into UCVM. A user model interface is defined by creating the init(), finalize(), getversion(), getlabel(), setparam(), and query() functions for the model interface, and populating the <code>ucvm_model_t</code> structure with these function pointers and some additional record keeping information. Configuration information for a specific model instance of that interface is placed in a <code>ucvm_modelconf_t</code> structure. Once a model is fully described with a <code>ucvm_model_t</code> interface and a <code>ucvm_modelconf_t</code> configuration, it can be registered for use with the <code>ucvm_add_user_model()</code> function and queried with the <code>ucvm_query()</code> function. | ||
+ | |||
=== Data Type Descriptions === | === Data Type Descriptions === | ||
Line 989: | Line 1,461: | ||
ucvm_data_t *data); | ucvm_data_t *data); | ||
} ucvm_model_t; | } ucvm_model_t; | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
</pre> | </pre> | ||
Line 1,076: | Line 1,531: | ||
-118.0000 34.0000 100.000 295.679 390.000 cvmh 2371.500 727.748 2051.320 none 0.000 0.000 0.000 crust 2371.500 727.748 2051.320 | -118.0000 34.0000 100.000 295.679 390.000 cvmh 2371.500 727.748 2051.320 none 0.000 0.000 0.000 crust 2371.500 727.748 2051.320 | ||
-118.0000 34.0000 1000.000 295.679 390.000 cvmh 2616.063 1082.813 2127.783 none 0.000 0.000 0.000 crust 2616.063 1082.813 2127.783 | -118.0000 34.0000 1000.000 295.679 390.000 cvmh 2616.063 1082.813 2127.783 none 0.000 0.000 0.000 crust 2616.063 1082.813 2127.783 | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | === Additional Examples === | ||
+ | |||
+ | The following example directs ucvm_query to query the Ely GTL and CVM-S crustal model by depth, using the Ely interpolation function to smooth the values. The Z-axis interpolation zone is set to 0-400m. | ||
+ | |||
+ | <pre> | ||
+ | % ./ucvm_query -m elygtl:ely,cvms -c gd -z 0.0,400.0 -f ../conf/ucvm.conf | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | The following example queries the USGS Bay Area (CenCalVM) and CVM-H models by elevation, tiling them into a composite model (where CenCalVM has priority). | ||
+ | |||
+ | <pre> | ||
+ | % ./ucvm_query -m cencal,cvmh -c ge -f ../conf/ucvm.conf | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | The following example queries the 1D model by depth, with surface elevation and Vs30 values taken from the alternate Yong-Wald Vs30 map. | ||
+ | |||
+ | <pre> | ||
+ | % ./ucvm_query -m 1d -p yong -c gd -f ../conf/ucvm.conf | ||
</pre> | </pre> | ||
Line 1,094: | Line 1,572: | ||
LDFLAGS = $(LDFLAGS) -L$(UCVM_INSTALL_DIR)/lib ucvm -L($CVMS_INSTALL_DIR)/lib cvms -L$(ETREE_INSTALL_DIR)/libsrc etree -L$(PROJ4_INSTALL_DIR)/lib proj | LDFLAGS = $(LDFLAGS) -L$(UCVM_INSTALL_DIR)/lib ucvm -L($CVMS_INSTALL_DIR)/lib cvms -L$(ETREE_INSTALL_DIR)/libsrc etree -L$(PROJ4_INSTALL_DIR)/lib proj | ||
-lgfortran -pthreads -lm | -lgfortran -pthreads -lm | ||
+ | |||
+ | |||
+ | === Example Programs === | ||
+ | |||
+ | Please see the ./examples directory of the software distribution for compiled examples that employ the C API. | ||
= History of UCVM Releases = | = History of UCVM Releases = | ||
+ | |||
+ | * 12.2.0 (Feb 9, 2012): Initial release | ||
Line 1,231: | Line 1,716: | ||
#'''Wald, D. J., and T. I. Allen (2007)''', Topographic slope as a proxy for seismic site conditions and amplification, Bull. Seism. Soc. Am., 97 (5), 1379-1395, doi:10.1785/0120060267. | #'''Wald, D. J., and T. I. Allen (2007)''', Topographic slope as a proxy for seismic site conditions and amplification, Bull. Seism. Soc. Am., 97 (5), 1379-1395, doi:10.1785/0120060267. | ||
#'''Wills, C. J., and K. B. Clahan (2006)''', Developing a map of geologically defined site-condition categories for California, Bull. Seism. Soc. Am., 96 (4A), 1483-1501, doi:10.1785/0120050179. | #'''Wills, C. J., and K. B. Clahan (2006)''', Developing a map of geologically defined site-condition categories for California, Bull. Seism. Soc. Am., 96 (4A), 1483-1501, doi:10.1785/0120050179. | ||
− | #'''Yong, A., Hough, S.E., Iwahashi, J., and A. Braverman (2012)''', A terrain-based site conditions map of California with implications for the contiguous United States, Bull. Seism. Soc. Am., | + | #'''Yong, A., Hough, S.E., Iwahashi, J., and A. Braverman (2012)''', A terrain-based site conditions map of California with implications for the contiguous United States, Bull. Seism. Soc. Am., Vol. 102, No. 1, pp. 114–128, February 2012, doi: 10.1785/0120100262. |
Latest revision as of 23:52, 9 June 2014
Redirect to:
Contents
- 1 Overview
- 2 Requirements
- 3 Installation
- 4 Framework Configuration
- 5 Framework Description
- 5.1 Concept of Operation
- 5.2 Querying Models Via Native Interface versus Querying UCVM
- 5.3 Utilities
- 5.4 C Application Programming Interface
- 6 Extracting Values from UCVM
- 7 History of UCVM Releases
- 8 Acknowledgements and Contact Info
- 9 Technical Notes
- 10 References
- 11 License and Disclaimer
Overview
The Unified Community Velocity Model Framework UCVM is a software package which provides a standard interface to several regional community velocity models. Any differences in coordinate projections within each model are hidden so that the researcher queries any of the models by longitude, latitude, and Z (where Z is depth from surface or elevation relative MSL). UCVM also provides several other unique capabilities and products, including:
- Seamlessly combine two or more models into a composite model for querying
- Optionally include a California statewide geotechnical layer and interpolate it with the crustal velocity models
- Extract a 3D mesh or CVM Etree (octree) of material properties from any combination of models
- Standard California statewide elevation and Vs30 data map is provided
- Miscellaneous tools are provided for creating 2D etree maps, and optimizing etrees
- Numerically smooth discontinuities at the interfaces between different regional models
- Add support for future velocity models with the extendable interface
Current list of supported velocity models: SCEC CVM-H, SCEC CVM-S, SCEC CVM-SI, SCEC CVM-NCI, Magistrale WFCVM, USGS CenCalVM, Graves Cape Mendocino, Lin-Thurber Statewide, Tape SoCal.
The API itself is written in the C language. The software is MPI-capable, and has been shown to scale to 3300+ cores on NICS Kraken in a mesh generator. It allows application programs to tailor which set of velocity models are linked in to minimize memory footprint. It is also compatible with grid resources that have data stage-in policies for jobs, and light-weight kernels that support static libraries only. This package has been tested on NCCS Jaguar, NICS Kraken, and USC HPCC.
Requirements
The system requirements are as follows:
- UNIX operating system (Linux)
- GNU gcc/gfortran compilers (MPI wrappers such as mpicc are OK)
- tar for opening the compressed files
- Euclid Etree library (recommend euclid3-1.3.tar.gz)
- Proj.4 Projection Library (recommend proj-4.7.0.tar.gz)
Optional dependencies include any of the following standard velocity models and packages:
- Standard community velocity models: SCEC CVM-H, SCEC CVM-S, SCEC CVM-SI, SCEC CVM-NCI, Magistrale WFCVM, USGS CenCalVM, Graves Cape Mendocino, Lin-Thurber Statewide, Tape SoCal
- NetCDF (network Common Data Form) library NetCDF
Installation
Download
- Start at SCEC website: http://scec.usc.edu/scecpedia/UCVM
- Navigate to the Downloads section of the UCVM web page.
- Click the download link to download the latest source code distribution file and any listed dependencies. This is typically posted as a tar and gzipped file (tgz format). The source code distribution file is large (350 Mb), so the download make take awhile.
CheckSum Test
Verify that the provided md5 checksum file matches the computed md5 for your downloaded tarball:
$ md5sum -c ucvm-12.2.0.tar.gz.md5
Basic Install
The general steps for installing UCVM are:
- Install Etree and Proj.4 packages
- Install one or more velocity models
- Define environment variables describing where Etree/Proj.4/models are located
- Run the UCVM installer
First, install the Etree and Proj.4 packages which are required by UCVM. The INSTALL/README instructions in each distribution describe how to install them,
but generally:
For the Euclid Etree library:
$ tar zxvf euclid3-1.3.tar.gz $ cd ./euclid3-1.3/libsrc $ make all
If installing the Etree library on a Lustre filesystem on a Cray system (NICS Kraken, NCCS Jaguar), you must enable the Cray IOBUF module prior to compiling the library. Also, edit the ./libsrc/Makefile to add the compile flags "-DUSE_IOBUF -DUSE_IOBUF_LOCAL_MACROS".
For the Proj.4 library (version 4.7.0):
$ tar xvf proj-4.7.0.tar $ cd proj-4.7.0 $ ./configure --prefix=<your desired install directory> $ make; make install
Be sure to note where you installed these two packages. You will need this information later in the installation process.
Then, install the velocity models that you wish to use with UCVM according to the documentation provided by those models. Ensure they are compiled with gcc/gfortran.
For the SCEC CVM-H model:
$ tar xvf cvmh-11.9.1.tar.gz $ cd cvmh-11.9.1 $ ./configure --prefix=<your desired_install directory> $ make; make install
For the SCEC CVM-S model:
$ tar xvf cvms-11.11.0.tar.gz $ cd cvms/src $ make all
Again, be sure to note where you installed the community velocity models.
Declare environment variables with the paths to the Etree library, Proj.4 library, and velocity models that you previously installed. Also, update your LD_LIBRARY_PATH environment variable to include the new library directories.
ETREE_DIR=<Etree install path> PROJ4_DIR=<Proj.4 install path> CVMH_DIR=<CVM-H install path> CVMS_DIR=<CVM-S install path> LD_LIBRARY_PATH=${ETREE_DIR}/libsrc:${PROJ4_DIR}/lib:${CVMH_DIR}/lib:${CVMS_DIR}/lib:${LD_LIBRARY_PATH} UCVM_INSTALL_DIR=<desired UCVM install path>
In a bash shell, for example, the ETREE_DIR environment variable can be declared with the command:
$ ETREE_DIR=/opt/etree;export ETREE_DIR
In a csh shell:
$ setenv ETREE_DIR /opt/etree
Now unpack the UCVM software distribution:
$ tar zxvf ucvm-12.2.0.tar.gz
The build configuration differs slightly depending on where you wish to run UCVM. On a standard Linux system with gcc, run the following commands to configure the software with CVM-H and CVM-S enabled:
$ cd ucvm-12.2.0 $ ./configure --prefix=${UCVM_INSTALL_DIR} --with-etree-include-path="${ETREE_DIR}/libsrc" --with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" --with-proj4-lib-path="${PROJ4_DIR}/lib" --enable-model-cvmh --enable-model-cvms --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" --with-gctpc-lib-path="${CVMH_DIR}/lib" --with-cvms-include-path="${CVMS_DIR}/include" --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/model" --with-cvms-model-path="${CVMS_DIR}/src"
The above example enables CVM-H and CVM-S but any of the following models may be enabled: SCEC CVM-H, SCEC CVM-S, SCEC CVM-SI, USGS CenCalVM, and WFCVM. For each velocity model you want to use, the configure script needs to be told to explicitly use that model (with the --enable-model-* flag) and where to find the velocity model's library files (.a or .so), header files (.h), and model files so that they can be linked into UCVM.
Note that unless you force static linking with the "--enable-static" command line option, you must update your LD_LIBRARY_PATH environment variable with the paths to the Etree, Proj.4, and model shared libraries.
The configure command is quite long, so an example shell script with this command is located in ./scripts/autoconf/basic_install.sh. It assumes that you previously defined the environment variables described in this installation guide. The script can be executed with the following shell commands:
$ cd scripts/autoconf $ ./basic_install.sh
Compile the software by executing the following command in the main directory:
$ make
Check that the package was built correctly by executing the following command in the main directory:
$ make check
Verify that all unit tests and acceptance tests pass. Depending on your system and your UCVM configuration, testing may take up to 15-30 minutes.
Install the software to the desired target location by executing the following command in the main directory:
$ make install
A full list of supported standard velocity models and options can be listed with the command:
$ ./configure --help
Updating the Configuration
The UCVM installer will automatically create a configuration file in ${UCVM_INSTALL_DIR}/conf/ucvm.conf with entries for all enabled velocity models and some standard settings. However, you may wish to edit this file to toggle model flags or to add new user models in the future. For example, the CVM-H model has a number of flags that may be set in the UCVM configuration file. Please see UCVM_User_Guide#Framework_Configuration for more details.
Advanced Install
Additional options must be passed to the ./configure script on a more exotic Linux system, such as a host that requires static linking (NICS Kraken), or has a Lustre high-performace filesystem (NCCS Jaguar, NICS Kraken). The following example shows how to enable static linking and Cray IOBUF buffering:
$ module add iobuf $ ./configure --prefix=${UCVM_INSTALL_DIR} --enable-static --enable-iobuf --enable-model-cvmh --enable-model-cvms --with-etree-include-path="${ETREE_DIR}/libsrc" --with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" --with-proj4-lib-path="${PROJ4_DIR}/lib" --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" --with-gctpc-lib-path="${CVMH_DIR}/lib" --with-cvms-include-path="${CVMS_DIR}/include" --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/model" --with-cvms-model-path="${CVMS_DIR}/src" CC=cc $ make; make check; make install
It is generally advisable to compile all components of UCVM using the same compilation system. NICS Kraken, for example, uses the Cray compiler wrappers (cc not gcc). As such, on this system, you must use CC to compile the Euclid library.
A few notes:
- Some systems require applications to be linked statically (example above).
- Lustre filesystems may require linking with Cray IOBUF module (NICS Kraken example above).
- The configure script needs to be told where to find the library file(s), the header file(s), and the model files for each velocity model that you wish to enable. The configure script will check that the libraries and headers are present, and it will setup UCVM with that model enabled and properly configured.
- The script will attempt to auto-detect a GNU compliant compiler. If an MPI wrapper is found, the MPI version of UCVM is compiled, otherwise the serial version is built. The compiler may be overridden with the option 'CC="mpicc"' for example.
- Additional compiler flags and linker flags may be passed to ./configure with the CFLAGS and LDFLAGS options. Generally, this is not needed but allows the package to be built on platforms with specific compiling/linking requirements.
The full list of options follows:
--enable-static enable static linking --enable-netcdf enable netCDF module --enable-iobuf enable IOBUF module --enable-model-cvmh enable model SCEC CVM-H --enable-model-cvms enable model SCEC CVM-S --enable-model-cencal enable model USGS CenCalVM --enable-model-cvmsi enable model SCEC CVM-SI --enable-model-cvmnci enable model SCEC CVM-NCI --enable-model-wfcvm enable model WFCVM --enable-model-cvmlt enable model Lin-Thurber --enable-model-cmrg enable model CMRG --enable-model-tape enable model Tape --with-etree-include-path location of the Etree headers --with-etree-lib-path location of the Etree libraries --with-proj4-include-path location of the Proj.4 headers --with-proj4-lib-path location of the Proj.4 libraries --with-netcdf-include-path location of the netCDF headers --with-netcdf-lib-path location of the netCDF libraries --with-cencal-include-path location of the USGS CenCalVM headers --with-cencal-lib-path location of the USGS CenCalVM libraries --with-cencal-model-path location of the USGS CenCalVM high-rez etree --with-cencal-extmodel-path location of the USGS CenCalVM extended etree --with-cvmh-include-path location of the SCEC CVM-H headers --with-cvmh-lib-path location of the SCEC CVM-H libraries --with-cvmh-model-path location of the SCEC CVM-H model files --with-gctpc-lib-path location of the CVM-H GCTPC libraries (CVM-H 11.2.0) --with-cvms-include-path location of the SCEC CVM-S headers --with-cvms-lib-path location of the SCEC CVM-S libraries --with-cvms-model-path location of the SCEC CVM-S model files --with-cvmsi-include-path location of the SCEC CVM-SI headers --with-cvmsi-lib-path location of the SCEC CVM-SI libraries --with-cvmsi-model-path location of the SCEC CVM-SI model files --with-cvmnci-include-path location of the SCEC CVM-NCI headers --with-cvmnci-lib-path location of the SCEC CVM-NCI libraries --with-cvmnci-model-path location of the SCEC CVM-NCI model files --with-wfcvm-include-path location of the WFCVM headers --with-wfcvm-lib-path location of the WFCVM libraries --with-wfcvm-model-path location of the WFCVM model files --with-cvmlt-include-path location of the CVMLT headers --with-cvmlt-lib-path location of the CVMLT libraries --with-cvmlt-model-path location of the CVMLT model files --with-cmrg-include-path location of the CMRG headers --with-cmrg-lib-path location of the CMRG libraries --with-cmrg-model-path location of the CMRG model files --with-tape-include-path location of the TAPE headers --with-tape-lib-path location of the TAPE libraries --with-tape-model-path location of the TAPE model files
Unit and Acceptance Tests
Both the unit tests and acceptance tests may be executed with the following command from the main directory:
$ make check
All tests should result in a PASS.
Trouble-shooting
If you see an error similar to the following while running either the tests or the UCVM programs:
error while loading shared libraries: libsomelib.so: cannot open shared object file: No such file or directory
This indicates that UCVM was linked against one or more shared libraries and the dynamic library loader cannot find the actual .so library at run-time. The solution is to update your LD_LIBRARY_PATH to include the directory containing the library mentioned in the error message. For example, the following command adds a new search directory to LD_LIBRARY_PATH in a csh shell:
$ setenv LD_LIBRARY_PATH /home/USER/opt/somepkg/lib:${LD_LIBRARY_PATH}
Alternatively, if you do not want to work with shared libraries, UCVM can be compiled statically with the "--enable-static" configuration option.
Proj. 4 Error: major axis or radius = 0 or not given
On systems with home filesystems that are not viewable to compute nodes (such as NICS Kraken), you may encounter errors with Proj.4 when trying to run any component of UCVM on compute nodes. This is due to the fact that Proj.4 actually relies on a file called proj_defs.dat which is located in the ${MAKE_INSTALL_LOCATION}/share/proj directory. So for example, suppose you had configured Proj.4 with ./configure --prefix=/not_viewable_to_compute_nodes/proj-4.7.0, Proj. 4 would then search for proj_defs.dat in /not_viewable_to_compute_nodes/proj-4.7.0/share/proj/proj_defs.dat. This will cause UCVM to throw the error "Proj.4 Error: major axis or radius = 0 or not given" and your job will fail.
The only way it seems to solve this issue is to actually make sure your --prefix directory is actually visible to the compute nodes and do a make install there. Documentation suggests that you can set the PROJ_LIB environment variable, however this seems to not work correctly without modifications to the Proj.4 source code.
Building Against CVM-H 11.2.0
UCVM is designed to work with CVM-H 11.9.1 by default, but it can optionally be built with CVM-H 11.2.0. Use this modified configure command to configure the package with CVM-H 11.2.0 and CVM-S:
$ cd ucvm-12.2.0 $ ./configure --prefix=${UCVM_INSTALL_DIR} --with-etree-include-path="${ETREE_DIR}/libsrc" --with-etree-lib-path="${ETREE_DIR}/libsrc" --with-proj4-include-path="${PROJ4_DIR}/include" --with-proj4-lib-path="${PROJ4_DIR}/lib" --enable-model-cvmh --enable-model-cvms --with-cvmh-include-path="${CVMH_DIR}/include" --with-cvmh-lib-path="${CVMH_DIR}/lib" --with-gctpc-lib-path="${CVMH_DIR}/gctpc/lib" --with-cvms-include-path="${CVMS_DIR}/include" --with-cvms-lib-path="${CVMS_DIR}/lib" --with-cvmh-model-path="${CVMH_DIR}/bin" --with-cvms-model-path="${CVMS_DIR}/src" CFLAGS="-D_UCVM_MODEL_CVMH_11_2_0"
Note that the CVM-H model and gctpc paths have changed, and there is now a CFLAGS argument. After configuring, you may compile, check, and install normally.
Framework Configuration
The main package configuration file is ${UCVM_INSTALL_DIR}/conf/ucvm.conf. This is where the paths to all configured models and maps are specified, as well as any model flags are defined. The UCVM installer sets up this file automatically. Yet there are a number of situations where you will want to modify it, such as to add a new model.
The following is an example configuration:
# UCVM config file # UCVM model path # # Change to reflect UCVM installation directory. # ucvm_interface=map_etree ucvm_mappath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/ucvm/ucvm.e # Pre-defined models # # SCEC CVM-S cvms_modelpath=/home/scec-00/USER/opt/aftershock/cvms/src # # SCEC CVM-H cvmh_modelpath=/home/scec-00/USER/opt/aftershock/cvmh-11.9.1/model # # USGS Bay Area high-rez and extended etrees cencal_modelpath=/home/scec-00/USER/opt/etree/USGSBayAreaVM-08.3.0.etree cencal_extmodelpath=/home/scec-00/USER/opt/etree/USGSBayAreaVMExt-08.3.0.etree # # SCEC CVM-SI cvmsi_modelpath=/home/scec-00/USER/opt/aftershock/cvmsi/model/i8 # # SCEC CVM-NCI cvmnci_modelpath=/home/scec-00/USER/opt/aftershock/cvmnci/model/i2 # # Wasatch Front CVM wfcvm_modelpath=/home/scec-00/USER/opt/aftershock/wfcvm/src # # Lin-Thurber Statewide cvmlt_modelpath=/home/scec-00/USER/opt/aftershock/cvmlt/model # # Cape Mendocino RG cmrg_modelpath=/home/scec-00/USER/opt/aftershock/cvm-cmrg/model/cmrg.conf # # Tape SoCal Model tape_modelpath=/home/scec-00/USER/opt/aftershock/cvm-tape/model/m16 # # 1D 1d_modelpath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/1d/1d.conf # # 1D GTL 1dgtl_modelpath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/1d/1d.conf # User-defined models # # Change to reflect paths to user model files # # Current supported interfaces: model_etree, model_patch # #user1_interface=model_etree #user1_modelpath=/home/username/model_etree/user1.etree #user2_interface=model_patch #user2_modelpath=/home/username/model_patch/user2.conf # User-defined maps # # Change to reflect paths to user map files # # Current supported interfaces: map_etree # # Optional Yong-Wald vs30 map yong_interface=map_etree yong_mappath=/home/scec-00/USER/opt/aftershock/ucvm-11.11.0_RC/model/ucvm/ucvm_yong_wald.e # Model flags # cvmh_param=USE_1D_BKG,True cvmh_param=USE_GTL,True
The following table describes each configuration item, where "*" indicates the model/map label. Model/map labels can be predefined strings such
as "cmvh" for CVM-H, or any arbitrary string label for the case of a user-defined model.
Key | Value(s) | Description |
---|---|---|
ucvm_interface | map_etree | The interface code needed to parse the UCVM map |
ucvm_mappath | ${UCVM_INSTALL_DIR}/model/ucvm/ucvm.e | String identifying location of the UCVM map Etree |
*_modelpath | A directory, URL, or file | String identifying location of model's primary or high-rez data |
*_extmodelpath | A directory, URL, or file | String identifying location of model's extended data (required for USGS CenCalVM extended Etree). |
*_interface | Models: model_etree, model_patch Maps: model_map |
The interface code needed to parse this user-defined model/map. |
*_mappath | A directory, URL, or file | String identifying location of user-defined map Etree |
*_param | Flag,Value | Set a model-specific configuration Flag to Value. The defined flags are USE_1D_BKG and USE_GTL for CVM-H, for toggling the 1D background model and GTL. |
Supported Standard Crustal Velocity Models
UCVM has built-in support for a number of standard community velocity models (CVMs). These standard models are referenced in ucvm_query and the API by their string labels. The following table lists the labels for these predefined CVMs. These are reserved labels and cannot be used for a user-defined model.
Note that "predefined" does not necessarily mean "installed". These are models that UCVM can query if it is linked with that model's libraries at install time. Some are pre-linked, such as lin-thurber and cmrg, the rest must be explicitly linked at install.
Label | Model | Type | Active Limit |
---|---|---|---|
cvmh | SCEC CVM-H | External, installed separately and linked at install | 1 |
cvms | SCEC CVM-S | External, installed separately and linked at install | 1 |
cencal | USGS Bay Area CenCalVM | External, installed separately and linked at install | 1 |
cvmnci | SCEC CVM-NCI | External, installed separately and linked at install | 1 |
cvmsi | SCEC CVM-SI | External, installed separately and linked at install | 1 |
wfcvm | Wasatch Front CVM (Utah) | External, installed separately and linked at install | 1 |
1d | Hadley-Kanamori 1D | Internal, included in UCVM and automatically linked | 1 |
lt | Lin-Thurber California Statewide | External, installed separately and linked at install | 1 |
cmrg | Cape Mendocino Rob Graves | External, installed separately and linked at install | 1 |
tape | Carl Tape SoCal | External, installed separately and linked at install | 1 |
cmuetree | CMU CVM Etree | Internal, included in UCVM, user specifies etree path in ./conf/ucvm.conf | 1 |
UCVM also defines two "model interfaces". A model interface is a specific file format convention for representing a velocity model. Any velocity model written to conform to one of these model interfaces may be easily imported and queried from UCVM. In addition, many such models may be active simultaneously, whereas only one version of a predefined model may be active at any given time. The two model interfaces are: SCEC CVM Etrees, and SCEC Patch Models. SCEC CVM Etrees employ a special schema and metadata format. SCEC Patch Models have a special conf file and binary file format. The section UCVM_User_Guide#Adding_a_User-defined_Velocity_Model discusses how to create and import these velocity models.
The currently supported model interfaces are summarized in the following table:
Interface | Description | Active Limit |
---|---|---|
model_etree | SCEC CVM Etree | 100 |
model_patch | SCEC Patch Model | 100 |
Supported Standard GTL Velocity Models
UCVM has built-in support for two geotechnical layer models. These near-surface models are intended to provide superior velocity information at shallow depths. The following table lists the labels for these predefined GTLs. These are reserved labels and cannot be used for a user-defined GTL.
Label | GTL |
---|---|
elygtl | Ely Vs30-derived GTL (Ely et al., 2010) |
1dgtl | Generic 1D, identical to 1d crustal model |
Interpolation functions are used to smooth GTL material properties with the underlying crustal model material properties. This smoothing is performed over an interpolation zone along the Z axis. Interpolation functions can be assigned on a per-GTL basis. Two predefined interpolation functions are provided to the user:
Label | Interpolation Function |
---|---|
linear | Linear interpolation |
ely | Ely interpolation relation (Ely et al., 2010) |
If the user enables a GTL model but does not specify an interpolation function, linear interpolation is used by default. If no interpolation zone is specified, a depth range of 0 m - 350m is used.
Supported Standard Maps
UCVM has built-in support for a number of standard maps for California. These standard maps contain elevation data (DEM) and Vs30 data for the region and are referenced in ucvm_query and the API by their string labels. The following table lists the labels for these predefined maps. These are reserved labels and cannot be used for a user-defined map.
Label | Map |
---|---|
ucvm | USGS NED 1 arcsec DEM, and Wills-Wald Vs30 (default) |
yong | USGS NED 1 arcsec DEM, and Yong-Wald Vs30 |
There is also a special map interface that supports reading SCEC Map Etrees. These are maps that conform to a special Etree schema and metadata format convention. The user may create their own maps in this format and import them into UCVM. Any number of maps may be defined, but only one may be active at any time. Maps are defined by the user in the UCVM configuration file (see UCVM_User_Guide#Adding_a_User-defined_Map).
The map interfaces are summarized in the following table:
Interface | Description |
---|---|
map_etree | SCEC Map Etree |
Supported Model Flags
Underlying models may be configured in one of two ways: with a key/value string in the UCVM configuration file, or by passing configuration parameters with the ucvm_setparam()
function. The following model flags are supported:
Parameter | Value | Models |
---|---|---|
USE_1D_BKG | True/False (default False) | CVM-H |
USE_GTL | True/False (default True) | CVM-H |
Adding New Velocity Models
Enabling Another Standard Velocity Model After Installation
If after installation you wish to link in another standard velocity model, you must re-install UCVM using the process described in UCVM_User_Guide#Installation. Add the configure options appropriate for the new model you wish to link into UCVM.
Upgrading the Version of a Standard Velocity Model After Installation
Generally, UCVM must be recompiled and re-installed using the new model libraries and headers. However, if you are certain that only the model files have changed (and the query interface has NOT changed), you can simply update that model's modelpath configuration in ${UCVM_INSTALL_DIR}/conf/ucvm.conf to reference the new files.
Using Multiple Versions of a Supported Velocity Model Simultaneously
Not supported.
Adding a User-defined Velocity Model
UCVM may be extended to support any user-defined velocity model. The simplest way to add a new model is to store that model as a SCEC Etree or Patch Model, and update ${UCVM_INSTALL_DIR}/conf/ucvm.conf with the model interface and path. For example:
For SCEC Etrees:
user1_interface=model_etree user1_modelpath=/home/username/model_etree/user1.etree
For Patch models:
user2_interface=model_patch user2_modelpath=/home/username/model_patch/user2.conf
SCEC Etrees can be created with the ./bin/ucvm2etree utility. Patch models can be created with the ./bin/patchmodel utility.
For other models that either are represented by tables, or have their own native API, UCVM may be extended to query from these models. However, this requires code modifications to the core UCVM library located in ./src/ucvm and recompilation. The general steps are:
- Define new model label UCVM_MODEL_MODELNAME in ucvm_dtypes.h
- Create ucvm_model_modelname.h/.c containing the glue code that links the UCVM query interface to the native model interface
- Modify ./src/ucvm/ucvm.c:
- Add ucvm_model_modelname.h include at top
- Add model lookup to ucvm_add_model() function
- Add model to ucvm_get_resources() function
- Modify ./src/ucvm/Makefile.am:
- Add ucvm_model_modelname.o to list of libucvm.a dependencies
- Modify ./configure.ac:
- Define "enable-modelname", "with-modelname-lib-path", and "with-modelname-incl-path" configure options
- Add library and header checks for your new model
- Modify installer to customize path to your new model in ucvm.conf:
- Add your new model to ./conf/Makefile.am
- Add config entry for your new model to ./conf/ucvm_template.conf
- Regenerate the Makefiles with ./scripts/autoconf/reconf.sh
- Reconfigure and recompile UCVM, making sure to pass your new model flags to the configure utility
Adding a User-defined Map
Update ${UCVM_INSTALL_DIR}/conf/ucvm.conf with the map interface and Etree path:
usermap1_interface=map_etree usermap2_mappath=/home/username/maps/usermap1.e
User-defined maps can be created with the ./bin/grd2etree utility.
Framework Description
Concept of Operation
Combining Velocity Models into a Composite Model
UCVM combines multiple regional velocity models, along with a DEM map and a Vs30 map, into one composite model for the purposes of querying surface elevation, Vs30, Vp, Vs, and density. Individual model setup and query details are abstracted under a uniform interface, and the application sees only the composite model. This simplifies application code and makes it easier for those programs to support new velocity models in the future.
Velocity models are combined into the composite by tiling them on top of one another. When UCVM is initialized, the user selects an ordered list of models to query for data. Query points are then submitted to each velocity model in that list, one model at a time in list order. The first model to return valid velocity data for the point is considered to have fullfilled that data request and subsequent models are not queried. Generally, no smoothing is performed at the interfaces between models (an exception is interpolation between a GTL and crustal model as described below). For each query point, the following data is returned: surface elevation, Vs30, Vp, Vs, and density. The figure at right illustrates how this tiling is performed.
Regional models may include: CVM-H, CVM-H, USGS Bay Area, 1D, or other user-defined models. Most models and maps have a maximum extent, outside of which no data is available. The exception is the generic 1D model as it has infinite extents. Applications must check the returned velocity and density values to ensure they are valid.
Querying with one or more GTLs
The API distinguishes between two types of regional velocity models: GTL models, and crustal models. GTL models are shallow near-surface velocity models called geotechnical layers. Crustal models are deeper velocity models such as tomographic models. When a user selects to query from one or more crustal models only, the API executes the previously described behavior. However, if one or more GTL models are selected, additional processing is performed.
When a GTL model is given in the ordered list of models, the API creates a default interpolation zone along the z-axis over which interpolation between this GTL and the underlying crustal models is performed. It also assigns a default (linear) interpolation function to smooth the velocities of the GTL and underlying crustal models. Both the interpolation zone and interpolation function can be changed by the user.
Querying for a point in this operating mode is now slightly different from the previous crustal-only case. As before, all query points are submitted to the crustal models in the ordered list of models (skipping any GTLs). However, the z coordinate of the points may be shifted down to the lower edge of the interpolation zone if the point falls within it. The crustal models are queried with this possibly modified 3D coordinate and the velocities/density values are saved. The same list of query points is then submitted to the GTL models in the ordered list of models (skipping any crustals) and the z coordinate of those points within the interpolation zone may be shifted up to the top edge. The GTL models are queried with this possibly modified 3D coordinate and the velocity/density values are saved.
The list of query points is then traversed one final time. Those points that fall above the interpolation zone will have their velocities/densities set to those of the GTL as processed by the interpolation function assigned to that GTL. Those points that fall within the interpolation zone and have valid GTL/crustal values will have their GTL/crustal values smoothed according to the assigned interpolation function. Points that fall below the interpolation zone and have valid crustal values will have their velocities/densities set to the crustal values. Note that points with invalid/incompatible velocities in either the GTL or crustal dataset cannot be smoothed and the interpolation results will be marked as invalid.
Querying Models Via Native Interface versus Querying UCVM
For the most part, querying any model through its native interface will yield the same material properties as UCVM. There are a few exceptions, which are noted here:
- When querying by elevation with CVM-H and any GTL, points that fall above the UCVM free surface and within the CVM-H domain are normalized to be relative to the CVM-H free surface. This is done to eliminate small discrepancies between the UCVM DEM and the CVM-H DEM.
- The same correction is performed when querying by elevation with CenCalVM and any GTL.
- When querying CenCalVM by depth, UCVM does not directly pass that point to the cencalvm interface since the definition of depth differs between UCVM and cencalvm. Instead, UCVM derives the elevation of the free surface (ground/air, ground/water) at the point of interest by drilling down through the model at that point and finding the highest octant with valid material properties. UCVM then converts the query point depth to elevation and queries cencalvm by elevation.
Utilities
ucvm_query
This is the command-line tool for querying CVMs. Any set of crustal and GTL velocity models may be selected and queried in order of preference. Points may be queried by (lon,lat,dep) or (lon,lat,elev) and the coordinate conversions for a particular model are handled transparently.
Usage: ucvm_query [-m models<:ifunc>] [-p user_map] [-c coordtype] [-f config] [-z zmin,zmax] < file.in Flags: -h This help message. -m Comma delimited list of crustal/GTL models to query in order of preference. GTL models may optionally be suffixed with ':ifunc' to specify interpolation function. -p User-defined map to use for elevation and vs30 data. -c Z coordinate mode: geo-depth (gd, default), geo-elev (ge). -f Configuration file. Default is ./ucvm.conf. -z Optional depth range for gtl/crust interpolation. Input format is: lon lat Z Output format is: lon lat Z surf vs30 crustal cr_vp cr_vs cr_rho gtl gtl_vp gtl_vs gtl_rho cmb_algo cmb_vp cmb_vs cmb_rho Notes: - If running interactively, type Cntl-D to end input coord list. Version: 12.2.0
The config file is generally ${UCVM_INSTALL_DIR}/conf/ucvm.conf, but you may pass in any custom configuration file.
ucvm2etree
This utility creates a SCEC-formatted CVM Etree extracted from a set of CVMs. The CVM Etree may then be imported into UCVM and queried like any other velocity model. This confers a number of advantages:
- Several different velocity models can be combined into one composite model and distributed as a single file.
- Any number of CVM Etrees can be easily imported into UCVM without recompilation (just a config change) since Etree support is built in.
- The Etree data structure is efficient and the library provides disk caching and buffering, resulting in very fast query performance.
Two versions of this utility are provided, a serial version and an MPI version.
Serial ucvm2etree
The serial ucvm2etree is suitable for small to medium sized etrees, and is executed as follows:
Usage: ucvm2etree [-h] -f config Flags: -f: Configuration file -h: Help message Version: 12.2.0
The following is an example ucvm2etree configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/ucvm2etree_example.conf:
# Domain corners coordinates (clockwise, degrees): proj=geo-bilinear lon_0=-119.288842 lat_0=34.120549 lon_1=-118.354016 lat_1=35.061096 lon_2=-116.846030 lat_2=34.025873 lon_3=-117.780976 lat_3=33.096503 # Domain dimensions (meters): x-size=180000.0000 y-size=135000.0000 z-size=61875.0000 # Blocks partition parameters: nx=32 ny=24 # Max freq, points per wavelength, Vs min max_freq=0.5 ppwl=4.0 vs_min=200.0 # Max allowed size of octants in meters max_octsize=10000.0 # Etree parameters and info title=ChinoHills_0.5Hz_200ms author=P_Small date=05/2011 outputfile=./cmu_cvmh_chino_0.5hz_200ms.e format=etree # UCVM parameters #ucvmstr=cvms ucvmstr=cvmh ucvm_interp_zrange=0.0,350.0 ucvmconf=../../conf/kraken/ucvm.conf # Scratch scratch=/lustre/scratch/patricks/scratch # # Buffering parameters used by MPI version only # # Etree buffer size in MB buf_etree_cache=128 # Max octants to buffer for flat file during extraction buf_extract_mem_max_oct=4194304 # Max octants to save in flat file before reporting full during extraction buf_extract_ffile_max_oct=16000000 # Max octants to read from input flat file during sorting buf_sort_ffile_max_oct=20000000 # Minimum number of octants between reports during merging buf_merge_report_min_oct=10000000 # MPI send/recv octant buffer size during merging buf_merge_sendrecv_buf_oct=4096 # Etree read/write octant buffer size during merging buf_merge_io_buf_oct=4194304
Parallel ucvm2etree
The MPI version is useful for large and very large etrees. It involves three programs: ucvm2etree-extract-MPI, ucvm2etree-sort-MPI, and ucvm2etree-merge-MPI. All three take the same configuration file as the one used by the serial version. The ucvmetree-extract-MPI utility generates the set of points that are to be used in the etree, extracts those points from the specified CVMs, and saves them in a set of N files (where N is the process count). The ucvm2etree-sort-MPI utility sorts the points in each flat file by octant key. The third utility, ucvm2etree-merge-MPI, merges these locally sorted points into the final etree file.
Import the new CVM Etree into UCVM by adding these lines to ${UCVM_INSTALL_DIR}/conf/ucvm.conf:
newmodel_interface=model_etree newmodel_modelpath=config.outputfile where: newmodel : name for your new model config.outputfile : outputfile from config passed into ucvm2etree
The following sections describe these three programs in more detail:
ucvm2etree-extract-MPI
Divides the etree region into C columns for extraction. This is an embarrassingly parallel operation. A dispatcher (rank 0) farms out each column to a worker in a pool of N cores for extraction. Each worker queries UCVM for the points in its column and writes a flat-file formatted etree. After program execution, there are N sub-etree files, each locally unsorted. The extractor must be run on 2^Y + 1 cores where Y>0 and (2^Y) < C. The output flat file format is a list of octants(24 byte addr, 16 byte key, 12 byte payload) in arbitrary Z-order.
Since the number of points in a column depends on the minimum Vs values within that column, some columns will have high octant counts and others will have very low octant counts. Having sub-etrees that vary greatly in size is not optimal for the sorting operations that follow, so ucvm2etree-extract-MPI implements a simple octant balancing mechanism. When a worker has extracted more than X octants (the default 16M octants), it reports to the dispatcher that it cannot extract any more columns and terminates. This strategy approximately balances the sub-etrees so that they may be loaded into memory by ucvm2etree-sort-MPI. In the case of very large extractions where the dispatcher reports that all workers have reported they are full yet columns remain to be extracted, increase the job size by a factor of 2 until there is room for all the columns.
ucvm2etree-sort-MPI
Sorts the sub-etrees produced by ucvm2etree-extract-MPI so that each file is in local pre-order (Z-order). Again, the is an embarrassingly parallel operation. Each rank in the job reads in one of the sub-etrees produced by the previous program, sorts the octants in Z-order, and writes the sorted octants to a new sub-etree. The sorter must be run on 2^Y cores where Y>0. The worker pool must be large enough to allow each worker to load all the octants from its assigned file into memory. By default, this octant limit is 20M octants. If a rank reports that the size of the sub-etree exceeds memory capacity, the 20M buffer size constant may be increased if memory allows, or alternatively, cvmbycols-extract-MPI may be rerun with a larger job size to reduce the number of octants per file.
ucvm2etree-merge-MPI
Merges N locally sorted etrees in flat file format into a final, compacted etree. This is essentially a merge sort on the keys from the addresses read from the local files. The cores at the lowest level of the merge tree each read in octants from two flat files in pre-order, merge sort the two sets of addresses, then pass the locally sorted list of addresses to a parent node for additional merging. This proceeds until the points rise to rank 1 which has a completely sorted list of etree addresses. Rank 0 takes this sorted list and performs a transactional append on the final Etree. The merger must be run on 2^N cores.
The program reads in input files that are in flat file format. In can output a merged Etree in either Etree format or flat file format. Although, due to space considerations, it strips the output flat file format to a pre-order list ot octants(16 byte key, 12 byte payload). The missing addr field is redundant and can be regenerated from the key field.
MPI Version Buffer Values
There are a number of buffer values in the ucvm.conf file that must be accurately set for extraction, sorting, and merging to take place. Please see below for a list of those values and a description of the purpose of each one.
buf_etree_cache
Units: MB
Recommended Value: 128 usually works well, you can increase or decrease as need be
For Process: ucvm2etree-extract-MPI
Description: Used in the merging process during the etree_open call. According to the documentation of etree_open, this is more precisely defined as the "internal buffer space being allocated in megabytes". The value defined represents the memory buffer for etree traversing, finding, writing, etc.
buf_extract_mem_max_oct
Units: Number of octants
Recommended Value: 2097152
For Process: ucvm2etree-extract-MPI
Description: Used in the extraction process as the number of bytes to buffer before writing to disk. It reserves, in memory, a placeholder for this number of octants. The size of each octant is 52 bytes and as such, the actual memory allocation is 52 * this value.
buf_extract_ffile_max_oct
Units: Number of octants
Recommended Value: Varies. It must be greater than total output file size divided by the number of processes.
For Process: ucvm2etree-extract-MPI
Description: This value is very critical! This defines the maximum output file size that each worker process can write. If this is not large enough, the worker will halt and return a "worker is full message". The best way to set this is to guess how large your extraction file might be (a relatively detailed Chino Hills e-tree, for example, is 250GB, and divide that number by 52 * the number or processes you have. So if I was building that mesh, on 32 processes, this value would need to be at least 162 million. However, this value also ties in with the number below, so please read it carefully as well.
buf_sort_ffile_max_oct
Units: Number of octants
Recommended Value: Must be at least equal to buf_extract_ffile_max_oct
For Process: ucvm2etree-sort-MPI
Description: Used in the sorting process to determine the max number of octants to read into memory. Therefore, for each process running on the server, there must be number of processes * 52 * buf_sort_ffile_max_oct bytes of memory available. So in our example above, if we were running those 32 processes on 16 dual core machines, we would need 162 million * 52 * 2 cores bytes of RAM or about 16 GB. All sorting is done in memory, not using the file on the disk.
buf_merge_report_min_oct
Units: Number of octants
Recommended Value: Personal preference - default is 10000000.
For Process: ucvm2etree-merge-MPI
Description:. This defines the number of octants that must be processed between progress reports in the merge process. This progress reports take the form of "Appended [at least buf_merge_report_min_oct] in N.NNs seconds". If you would like more frequent updates, set this value to be smaller.
buf_merge_sendrecv_buf_oct
Units: Bytes
Recommended Value: 4096
For Process: ucvm2etree-merge-MPI
Description: The number of bytes to merge per loop during the merge process. It is recommended to keep this value as is.
buf_merge_io_buf_oct
Units: Number of octants
Recommended Value: 4194304
For Process: ucvm2etree-merge-MPI
Description: Number of octants to read in and store in memory for merging. After being merged, the memory is then flushed to disk and the process repeats. Since this is a value in octants, it will require the machine to have at least buf_merge_io_buf_oct * 52 bytes of memory.
Etree Summary
In order to understand how ucvm2etree works, some knowledge of the Etree data structure is required. The fundamental unit of measure in an Etree is the tick. This is a distance expressed in meters, and is derived from the length of the longest side of the geographic region:
tick size = (max(len x, len y, len z) / 2^31) m
In essence, the geographic region is divided up into a set of 2^31 ticks.
The funamental unit of data in an Etree is the octant. An octant contains an address (location code) and a payload (material properties). Octants are organized into a tree structure called an octree, where each node has up to eight children. Each level in this octree represents an octant size (alternatively: edge size, or resolution). Moving from level N to level N+1 in the octree increases resolution by 2x. Moving from level N to level N-1 reduces resolution by 2x. The resolution for a level can be determined with:
res from level N = (max_len / 2^N) m
The number of ticks in an octant of level N is:
ticks for level N = 2^N
And thus the octant size in level N is:
octant size = (2^N * (max(len x, len y, len z) / 2^31)) m
Given a particular resolution, the appropriate level to represent it in the etree can be determined with:
level N from res = (int)(log(max_len/res)/log(2)) + 1 (rounded up to next integer)
These relations are important in understanding the algorithm described in the following sections.
Algorithm Description
This program uses several algorithmic approaches to increase extraction speed and minimize the amount of disk space required to store the Etree. These are described in the following sections.
Gridding of Geographic Coordinates
Gridding of the geographic bounding box is performed either using bilinear interpolation of lat/long corners, or by a UCVM projection specification (proj4 projection string, rot angle, lat0, lon0). Grid points are located at cell center.
Optimized Etree Resolution Based on Frequency, PPWL, and Minimum Vs
One of the strengths of the Etree (octree) representation is that resolution can be adjusted so that a particular region is sampled at a lower/higher interval than surrounding areas. You are able to achieve high resolution in just those places where it is needed, and the rest of the region can be sampled at a much lower rez thereby saving considerable disk space.
With this flexibility comes the difficulty of determining what resolution is needed at all points within a meshing region to support Computational Seismology simulations. ucvm2etree uses an algorithm to deduce this resolution based off of three input parameters: maximum frequency to support, the number of points per wavelength that is desired, and the minimum Vs to support. The following relation is used to derive the resolution for a region:
res = vs_min / (ppwl * max_freq) where: res: resolution for the region with vs_min, in meters vs_min: Minimum Vs found in region, in meters/sec ppwl: desired points per wavelength max_freq: maximum frequency to support, in Hz
Using the above relation, the 3D space is decomposed into small columns, and the resolution in each column is tailored to the vs values found at each depth. The algorithm can be summarized as follows:
- Divide region in set of x,y columns, where the length of each column equals its width.
- For each column:
- Starting at depth = 0m, extract a 2D grid for the column at the resolution needed to support the max frequency, pounts/wavelength, and minimum Vs, with a depth that is one-half the current octant edge size (that is, Z axis is cell center as well).
- If the local minimum Vs is in this 2D grid allows for a lower resolution, and the current z value is divisible by the lower resolution, requery the CVM at the lower resolution. Otherwise, if the local min Vs in this grid needs a higher resolution, increase the resolution to what is required (up to the maximum resolution dictated by the configured global minimum Vs) and requery the CVM.
- Write this layer to a flat file (a single file when run serially, or a file local to this core when run in parallel)
- Step the depth down by the resolution of this 2D grid and loop to the next layer in the column.
- Loop to the next column
- Starting at depth = 0m, extract a 2D grid for the column at the resolution needed to support the max frequency, pounts/wavelength, and minimum Vs, with a depth that is one-half the current octant edge size (that is, Z axis is cell center as well).
The ucvm2etree software enforces a minimum and maximum resolution for octants. Given a certain set of input parameters, these resolutions can be derived as follows:
If vs_min = 200 m/s, max_freq = 4.0 Hz, and ppwl = 8, max_dim_length = 180000.0 m, max_cellsize = 500.0 m, then Maximum Resolution: res = 200.0 / (8.0 * 4.0) = 6.25 m (minimum resolution needed to support Vs=200 m/s) N = (log(180000.0/6.25)/log(2)) + 1 = 15 max_res = 180000.0 / 2^15 ticks ~= 5.49 m Therefore, maximum resolution (minimum edge size) for these parameters is 5.49 m. Minimum Resolution: res = 500.0 m (the maximum cell size to support) N = (log(180000.0/500.0)/log(2)) + 1 = 9 min_res = 180000.0 / 2^9 ticks ~= 351.56 m Therefore, minimum resolution (maximum edge size) for these parameters is 351.56 m.
- Minimum resolution represents a floor on the possible resolution of an etree. This is protection against very large octants from being formed by high Vs values.
High-performance Etree merging and compaction
Generation of an Etree in a parallel environment is challenging since the Euclid etree interface only allows serial write access to the etree file. Points may only be inserted, or they may be appended in key order, by a single writer. This makes etee construction and compaction time consuming. To circumvent this issue, a high performance etree merge/compaction algorithm has been developed.
Simply put, it is essentially a merge sort performed on 2^N small flat files that contain all of the points needed for the final etree. The flat files contain material properties previously extracted from UCVM. Once these small, local flat files exist on disk, an MPI job of 2^N cores is then able to sort these octants by their keys in O(n*log n) time. Once all the octants are sorted, a single core may then open the final etree/flat file for writing in append mode, and simply stream the sorted octants to disk efficiently.
Conceptually, the cores in this 2^N MPI job are organized into a tree-like structure with each node containing two children. The bottom layer of this tree has nodes that each do a pre-order traversal on two flat files. They sort the combined octants read from each etree in pre-order, and then pass them up to a parent node. The nodes from the top layer to the second to bottom layer each receive locally sorted lists of octants from their two children and once again sort the combined list of octants in pre-order. This newly sorted list is passed upward in this manner until the root node of the tree receives two locally sorted halves of the entire set of etree points. It then sorts them, and appends the fully sorted stream of octants to the final file.
ucvm2mesh
This utility creates a 3D regular mesh extracted from a set of CVMs. The CVM Etree may then be used in a wave propagation simulation such as AWP-ODC. It can produce meshes in a number of formats
- AWP 3 value: float Vp, float Vs, float Rho
- AWP 5 value: float Vp, float Vs, float Rho, float Qp, float Qs
- AWP 8 vlaue: int i, int j, int k, float Vp, float Vs, float Rho, float Qp, float Qs
- SORD: One file each for Vp, Vs, Rho (double precision)
- netCDF (with optional netCDF package installed, and the mesh2netcdf utility)
Two versions of this utility are provided, a serial version and an MPI version.
Serial ucvm2mesh
The serial ucvm2etree is suitable for small to medium sized meshes, and is executed as follows:
Usage: ucvm2mesh [-h] -f config Flags: -f: Configuration file -h: Help message Version: 12.2.0
The following is an example ucvm2mesh configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/ucvm2mesh_example.conf:
# List of CVMs to query ucvmlist=cvmh # UCVM conf file ucvmconf=../conf/kraken/ucvm.conf # Gridding cell centered or vertex gridtype=CENTER # Spacing of cells spacing=2000.0 # Projection proj=+proj=utm +datum=WGS84 +zone=11 rot=-40.0 x0=-122.3 y0=34.7835 z0=0.0 # Number of cells along each dim nx=384 ny=248 nz=25 # Partitioning of grid among processors px=2 py=2 pz=5 # Vs/Vp minimum vp_min=0 vs_min=0 # Mesh and grid files, format meshfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.media gridfile=/lustre/scratch/user/mesh_cvmh_ijk12_2000m.grid meshtype=IJK-12 # Location of scratch dir scratch=/lustre/scratch/user/scratch
Parallel ucvm2mesh
The MPI version, ucvm2mesh-mpi, is useful for large and very large meshes. It accepts the same configuration file as the serial version. It provides an additional command-line option for stage-out of the mesh files from the locations specified in the conf file to a final directory location. This is useful on HPC systems that use temporary scratch space.
Usage: ucvm2mesh-mpi [-h] [-o dir] -f config Flags: -f: Configuration file -o: Stage-out directory -h: Help message Version: 12.2.0
Performance Enhancements for ucvm2mesh on Lustre Filesystems
When running ucvm2mesh on Lustre filesystems (and other systems with similar structures), you may experience some performance issues that result in less than optimal performance. By default, Lustre uses one OST or disk per file. As such, running ucvm2mesh-mpi with a significant number of cores results in all those processes attempting to seek and write to the same file on the same OST/disk. This can result in a disk bottleneck that gives very poor performance, where it may take up to 10 minutes to write a few hundred MBs.
This can be fixed by increasing the number of OSTs/disks to which the output directory and files are written. This is called striping. Generally, you will want to set the stripes for the output directory to be either the recommended amount for large files or the maximum amount allowed. The command to set the number of stripes on Kraken is lfs setstripe [dir] --count [count]. Count can be a fixed number or it can be -1 for the maximum supported by the operating system. The Kraken documentation suggests that 80 is a reasonable amount for very large files. So, for example, suppose we were generating a large mesh with our stage out directory being /lustre/scratch/username/meshes/outstripe. We would first make that directory and then run:
lfs setstripe /lustre/scratch/username/meshes/outstripe --count 80
Then we would have our configuration file output to that directory:
meshfile=/lustre/scratch/username/meshes/outstripe/USC_cvms_awp.media gridfile=/lustre/scratch/username/meshes/outstripe/USC_cvms_awp.media.grid
When we qsub the job, it will automatically take advantage of striping and distribute the load across multiple targets.
It is also important to remember that more cores is not always better. Additional cores can result in increased write requests that may cause performance decreases. It is important to find the optimal balance between the number of processes writing to the file and the number of stripes. For exampe, for a large 600km x 400km x 40km meshes, a core count of 600 and stripe count of 80 on NICS Kraken will result in reasonably good performance.
grd_query
This command-line tool queries data from a set of ArcGIS grid files in GridFloat format.
Usage: grd_query [-e] [-v] -d datadir [-b bkgdir] < file.in Flags: -e Employ GRD bathymetry heuristic. -v Employ GRD vs30 heuristic. -d Main ArcGIS Gridfloat data directory. Default is ./ned. -b Background/Bathymetry data directory. Default is ./bath. -h This help message. Input format is: lon lat Output format is: lon lat val valid Notes: - If running interactively, type Cntl-D to end input coord list. Version: 12.2.0
grd2etree
This utility extracts a SCEC-formatted Etree map from a set of DEM and Vs30 grid files in ArcGIS Gridfloat format.
Usage: grd2etree [-h] -f config Flags: -f: Configuration file -h: Help message Version: 12.2.0
The following is an example grd2etree configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/grd2etree_example.conf:
# grd2etree UCM Map config file # Domain corners coordinates (degrees): proj=+proj=aeqd +lat_0=36.0 +lon_0=-120.0 +x_0=0.0 +y_0=0.0 lon_0=-129.75 lat_0=40.75 rot=55.0 # Domain dimensions (meters): x-size=1800000.0 y-size=900000.0 # Spacing spacing=250.0 # Etree parameters and info title=UCVM_Elev_Vs30_Map_Wills_Wald author=P_Small date=05/2011 outputfile=../../model/ucvm/ucvm.e # Grid data directories elev_hr_dir=/home/scec-00/USER/opt/ned elev_lr_dir=/home/scec-00/USER/opt/bath vs30_hr_dir=/home/scec-00/USER/opt/vs30/wills_gridfloat vs30_lr_dir=/home/scec-00/USER/opt/vs30/wald_gridfloat
Import the new Etree Map into UCVM by adding these lines to ${UCVM_INSTALL_DIR}/conf/ucvm.conf:
newmodel_interface=map_etree newmodel_modelpath=config.outputfile where: newmodel : name for your new model config.outputfile : outputfile from config passed into grd2etree
patchmodel
The utility provides a post-processing "patching" mechanism to smooth out discontinuities at the interfaces between different regional models. The user identifies a 3D geographic region in the form of an open box where smoothing is to be performed. The patchmodel program is run to extract the material properties along the vertical surfaces of this box at a fixed resolution (eg: 250 m) from 0m depth to "z-size" depth and it saves them into a new model which may then be imported into and queried by UCVM. Points that fall within the smoothing region are interpolated using an inverse distance weighting function with the 10% closest surface points on the same z-plane. A power factor of 2.0 is used for weighting. Linear interpolation is used along the z-axis to smooth out the edge values.
Usage: patchmodel [-h] -f config Flags: -f: Configuration file -h: Help message Version: 12.2.0
The following is an example patchmodel configuration file. It can also be found in ${UCVM_INSTALL_DIR}/conf/example/patch_example.conf:
# Example Patchmodel config file # Version version=cvmh_p1_2011/05/24 # UCVM config ucvmstr=cencal,cvmh,lin-thurber,1d ucvm_interp_zrange=0.0,350.0 ucvmconf=../../conf/aftershock/ucvm.conf # Projection proj=+proj=aeqd +lat_0=36.0 +lon_0=-120.0 +x_0=0.0 +y_0=0.0 lon_0=-121.3 lat_0=35.6 rot=90.0 # Domain dimensions (meters): x-size=550000.0 y-size=50000.0 z-size=112500.0 # Spacing (meters) spacing=250.0 # Output paths modelname=cvmh_p1 modelpath=/home/scec-00/USER/opt/aftershock/ucvm/model/patch
Import the new patch into UCVM by adding these lines to ${UCVM_INSTALL_DIR}/conf/ucvm.conf:
newmodel_interface=model_patch newmodel_modelpath=config.modelpath/config.modelname.conf where: newmodel : name for your new model config.modelpath : modelpath from config passed into patchmodel config.modelname : modelname from config passed into patchmodel Note: Be sure to append ".conf" to newmodel_modelpath
ecompact
This optimization utility compacts an etree file with any payload and schema. It does this by removing empty space in the Etree data structure.
Usage: ecompact in_etree out_etree
ecoalesce
This optimization utility coalesces an etree file that conforms to either the SCEC CVM Etree or CMU CVM Etree conventions. It does this by replacing eight adjacent octants with identical material properties at level N with a single octant containing the same material properties at level N-1.
Usage: ecoalesce in_etree out_etree
C Application Programming Interface
UCVM provides a programming interface for querying Vp, Vs, and rho from any user-defined model. The main UCVM interface is as follows:
#ifndef UCVM_H #define UCVM_H #include <stdarg.h> #include "ucvm_dtypes.h" /* Initializer */ int ucvm_init(const char *config); /* Finalizer */ int ucvm_finalize(); /* Enable specific model(s), by string list, by label, or by ucvm_model_t */ int ucvm_add_model_list(const char *list); int ucvm_add_model(const char *label); int ucvm_add_user_model(ucvm_model_t *m, ucvm_modelconf_t *conf); /* Use specific map (elev, vs30) by label */ int ucvm_use_map(const char *label); /* Associate specific interp func with GTL model, by label or by ucvm_ifunc_t */ int ucvm_assoc_ifunc(const char *mlabel, const char *ilabel); int ucvm_assoc_user_ifunc(const char *mlabel, ucvm_ifunc_t *ifunc); /* Get label for a model */ int ucvm_model_label(int m, char *label, int len); /* Get label for an interpolation function */ int ucvm_ifunc_label(int f, char *label, int len); /* Get version for a model */ int ucvm_model_version(int m, char *ver, int len); /* Set parameters (see ucvm_dtypes.h for valid param flags) */ int ucvm_setparam(ucvm_param_t param, ...); /* Query underlying models */ int ucvm_query(int n, ucvm_point_t *pnt, ucvm_data_t *data); /* Get info on installed models */ int ucvm_install_info(char *info, int len); #endif
The general order of operations for using this interface is:
- Initialization (
ucvm_init()
) - Add models in order of preference using
ucvm_add_model()
. These may be either crustal or GTL models. See below for specific details. - Set any models-specific flags using
ucvm_setparam
. These will override any flags specified in ucvm.conf. For example, to enable the 1D background model with CVM-H through the API, execute theucvm_setparam()
function as follows:
ucvm_setparam(UCVM_PARAM_MODEL_CONF, UCVM_MODEL_CVMH, "USE_1D_BKG", "True");
- If GTLs have been added:
- Associate an interpolation function with each using
ucvm_assoc_ifunc()
. Otherwise, linear interpolation will be used. - Set an interpolation depth range using
ucvm_setparam(UCVM_PARAM_IFUNC_ZRANGE, ...)
. Otherwise the interpolation range is set to 0,0.
- Associate an interpolation function with each using
- Query points of interest with
ucvm_query()
- Finalization (
ucvm_finalize()
)
Notes on enabling models:
- Models are referenced either by a pre-defined string id or by a populated
ucvm_model_t
structure. - Multiple models may be registered, and they are queried in order of registration.
- A helper function
ucvm_add_model_list()
is provided to quickly add multiple models. The string format is "gtl_model<:gtl_interp>,crustal_model,..." where "<:gtl_interp>" is an optional GTL interpolation function specification. For example, the string "elygtl:ely,cvmh" specifies the Ely GTL and CVM-H, adn the use of the Ely interpolation function to smooth between the GTL and crustal models. - Pre-defined models include: CVM-S, CVM-H, CenCal, Lin-Thurber, Rob Graves Cape Mendocino, 1D, Ely GTL, any SCEC Etree or patch listed in the UCVM configuration file. These models are enabled by calling
ucvm_add_model()
with their name string identifier as listed in either ucvm_dtypes.h (pre-defined) or in the UCVM configuration file (user-defined SCEC Etree, UCVM patch). - Most predefined model interfaces only support one model instance at a time. However, multiple Etrees and patches may be enabled. Etrees and patches are defined in the UCVM configuration file with the following commands:
# Etree definition and path to etree usermodel1_interface=model_etree usermodel1_modelpath=/home/... # Patch definition and path to configuration file usermodel2_interface=model_patch usermodel2_modelpath=/home/... where: usermodel1/usermodel2 : replaced with desired string identifier for the etree/patch model
- The programmer may also define their own models and integrate them into UCVM. A user model interface is defined by creating the init(), finalize(), getversion(), getlabel(), setparam(), and query() functions for the model interface, and populating the
ucvm_model_t
structure with these function pointers and some additional record keeping information. Configuration information for a specific model instance of that interface is placed in aucvm_modelconf_t
structure. Once a model is fully described with aucvm_model_t
interface and aucvm_modelconf_t
configuration, it can be registered for use with theucvm_add_user_model()
function and queried with theucvm_query()
function.
Data Type Descriptions
The full list of defined data types are listed in /include/ucvm_dtypes.h. The following is a general description of the useful application definitions:
Points are specified with a ucvm_point_t
structure:
/* 3D point */ typedef struct ucvm_point_t { double coord[3]; } ucvm_point_t;
Return data is specified with a ucvm_data_t
structure:
typedef struct ucvm_data_t { double surf; /* Elevation of free surface in meters */ double vs30; /* Vs30 value in m/s */ ucvm_domain_t domain; /* Domain where the point lies: GTL model, interpolation zone, crustal model, none. */ double shift_cr; /* Depth shift applied to point when querying crustal models. Only used when a GTL is enabled. */ double shift_gtl; /* Depth shift applied to point when querying GTL models. Only used when a GTL is enabled. */ ucvm_prop_t crust; /* Material properties from crustal model */ ucvm_prop_t gtl; /* Material properties from GTL model */ ucvm_prop_t cmb; /* Combined GTL/crustal material properties. This is the final property set that applications should reference. */ } ucvm_data_t; where ucvm_prop_t is: /* Material properties */ typedef struct ucvm_prop_t { int source; /* Model idenfitifer */ double vp; /* Velocity in m/s */ double vs; /* Velocity in m/s */ double rho; /* Density in g/m^3 */ } ucvm_prop_t;
Model interfaces (both crustal and GTL) are defined with a ucvm_model_t
structure:
/* Model */ typedef struct ucvm_model_t { ucvm_mtype_t mtype; int (*init)(int id, ucvm_region_t *r, const char *config); int (*finalize)(); int (*getversion)(int id, char *ver, int len); int (*getlabel)(int id, char *ver, int len); int (*setparam)(int id, int param, ...); int (*query)(int id, ucvm_ctype_t cmode, int n, ucvm_point_t *pnt, ucvm_data_t *data); } ucvm_model_t;
Other Interfaces
2D Grid Interface
Provides an interface for generating 2D regular grids in any USGS map projection. Include the header "ucvm_grid.h" to use these functions.
/* Generate grid from projection and dimensions */ int ucvm_grid_gen(ucvm_proj_t *iproj, ucvm_trans_t *trans, ucvm_proj_t *oproj, ucvm_dim_t *dims, double spacing, ucvm_point_t *pnts); /* Generate grid from projection and dimensions */ int ucvm_grid_gen_file(ucvm_proj_t *iproj, ucvm_trans_t *trans, ucvm_proj_t *oproj, ucvm_dim_t *dims, double spacing, const char *filename); /* Convert point list from one projection to another */ int ucvm_grid_convert(ucvm_proj_t *iproj, ucvm_proj_t *oproj, size_t n, ucvm_point_t *pnts); /* Convert point list from one projection to another */ int ucvm_grid_convert_file(ucvm_proj_t *iproj, ucvm_proj_t *oproj, size_t n, const char *filename);
Extracting Values from UCVM
Two methods are provided for extracting material properties from UCVM: command-line tools, and a C API. These are described in the sections below. The following table summarizes the model features supported by each interface:
Extracting Values on the Command Line
The ucvm_query utility is provided for querying a set of community velocity models. Points may either be input interactively or via a text file, and can be either (lon, lat, elevation) or (lon, lat, depth).
The following output shows an interactive example on three points (followed by a Cntl-D to terminate input) using the CVM-S model. Starting in the UCVM installation directory, enter these commands:
% cd bin % ./ucvm_query -m cvms -f ../conf/ucvm.conf Using Geo Depth coordinates as default mode. -118.0 34.0 0.0 -118.0 34.0 100.0 -118.0 34.0 1000.0 <Cntrl-D> -118.0000 34.0000 0.000 295.679 390.000 cvms 696.491 213.000 1974.976 none 0.000 0.000 0.000 crust 696.491 213.000 1974.976 -118.0000 34.0000 100.000 295.679 390.000 cvms 1683.174 603.470 2130.773 none 0.000 0.000 0.000 crust 1683.174 603.470 2130.773 -118.0000 34.0000 1000.000 295.679 390.000 cvms 3660.809 2056.628 2443.042 none 0.000 0.000 0.000 crust 3660.809 2056.628 2443.042
Similarly, here is an example for the CVM-H model:
% cd bin % ./ucvm_query -m cvmh -f ../conf/ucvm.conf Using Geo Depth coordinates as default mode. -118.0 34.0 0.0 -118.0 34.0 100.0 -118.0 34.0 1000.0 <Cntrl-D> -118.0000 34.0000 0.000 295.679 390.000 cvmh 824.177 195.000 1084.062 none 0.000 0.000 0.000 crust 824.177 195.000 1084.062 -118.0000 34.0000 100.000 295.679 390.000 cvmh 2371.500 727.748 2051.320 none 0.000 0.000 0.000 crust 2371.500 727.748 2051.320 -118.0000 34.0000 1000.000 295.679 390.000 cvmh 2616.063 1082.813 2127.783 none 0.000 0.000 0.000 crust 2616.063 1082.813 2127.783
Additional Examples
The following example directs ucvm_query to query the Ely GTL and CVM-S crustal model by depth, using the Ely interpolation function to smooth the values. The Z-axis interpolation zone is set to 0-400m.
% ./ucvm_query -m elygtl:ely,cvms -c gd -z 0.0,400.0 -f ../conf/ucvm.conf
The following example queries the USGS Bay Area (CenCalVM) and CVM-H models by elevation, tiling them into a composite model (where CenCalVM has priority).
% ./ucvm_query -m cencal,cvmh -c ge -f ../conf/ucvm.conf
The following example queries the 1D model by depth, with surface elevation and Vs30 values taken from the alternate Yong-Wald Vs30 map.
% ./ucvm_query -m 1d -p yong -c gd -f ../conf/ucvm.conf
Extracting Values in a C Program
Working code examples that use the UCVM A API can be found in the directory ${UCVM_INSTALL_DIR}/examples. Feel free to use these examples as a basis for your own programs. In general, using the API in your program requires the following source/Makefile updates.
Include the ucvm.h header in your source code at a minimum. Eg:
#include "ucvm.h"
Set the include path to the location of the UCVM headers, the Etree and Proj.4 headers, and all of the standard model include paths. Eg, for GNU C compiler and CVM-S:
CFLAGS = $(CFLAGS) -I$(UCVM_INSTALL_DIR)/include -I$(CVMS_INSTALL_DIR)/include -I$(ETREE_INSTALL_DIR)/libsrc -I$(PROJ4_INSTALL_DIR)/include
Identify the library paths and names to UCVM static library, the Etree and Proj.4 libraries, and all of the standard model library paths needed by your application to the linker. Depending on platform, also include the pthreads and math libraries. Eg, for GNU linker and CVM-S (note the use of the gfortran lib since CVM-S is Fortran):
LDFLAGS = $(LDFLAGS) -L$(UCVM_INSTALL_DIR)/lib ucvm -L($CVMS_INSTALL_DIR)/lib cvms -L$(ETREE_INSTALL_DIR)/libsrc etree -L$(PROJ4_INSTALL_DIR)/lib proj -lgfortran -pthreads -lm
Example Programs
Please see the ./examples directory of the software distribution for compiled examples that employ the C API.
History of UCVM Releases
- 12.2.0 (Feb 9, 2012): Initial release
Acknowledgements and Contact Info
Support for the development and maintenance of the UCVM framework has been provided by the Southern California Earthquake Center (SCEC). SCEC is funded by NSF Cooperative Agreement EAR-0106924 and USGS Cooperative Agreement 02HQAG0008.
Contributions to this manual were made by: Patrick Small, and Philip Maechling.
Please email software@scec.org for help on downloading and using UCVM, and for any suggestions for the delivery of the code or for this manual.
Please reference at least Small et al (2011) if you use this software framework; other references should be considered, depending on the purpose.
Technical Notes
SCEC CVM Etree Convention
Etree Model Format
- Variable resolution based off of Vs, max supported frequency, and points per wavelength. The resolution at any particular region will be dependent upon the local Vs value (relation shown below). However, a minimum Vs at Etree creation will clamp the resolution to a maximum value regardless of how low the local Vs is found to be.
rez = vs / (ppwl * freq) where: vs: Vs at point of interest (m/s) ppwl: points per wavelength freq: max supported frequency (Hz)
- Model data is referenced relative to free surface. Free surface is defined at air/ground, water/ground interface. UCVM topography and bathymetry can be used to determine if a point is in the air/water and default properties returned for those domains.
- Stored properties are Vp, Vs, and density.
Etree Metadata Description
The Etree metadata tag describes the payload of the octants, the extents of the coverage region, and the map projection. It will have the following format, with fields delimited by vertical bars:
metatype_version|title|author|date|vs_min|max_freq|ppwl|format|proj.4_string|geo_origin|rot_angle|xyz_dims|xyz_ticks where: metatype_version: "SCEC_CVM_V1" string of max length 64 chars title: free-format title string of max length 64 chars author: free-format author string of max length 64 chars date: YY/MM/DD date string vs_min: vs minimum used to determine max resolution max_freq: maximum supported frequency in Hz ppwl: Points per wavelength format: data format specification of octants. Uses string format: "name,data_type,size,units". eg: "vp,float,4,m/s;vs,float,4,m/s;rho,float,4,kg/m^3" proj.4_string: Proj.4 projection string, eg: "+proj=aeqd +lat_0=36.0 +lon_0=-120.0 +x_0=0.0 +y_0=0.0" geo_origin: Origin point of CVM region in geo coordinates, eg: "-129.25,41.0" rot_angle: Rotation angle of bounding box in degrees, eg "55.0" xyz_dims: Dimensions of bounding box in km, eg "1800,800,100" xyz_ticks: Number of ticks along each dimension
SCEC Patch Model Format
A patch model consists of five files: a configuration text file, and four binary files with the surface data. All of these files are automatically generated by the patchmodel utility. The sections below describe the file formats.
Configuration file
The patch model configuration file specifies the origin, dimensions, and map projection of the 3D region covered by the patch. An example follows:
# cvmh patch conf file # Version version=cvmh_p1_2011/05/24 # Projection proj=+proj=aeqd +lat_0=36.0 +lon_0=-120.0 +x_0=0.0 +y_0=0.0 lon_0=-121.300000 lat_0=35.600000 rot=90.000000 # Spacing spacing=250.000000 # Dimensions x-size=550000.000000 y-size=50000.000000 z-size=112500.000000 # Paths to surface files surf_0_0_path=/home/scec-00/USER/opt/aftershock/ucvm/model/patch/cvmh_p1_surf_0_0.bin surf_1_0_path=/home/scec-00/USER/opt/aftershock/ucvm/model/patch/cvmh_p1_surf_1_0.bin surf_0_1_path=/home/scec-00/USER/opt/aftershock/ucvm/model/patch/cvmh_p1_surf_0_1.bin surf_1_1_path=/home/scec-00/USER/opt/aftershock/ucvm/model/patch/cvmh_p1_surf_1_1.bin
Surface Files
The surface file format is a 2D array of three floating point values: vp, vs, and density. The assignment of x versus y axis in each file is dependent on the orientation of the surface with respect to the box's origin point.
SCEC Map Etree Convention
Etree Model Format
- 3D Etree with a single octant depth along Z-axis (effectively a 2D grid)
- Fixed resolution based off of maximum desired grid spacing in grd2etree configuration.
- Sampled with grid registration to allow smoothing of adjacent octants in UCVM.
- Stored values at each grid point are: surface elevation (float), Vs30 (float).
Etree Metadata Description
The Etree metadata tag describes the payload of the octants, the extents of the coverage region, and the map projection. It will have the following format, with fields delimited by vertical bars:
metatype_version|title|author|date|spacing|format|proj.4_string|geo_origin|rot_angle|xy_dims|xyz_ticks where: metatype_version: "SCEC_MAP_V1" string of max length 64 chars title: free-format title string of max length 64 chars author: free-format author string of max length 64 chars date: YY/MM/DD date string spacing: grid spacing in meters format: data format specification of octants. Uses string format: "name,data_type,size,units". eg: "surf,float,4,m/s;vs30,float,4,m/s;" proj.4_string: Proj.4 projection string, eg: "+proj=aeqd +lat_0=36.0 +lon_0=-120.0 +x_0=0.0 +y_0=0.0" geo_origin: Origin point of CVM region in geo coordinates, eg: "-129.25,41.0" rot_angle: Rotation angle of bounding box in degrees, eg "55.0" xy_dims: Dimensions of bounding box in km, eg "1800,800" xyz_ticks: Number of ticks along each dimension
References
- Ely, G., T. H. Jordan, P. Small, P. J. Maechling (2010), A Vs30-derived Near-surface Seismic Velocity Model Abstract S51A-1907, presented at 2010 Fall Meeting, AGU, San Francisco, Calif., 13-17 Dec. [Ely2010-AGU.pdf]
- Graves, R. (1994), Rupture History and Strong Motion Modeling of the 1992 Cape Mendocino Earthquake, USGS External Grant Report
- Lin, G., C. H. Thurber, H. Zhang, E. Hauksson, P. Shearer, F. Waldhauser, T. M. Brocher, and J. Hardebeck (2010), A California statewide three-dimensional seismic velocity model from both absolute and differential Times, Bull. Seism. Soc. Am., 100, in press. supplemental
- Small, P., P. Maechling, T. Jordan, G. Ely, and R. Taborda (2011), SCEC UCVM - Unified California Velocity Model, in 2011 Southern California Earthquake Center Annual Meeting, Proceedings and Abstracts, vol. TBD, p. TBD.
- Taborda R., López J., O'Hallaron D., Tu T. and Bielak J. (2007), A review of the current approach to CVM-Etrees, SCEC Annual Meeting, Palm Springs, CA, USA, September 8–12. [1]
- Wald, D. J., and T. I. Allen (2007), Topographic slope as a proxy for seismic site conditions and amplification, Bull. Seism. Soc. Am., 97 (5), 1379-1395, doi:10.1785/0120060267.
- Wills, C. J., and K. B. Clahan (2006), Developing a map of geologically defined site-condition categories for California, Bull. Seism. Soc. Am., 96 (4A), 1483-1501, doi:10.1785/0120050179.
- Yong, A., Hough, S.E., Iwahashi, J., and A. Braverman (2012), A terrain-based site conditions map of California with implications for the contiguous United States, Bull. Seism. Soc. Am., Vol. 102, No. 1, pp. 114–128, February 2012, doi: 10.1785/0120100262.
License and Disclaimer
License
Copyright (c)2011 Southern California Earthquake Center (SCEC) and the University of Southern California (USC). This copyright applies to the UCVM codebase as a whole, or any individual distributed applications. The individual contributions of government employees, which may be identified on a per-file basis using Subversion (SVN) (see developer site), are in the public domain.
The software and content provided on this website are made available under the terms of the Apache License, Version 2.0. A copy of the License is provided by the Apache Software Foundation (http://www.apache.org/licenses/LICENSE-2.0).
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Disclaimer
THIS SOFTWARE AVAILABLE ON THE UCVM WEBSITE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE SOUTHERN CALIFORNIA EARTHQUAKE CENTER (SCEC), THE UNIVERSITY OF SOUTHERN CALIFORNIA (USC), OR ANY OF THEIR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Without limiting the foregoing, SCEC and USC make no warranty that:
1. The software will meet your requirements.
2. The software will be uninterrupted, timely, secure or error-free.
3. The results that may be obtained from the use of the software will be effective, accurate or reliable.
4. The quality of the software will meet your expectations.
5. Any errors in the software obtained from the UCVM web site will be corrected.
Software and its documentation made available on the UCVM web site:
6. Could include technical or other mistakes, inaccuracies or typographical errors. UCVM contributors may make changes to the software or documentation made available on its web site.
7. May be out of date and SCEC, its contributors, and USC make no commitment to update such materials.
SCEC, its contributors, and USC assume no responsibility for errors or
omissions in the software or documentation available from the UCVM web site.
In no event shall SCEC, its contributors, or USC be liable to you or any third parties for any special, punitive, incidental, indirect or consequential damages of any kind, or any damages whatsoever, including, without limitation, those resulting from loss of use, data or profits, whether or not SCEC, its contributors, or USC has been advised of the possibility of such damages, and on any theory of liability, arising out of or in connection with the use of this software.
The use of the software downloaded through the UCVM site is done at your own discretion and risk and with agreement that you will be solely responsible for any damage to your computer system or loss of data that results from such activities. No advice or information, whether oral or written, obtained by you from SCEC, the UCVM website, its contributors, or USC shall create any warranty for the software.