UCVM Install

Background Infor and Resources

Confirm User is Added to SCEC Allocation: Create /project Storage /project/scec_608/<username>

CARC System Overview (hardware and queues):

• https://www.carc.usc.edu/user-information/user-guides/hpc-basics/discovery-resources

Storage Areas:

• https://www.carc.usc.edu/user-information/user-guides/data-management/storage-file-systems

Intro to scientific computing:

• https://software-carpentry.org/lessons/
• https://swcarpentry.github.io/shell-novice/
• https://www.tutorialspoint.com/how-to-use-nano-text-editor

Starting a Project group at CARC:

• https://www.carc.usc.edu/user-information/getting-started

Overview of Installing UCVM on Discovery

1) Confirm USC CARC Discovery account and allocation access

2) Identify location of Storage on /project/scec_608/<username>

3) Confirm Access to USC CARC OnDemand for transferring files

• [[https://www.carc.usc.edu/user-information/user-guides/hpc-basics/getting-started-ondemand USC CARC OnDemand)

4) ssh into Discovery - Use keepalive to avoid frequent logouts

• ssh -o "ServerAliveInterval 60" username@discovery2.usc.edu

5) select locations for 3 directories, working, src, and binary

• /home1/maechlin/test_ucvm
• /project/scec_608/maechlin/ucvm227_src
• /project/scec_608/maechlin/ucvm227

6) Install or Update anaconda in Discovery account

• test "which python"
• In not using conda, Install anaconda
  • copy anaconda from my account on projects
  • in test_ucvm_directory, run cp /project/scec_608/maechlin/Anaconda3-2022.10-Linux-x86_64.sh .
  • chmod +x *.sh
  • ./Anaconda3-2022.10-Linux-x86_64.sh
  • when asked for installation directory give directory like
  • /project/scec_608/maechlin/anaconda3
• If you have conda installed, run
  • update conda update

7) Update .bashrc to specific required compilers, and add path to ucvm setup script (which is not yet installed) module purge module load gcc/8.3.0 module load openmpi/4.0.2 module load pmix/3.1.3

1. Setup UCVM Environment

export LD_PRELOAD=/spack/apps/gcc/8.3.0/lib64/libstdc++.so.6 source /project/scec_608/<username>/ucvm227/conf/ucvm_env.sh

8) logout and log in and confirm correct environment (python gcc)

9) in ucvm227_src clone dev branch of ucvm (contains all available models)

• git clone https://github.com/SCECcode/ucvm.git

9a.) move into largefiles and run get_largefiles.py, and stage_largefiles.py, and then ucvm_setup.py

10) when prompted specify installation of binary directory for ucvm

• e.g. /project/scec_608/maechlin/ucvm227

11) when prompted, answer yes to all the velocity models available

12) log out/log in so UCVM environment is setup.

13) move to test_ucvm directory and confirm ucvm_query runs from test_ucvm, and ucvm_query -H lists all the velocity models

Phase 2 - Install Plotting Scripts

14.1) Check install conda envs

conda env list

14) install conda python2 virtual environment and plotting libraries

conda create -n python2 python=2.7.15 scipy pip numpy matplotlib basemap basemap-data-hires

15) move to ucvm227 bin directory and clone ucvm_plotting

• git clone https://github.com/ucvm_plotting.git

16) run plotting installation

• [ucvm_plotting]

17) confirm plotting runs from test_ucvm directory

18) activate python2 Add this command to end of .bashrc file

• conda activate python2

19) run plotting examples script

20) transfer plots to local computer using OnDemand

Recording of UCVM Installation Process

• Video of Zoom Call Installing UCVM on Discovery

Detailed Instructions for Installation on discovery.usc.edu

These instructions are a previous version, with somewhat more detail, for installating UCVM on Discovery. Some details listed here may have changed. Please contact software@scec.org if you have questions or problems.

0) confirm login and home directory /home1/<username>
1) create test_ucvm subdirectory for use later
3) confirm /project/scec_608/<username> exists
4) copy Anaconda Linux install script from /project/maechlin_162
5) run script and give installation directory as
  /project/scec_608/<username>/anaconda3 as install path
6) When it prompts to update .bashrc (or .bash_profile) say yes

7) Confirm .bashrc contains
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/project/scec_608/<username>/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/project/scec_608/<username>/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

8) add module unload/load command into .bash_profile
module purge
module load gcc/8.3.0
module load openmpi/4.0.2
module load pmix/3.1.3
# Setup UCVM Environment
export LD_PRELOAD=/spack/apps/gcc/8.3.0/lib64/libstdc++.so.6
$ source /project/scec_608/<username>/ucvm_bin/conf/ucvm_env.sh

9) log out and log back in
10) confirm anaconda python is picked up
$ which python
11) create /project/scec_608/<username>/dev directory
12) We will use the main branch of the git repo. CD into directory and type
$ git clone https://github.com/SCECcode/ucvm.git

This will create a subdirectory called ucvm. CD into that subdirectory.

13) cd into /project/scec_608/<username>/dev/ucvm

Currently, we stage the largefiles externally, giving the user options on how they
assemble the list of needed files. The script are designed to be run in the UCVM/largefiles dirctory

First, the user runs a script to get the largefiles.

$ cd largefiles

User then run the following script.
./get_large_files.py

This script asks user which model should be downloaded then the selected model files and other large files are copied from USC into the local directory.

User should be selective about which models then install, because some of the models are several gigabytes in size. Please review the supported models list to identify the velocity models of interest.

== Confirm the download of the models ==
Second, the user runs a script to confirm the largefiles are intact.
This script prints out the number of OK files and the number of files with errors.
If any files have errors, those files should be re-downloaded.
./check_largefiles_md5.py

Third, the user runs the stage_files script:
This copies the files from the largefiles directory into the ucvm directory tree
./stage_large_files.py


Finally, the user moves out of the UCVMC/largefiles directory, into the UCVM home directory.
From the UCVM/ home directory, the user runs:
./ucvm_setup.py

This will run the configure, make, install commands to build the libraries and models.

Set your environment variables

set src directory as /project/scec_608/<username>/ucvm_src
set bin directory as /project/scec_608/<username>/ucvm_bin


Once you have set these environment variables, return to the UCVM source directory and type
make check
This will run the UCVM unit and acceptance tests. If all tests pass. UCVM is correctly installed
and ready to use on your computer.

To try out ucvm, once the tests pass, move to the UCVM installation directory, and run an example query.

As an example, you can run ucvm_query:

cd /project/scec_608/chukwueb/ucvm_bin
./bin/ucvm_query -f ./conf/ucvm.conf -m cvms < ./tests/inputs/test_latlons.txt


You will then see the following output:
Using Geo Depth coordinates as default mode.
 -118.0000  34.0000   0.000  280.896  390.000    cvms  696.491  213.000  1974.976    none   0.000   0.000   0.000   crust  696.491  213.000  1974.976
 -118.0000  34.0000   50.000  280.896  390.000    cvms  1669.540  548.000  2128.620    none   0.000   0.000   0.000   crust  1669.540  548.000  2128.620
 -118.0000  34.0000  100.000  280.896  390.000    cvms  1683.174  603.470  2130.773    none   0.000   0.000   0.000   crust  1683.174  603.470  2130.773
-118.0000  34.0000  500.000  280.896  390.000    cvms  3097.562  1656.495  2354.105    none   0.000   0.000   0.000   crust  3097.562  1656.495  2354.105
 -118.0000  34.0000  1000.000  280.896  390.000    cvms  3660.809  2056.628  2443.042    none   0.000   0.000   0.000   crust  3660.809  2056.628  2443.042
Installation complete. Installation log file saved at ./setup_log.sh

15) Confirm entry in .bash_profile has Setup path env | grep ucvm env | grep UCVM

16) move to /home1/<username>/test_ucvm list the installed modules

ucvm_query -H

ucvm_query with a list of lon-lat-depth in a file

ucvm_query -f /project/username/ucvm_bin/conf/ucvm.conf -m cvmsi < /project/username/ucvm_bin/tests/inputs/test_latlons.txt

Terminal Message to User - Successful Installation Completes

Done installing UCVM!
Thank you for installing UCVM. 

Log out log back

%ucvm_query -H

== After Install Completes ==
*which ucvm_query’ to see if your environment is setup
*source conf/ucvm_env.sh script from either install directory or source directory
*ucvm_query -H
*installed_models.py
<pre>
(base) [maechlin@discovery2 ~]$ installed_models.py
[b'1d', b'bbp1d', b'cvms', b'cvmh', b'cencal', b'cvmsi', b'albacore', b'cvms5', b'cca', b'cs173', b'cs173h']

• Now, you can delete src directory if you do not plan to review, modify UCVM source code, or debug UCVM software. Deleting it saves about 150GB of disk storage if you were doing a complete install of all models.
• goto <install>/tests directory: CARC: /project/maechlin_162/ucvm_bin/tests
• ./run-testing

/project/maechlin_162/ucvm_bin/tests
(base) [maechlin@discovery2 tests]$ ./run-testing
Test: UCVM lib initialization
PASS
Test: UCVM lib add model 1D
PASS
Test: UCVM lib query 1D
PASS
Test: UCVM lib get model label 1D
PASS
Test: UCVM lib setparam querymode geo-depth 1D
PASS
Test: UCVM lib setparam querymode geo-elev 1D
PASS
Test: UCVM lib model version 1D
PASS
Test: UCVM lib add model USGS CenCal
PASS
Test: UCVM lib add model SCEC CVM-H
PASS
Test: UCVM lib add model SCEC CVM-S
PASS
Test: UCVM lib add model SCEC CVM-SI
PASS
Test: UCVM lib add model CVMS5
PASS
Test: UCVM lib add model CCA
PASS
Test: UCVM lib add model CS173
WARNING: Could not load model into memory. Reading the model from the
hard disk may result in slow performance.
PASS
Test: UCVM lib add model CS173H
WARNING: Could not load model into memory. Reading the model from the
hard disk may result in slow performance.
PASS
Test: ucvm library 1d model w/ large grid
PASS
Runnning test test_ssh_generate
[SUCCESS]
Runnning test test_vs30_query
[SUCCESS]
Running examples_api query_1d_gtl
[SUCCESS]
Running examples_programs_basin z2500 basin_query_cvmh
[SUCCESS]
Running examples_programs_basin z1000 basin_query_cvmh
[SUCCESS]
Running examples_programs_basin basin_query_mpi_cvms5
[SUCCESS]
Running examples_programs_basin basin_query_mpi_complete_cencal_cvms
[SUCCESS]
Running examples_programs_basin basin_query_mpi_complete_cencal_cvms5
[SUCCESS]
Running examples_programs_basin basin_query_mpi_complete_cencal_cvmsi
[SUCCESS]
Running examples_programs_ucvm ucvm_query_cvmh
[SUCCESS]
Running examples_programs_ucvm ucvm_query_cencal_cvms5
[SUCCESS]
Running examples_programs_ucvm ucvm_query_cencal_cvms
[SUCCESS]
Running examples_programs_ucvm2etree ucvm2etree_cvmh
[SUCCESS]
Running examples_programs_ucvm2mesh ucvm2mesh_cvmh
[SUCCESS]
Running examples_programs_mesh ucvm2mesh_mpi_cvmh
[SUCCESS]
Running examples_programs_mesh ucvm2mesh_mpi_cvmsi
[SUCCESS]
Running examples_programs_mesh ucvm2mesh_mpi_layer_cvmsi
[SUCCESS]
Running examples_programs_mesh ucvm2mesh_mpi_layer_cvms
[SUCCESS]

Testing Notes

• Testing on CARC discovery:
• Download onto /project filesystem - 20 minutes
• Compiled on headnode - 20 minutes
• Tests run on headnode - 20 minutes

Test Additions

Confirm each commmand line parameter

(base) [maechlin@discovery2 ~]$ ucvm_query -H
Usage: ucvm_query [-m models<:ifunc>] [-p user_map] [-c coordtype] [-f config] [-z zmin,zmax] [-b] < file.in

Flags:
	-h This help message.
	-H Detail help message.
	-m Comma delimited list of crustal/GTL models to query in order
	   of preference. GTL models may optionally be suffixed with ':ifunc'
	   to specify interpolation function.
	-c Z coordinate mode: geo-depth (gd, default), geo-elev (ge).
	-f Configuration file. Default is ./ucvm.conf.
	-p User-defined map to use for elevation and vs30 data.
	-v Display model version information only.
	-z Optional depth range for gtl/crust interpolation.

	-b Optional output in json format

	-l Optional input lon,lat,Z(depth/elevation)

Input format is:
	lon lat Z

Output format is:
	lon lat Z surf vs30 crustal cr_vp cr_vs cr_rho gtl gtl_vp gtl_vs gtl_rho cmb_algo cmb_vp cmb_vs cmb_rho

Notes:
	- If running interactively, type Cntl-D to end input coord list.

Version: 19.4.0

Installed Resources:
          1d : crustal model
       bbp1d : crustal model
    cmuetree : crustal model
       1dgtl : gtl
      elygtl : gtl
        cvms : crustal model
        cvmh : crustal model
      cencal : crustal model
       cvmsi : crustal model
    albacore : crustal model
       cvms5 : crustal model
         cca : crustal model
       cs173 : crustal model
      cs173h : crustal model
      linear : ifunc
         ely : ifunc
        ucvm : map
        yong : map
 model_etree : model i/f
 model_patch : model i/f
   map_etree : map i/f

Running UCVM on Discovery

This link provides detailed information about the USC CARC computing environment and information that might be needed to run UCVM, particularly large-scale UCVM queries, using USC CARC discovery.usc.edu system.

• Running UCVM on Discovery

Related Entries

• UCVM
• SCEC Home Page