UCVM Install
Contents
- 1 Background Infor and Resources
- 2 Overview of Installing UCVM on Discovery
- 3 Phase 2 - Install Plotting Scripts
- 4 Recording of UCVM Installation Process
- 5 Detailed Instructions for Installation on discovery.usc.edu
- 6 Terminal Message to User - Successful Installation Completes
- 7 Testing Notes
- 8 Test Additions
- 9 Running UCVM on Discovery
- 10 Related Entries
Background Infor and Resources
Confirm User is Added to SCEC Allocation: Create /project Storage /project/scec_608/<username>
CARC System Overview (hardware and queues):
Storage Areas:
Intro to scientific computing:
- https://software-carpentry.org/lessons/
- https://swcarpentry.github.io/shell-novice/
- https://www.tutorialspoint.com/how-to-use-nano-text-editor
Starting a Project group at CARC:
Overview of Installing UCVM on Discovery
1) Confirm USC CARC Discovery account and allocation access
2) Identify location of Storage on /project/scec_608/<username>
3) Confirm Access to USC CARC OnDemand for transferring files
- [[https://www.carc.usc.edu/user-information/user-guides/hpc-basics/getting-started-ondemand USC CARC OnDemand)
4) ssh into Discovery - Use keepalive to avoid frequent logouts
- ssh -o "ServerAliveInterval 60" username@discovery2.usc.edu
5) select locations for 3 directories, working, src, and binary
- /home1/maechlin/test_ucvm
- /project/scec_608/maechlin/ucvm227_src
- /project/scec_608/maechlin/ucvm227
6) Install or Update anaconda in Discovery account
- test "which python"
- In not using conda, Install anaconda
- copy anaconda from my account on projects
- in test_ucvm_directory, run cp /project/scec_608/maechlin/Anaconda3-2022.10-Linux-x86_64.sh .
- chmod +x *.sh
- ./Anaconda3-2022.10-Linux-x86_64.sh
- when asked for installation directory give directory like
- /project/scec_608/maechlin/anaconda3
- If you have conda installed, run
- update conda update
7) Update .bashrc to specific required compilers, and add path to ucvm setup script (which is not yet installed) module purge module load gcc/8.3.0 module load openmpi/4.0.2 module load pmix/3.1.3
- Setup UCVM Environment
export LD_PRELOAD=/spack/apps/gcc/8.3.0/lib64/libstdc++.so.6 source /project/scec_608/<username>/ucvm227/conf/ucvm_env.sh
8) logout and log in and confirm correct environment (python gcc)
9) in ucvm227_src clone dev branch of ucvm (contains all available models)
- git clone https://github.com/SCECcode/ucvm.git
9a.) move into largefiles and run get_largefiles.py, and stage_largefiles.py, and then ucvm_setup.py
10) when prompted specify installation of binary directory for ucvm
- e.g. /project/scec_608/maechlin/ucvm227
11) when prompted, answer yes to all the velocity models available
12) log out/log in so UCVM environment is setup.
13) move to test_ucvm directory and confirm ucvm_query runs from test_ucvm, and ucvm_query -H lists all the velocity models
Phase 2 - Install Plotting Scripts
14.1) Check install conda envs
conda env list
14) install conda python2 virtual environment and plotting libraries
conda create -n python2 python=2.7.15 scipy pip numpy matplotlib basemap basemap-data-hires
15) move to ucvm227 bin directory and clone ucvm_plotting
- git clone https://github.com/ucvm_plotting.git
16) run plotting installation
17) confirm plotting runs from test_ucvm directory
18) activate python2 Add this command to end of .bashrc file
- conda activate python2
19) run plotting examples script
20) transfer plots to local computer using OnDemand
Recording of UCVM Installation Process
Detailed Instructions for Installation on discovery.usc.edu
These instructions are a previous version, with somewhat more detail, for installating UCVM on Discovery. Some details listed here may have changed. Please contact software@scec.org if you have questions or problems.
0) confirm login and home directory /home1/<username>
1) create test_ucvm subdirectory for use later
3) confirm /project/scec_608/<username> exists
4) copy Anaconda Linux install script from /project/maechlin_162
5) run script and give installation directory as
/project/scec_608/<username>/anaconda3 as install path
6) When it prompts to update .bashrc (or .bash_profile) say yes
7) Confirm .bashrc contains
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/project/scec_608/<username>/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh" ]; then
. "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/project/scec_608/<username>/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
8) add module unload/load command into .bash_profile
module purge
module load gcc/8.3.0
module load openmpi/4.0.2
module load pmix/3.1.3
# Setup UCVM Environment
export LD_PRELOAD=/spack/apps/gcc/8.3.0/lib64/libstdc++.so.6
$ source /project/scec_608/<username>/ucvm_bin/conf/ucvm_env.sh
9) log out and log back in
10) confirm anaconda python is picked up
$ which python
11) create /project/scec_608/<username>/dev directory
12) We will use the main branch of the git repo. CD into directory and type
$ git clone https://github.com/SCECcode/ucvm.git
This will create a subdirectory called ucvm. CD into that subdirectory.
13) cd into /project/scec_608/<username>/dev/ucvm
Currently, we stage the largefiles externally, giving the user options on how they
assemble the list of needed files. The script are designed to be run in the UCVM/largefiles dirctory
First, the user runs a script to get the largefiles.
$ cd largefiles
User then run the following script.
./get_large_files.py
This script asks user which model should be downloaded then the selected model files and other large files are copied from USC into the local directory.
User should be selective about which models then install, because some of the models are several gigabytes in size. Please review the supported models list to identify the velocity models of interest.
== Confirm the download of the models ==
Second, the user runs a script to confirm the largefiles are intact.
This script prints out the number of OK files and the number of files with errors.
If any files have errors, those files should be re-downloaded.
./check_largefiles_md5.py
Third, the user runs the stage_files script:
This copies the files from the largefiles directory into the ucvm directory tree
./stage_large_files.py
Finally, the user moves out of the UCVMC/largefiles directory, into the UCVM home directory.
From the UCVM/ home directory, the user runs:
./ucvm_setup.py
This will run the configure, make, install commands to build the libraries and models.
Set your environment variables
set src directory as /project/scec_608/<username>/ucvm_src
set bin directory as /project/scec_608/<username>/ucvm_bin
Once you have set these environment variables, return to the UCVM source directory and type
make check
This will run the UCVM unit and acceptance tests. If all tests pass. UCVM is correctly installed
and ready to use on your computer.
To try out ucvm, once the tests pass, move to the UCVM installation directory, and run an example query.
As an example, you can run ucvm_query:
cd /project/scec_608/chukwueb/ucvm_bin
./bin/ucvm_query -f ./conf/ucvm.conf -m cvms < ./tests/inputs/test_latlons.txt
You will then see the following output:
Using Geo Depth coordinates as default mode.
-118.0000 34.0000 0.000 280.896 390.000 cvms 696.491 213.000 1974.976 none 0.000 0.000 0.000 crust 696.491 213.000 1974.976
-118.0000 34.0000 50.000 280.896 390.000 cvms 1669.540 548.000 2128.620 none 0.000 0.000 0.000 crust 1669.540 548.000 2128.620
-118.0000 34.0000 100.000 280.896 390.000 cvms 1683.174 603.470 2130.773 none 0.000 0.000 0.000 crust 1683.174 603.470 2130.773
-118.0000 34.0000 500.000 280.896 390.000 cvms 3097.562 1656.495 2354.105 none 0.000 0.000 0.000 crust 3097.562 1656.495 2354.105
-118.0000 34.0000 1000.000 280.896 390.000 cvms 3660.809 2056.628 2443.042 none 0.000 0.000 0.000 crust 3660.809 2056.628 2443.042
Installation complete. Installation log file saved at ./setup_log.sh
15) Confirm entry in .bash_profile has Setup path env | grep ucvm env | grep UCVM
16) move to /home1/<username>/test_ucvm
list the installed modules
ucvm_query -H
ucvm_query with a list of lon-lat-depth in a file
ucvm_query -f /project/username/ucvm_bin/conf/ucvm.conf -m cvmsi < /project/username/ucvm_bin/tests/inputs/test_latlons.txt
Terminal Message to User - Successful Installation Completes
Done installing UCVM! Thank you for installing UCVM. Log out log back %ucvm_query -H == After Install Completes == *which ucvm_query’ to see if your environment is setup *source conf/ucvm_env.sh script from either install directory or source directory *ucvm_query -H *installed_models.py <pre> (base) [maechlin@discovery2 ~]$ installed_models.py [b'1d', b'bbp1d', b'cvms', b'cvmh', b'cencal', b'cvmsi', b'albacore', b'cvms5', b'cca', b'cs173', b'cs173h']
- Now, you can delete src directory if you do not plan to review, modify UCVM source code, or debug UCVM software. Deleting it saves about 150GB of disk storage if you were doing a complete install of all models.
- goto <install>/tests directory: CARC: /project/maechlin_162/ucvm_bin/tests
- ./run-testing
/project/maechlin_162/ucvm_bin/tests (base) [maechlin@discovery2 tests]$ ./run-testing Test: UCVM lib initialization PASS Test: UCVM lib add model 1D PASS Test: UCVM lib query 1D PASS Test: UCVM lib get model label 1D PASS Test: UCVM lib setparam querymode geo-depth 1D PASS Test: UCVM lib setparam querymode geo-elev 1D PASS Test: UCVM lib model version 1D PASS Test: UCVM lib add model USGS CenCal PASS Test: UCVM lib add model SCEC CVM-H PASS Test: UCVM lib add model SCEC CVM-S PASS Test: UCVM lib add model SCEC CVM-SI PASS Test: UCVM lib add model CVMS5 PASS Test: UCVM lib add model CCA PASS Test: UCVM lib add model CS173 WARNING: Could not load model into memory. Reading the model from the hard disk may result in slow performance. PASS Test: UCVM lib add model CS173H WARNING: Could not load model into memory. Reading the model from the hard disk may result in slow performance. PASS Test: ucvm library 1d model w/ large grid PASS Runnning test test_ssh_generate [SUCCESS] Runnning test test_vs30_query [SUCCESS] Running examples_api query_1d_gtl [SUCCESS] Running examples_programs_basin z2500 basin_query_cvmh [SUCCESS] Running examples_programs_basin z1000 basin_query_cvmh [SUCCESS] Running examples_programs_basin basin_query_mpi_cvms5 [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvms [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvms5 [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvmsi [SUCCESS] Running examples_programs_ucvm ucvm_query_cvmh [SUCCESS] Running examples_programs_ucvm ucvm_query_cencal_cvms5 [SUCCESS] Running examples_programs_ucvm ucvm_query_cencal_cvms [SUCCESS] Running examples_programs_ucvm2etree ucvm2etree_cvmh [SUCCESS] Running examples_programs_ucvm2mesh ucvm2mesh_cvmh [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_cvmh [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_cvmsi [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_layer_cvmsi [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_layer_cvms [SUCCESS]
Testing Notes
- Testing on CARC discovery:
- Download onto /project filesystem - 20 minutes
- Compiled on headnode - 20 minutes
- Tests run on headnode - 20 minutes
Test Additions
Confirm each commmand line parameter
(base) [maechlin@discovery2 ~]$ ucvm_query -H
Usage: ucvm_query [-m models<:ifunc>] [-p user_map] [-c coordtype] [-f config] [-z zmin,zmax] [-b] < file.in
Flags:
-h This help message.
-H Detail help message.
-m Comma delimited list of crustal/GTL models to query in order
of preference. GTL models may optionally be suffixed with ':ifunc'
to specify interpolation function.
-c Z coordinate mode: geo-depth (gd, default), geo-elev (ge).
-f Configuration file. Default is ./ucvm.conf.
-p User-defined map to use for elevation and vs30 data.
-v Display model version information only.
-z Optional depth range for gtl/crust interpolation.
-b Optional output in json format
-l Optional input lon,lat,Z(depth/elevation)
Input format is:
lon lat Z
Output format is:
lon lat Z surf vs30 crustal cr_vp cr_vs cr_rho gtl gtl_vp gtl_vs gtl_rho cmb_algo cmb_vp cmb_vs cmb_rho
Notes:
- If running interactively, type Cntl-D to end input coord list.
Version: 19.4.0
Installed Resources:
1d : crustal model
bbp1d : crustal model
cmuetree : crustal model
1dgtl : gtl
elygtl : gtl
cvms : crustal model
cvmh : crustal model
cencal : crustal model
cvmsi : crustal model
albacore : crustal model
cvms5 : crustal model
cca : crustal model
cs173 : crustal model
cs173h : crustal model
linear : ifunc
ely : ifunc
ucvm : map
yong : map
model_etree : model i/f
model_patch : model i/f
map_etree : map i/f
Running UCVM on Discovery
This link provides detailed information about the USC CARC computing environment and information that might be needed to run UCVM, particularly large-scale UCVM queries, using USC CARC discovery.usc.edu system.
