UCVM Install
Contents
Overview of Installing UCVM on Discovery
1) Confirm USC CARC Discovery account and allocation access
2) Identify location of Storage on /project/scec_608/<username>
3) Confirm Access to USC CARC OnDemand for transferring files
- [[https://www.carc.usc.edu/user-information/user-guides/hpc-basics/getting-started-ondemand USC CARC OnDemand)
4) ssh into Discovery - Use keepalive to avoid frequent logouts
- ssh -o "ServerAliveInterval 60" username@discovery.usc.edu
5) select locations for 3 directories, working, src, and binary
- /home1/maechlin/test_ucvm
- /project/scec_608/maechlin/ucvm22_src
- /project/scec_608/maechlin/ucvm22
6) Install or Update anaconda in Discovery account
- test "which python"
- In not using conda, Install anaconda
- copy anaconda from my account on projects
- in test_ucvm_directory, run cp /project/scec_608/maechlin/Anaconda3-2022.05-Linux-x86_64.sh .
- chmod +x *.sh
- ./Anaconda3-2022.05-Linux-x86_64.sh
- when asked for installation directory give directory like
- /project/scec_608/maechlin/anaconda3
- If you have conda installed, run
- update conda update
7) Update .bashrc to specific required compilers, and add path to ucvm setup script (which is not yet installed)
8) logout and log in and confirm correct environment (python gcc)
9) in ucvm22_src clone dev branch of ucvm (contains all available models)
- git clone -b dev https://github.com/SCECcode/ucvm.git
9a.) move into largefiles and run get_largefiles, and stagelargefiles, and ucvm_setup.py
10) when prompted specify installation of binary directory for ucvm
- e.g. /project/scec_608/maechlin/ucvm22
11) when prompted, answer yes to all the velocity models available
12) log out/log in so UCVM environment is setup.
13) move to test_ucvm directory and confirm ucvm_query runs from test_ucvm, and ucvm_query -H lists all the velocity models
14) install conda python2 virtual environment and plotting libraries
15) move to ucvm22 bin directory and clone ucvm_plotting
16) run plotting installation
17) confirm plotting runs from test_ucvm directory
18) activate python2
19) run plotting examples script
20) transfer plots to local computer using OnDemand
Detailed Instructions for Installation on discovery.usc.edu
These instructions are a previous version, with somewhat more detail, for installating UCVM on Discovery. Some details listed here may have changed. Please contact software@scec.org if you have questions or problems.
0) confirm login and home directory /home1/<username> 1) create test_ucvm subdirectory for use later 3) confirm /project/scec_608/<username> exists 4) copy Anaconda Linux install script from /project/maechlin_162 5) run script and give installation directory as /project/scec_608/<username>/anaconda3 as install path 6) When it prompts to update .bashrc (or .bash_profile) say yes 7) Confirm .bashrc contains # >>> conda initialize >>> # !! Contents within this block are managed by 'conda init' !! __conda_setup="$('/project/scec_608/<username>/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" if [ $? -eq 0 ]; then eval "$__conda_setup" else if [ -f "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh" ]; then . "/project/scec_608/<username>/anaconda3/etc/profile.d/conda.sh" else export PATH="/project/scec_608/<username>/anaconda3/bin:$PATH" fi fi unset __conda_setup # <<< conda initialize <<< 8) add module unload/load command into .bash_profile module purge module load gcc/8.3.0 module load openmpi/4.0.2 module load pmix/3.1.3 # Setup UCVM Environment export LD_PRELOAD=/spack/apps/gcc/8.3.0/lib64/libstdc++.so.6 $ source /project/scec_608/<username>/ucvm_bin/conf/ucvm_env.sh 9) log out and log back in 10) confirm anaconda python is picked up $ which python 11) create /project/scec_608/<username>/dev directory 12) We will use the main branch of the git repo. CD into directory and type $ git clone https://github.com/SCECcode/ucvm.git This will create a subdirectory called ucvm. CD into that subdirectory. 13) cd into /project/scec_608/<username>/dev/ucvm Currently, we stage the largefiles externally, giving the user options on how they assemble the list of needed files. The script are designed to be run in the UCVM/largefiles dirctory First, the user runs a script to get the largefiles. $ cd largefiles User then run the following script. ./get_large_files.py This script asks user which model should be downloaded then the selected model files and other large files are copied from USC into the local directory. User should be selective about which models then install, because some of the models are several gigabytes in size. Please review the supported models list to identify the velocity models of interest. == Confirm the download of the models == Second, the user runs a script to confirm the largefiles are intact. This script prints out the number of OK files and the number of files with errors. If any files have errors, those files should be re-downloaded. ./check_largefiles_md5.py Third, the user runs the stage_files script: This copies the files from the largefiles directory into the ucvm directory tree ./stage_large_files.py Finally, the user moves out of the UCVMC/largefiles directory, into the UCVM home directory. From the UCVM/ home directory, the user runs: ./ucvm_setup.py This will run the configure, make, install commands to build the libraries and models. Set your environment variables set src directory as /project/scec_608/<username>/ucvm_src set bin directory as /project/scec_608/<username>/ucvm_bin Once you have set these environment variables, return to the UCVM source directory and type make check This will run the UCVM unit and acceptance tests. If all tests pass. UCVM is correctly installed and ready to use on your computer. To try out ucvm, once the tests pass, move to the UCVM installation directory, and run an example query. As an example, you can run ucvm_query: cd /project/scec_608/chukwueb/ucvm_bin ./bin/ucvm_query -f ./conf/ucvm.conf -m cvms < ./tests/inputs/test_latlons.txt You will then see the following output: Using Geo Depth coordinates as default mode. -118.0000 34.0000 0.000 280.896 390.000 cvms 696.491 213.000 1974.976 none 0.000 0.000 0.000 crust 696.491 213.000 1974.976 -118.0000 34.0000 50.000 280.896 390.000 cvms 1669.540 548.000 2128.620 none 0.000 0.000 0.000 crust 1669.540 548.000 2128.620 -118.0000 34.0000 100.000 280.896 390.000 cvms 1683.174 603.470 2130.773 none 0.000 0.000 0.000 crust 1683.174 603.470 2130.773 -118.0000 34.0000 500.000 280.896 390.000 cvms 3097.562 1656.495 2354.105 none 0.000 0.000 0.000 crust 3097.562 1656.495 2354.105 -118.0000 34.0000 1000.000 280.896 390.000 cvms 3660.809 2056.628 2443.042 none 0.000 0.000 0.000 crust 3660.809 2056.628 2443.042 Installation complete. Installation log file saved at ./setup_log.sh
15) Confirm entry in .bash_profile has Setup path env | grep ucvm env | grep UCVM
16) move to /home1/<username>/test_ucvm
list the installed modules
ucvm_query -H
ucvm_query with a list of lon-lat-depth in a file
ucvm_query -f /project/username/ucvm_bin/conf/ucvm.conf -m cvmsi < /project/username/ucvm_bin/tests/inputs/test_latlons.txt
Terminal Message to User - Successful Installation Completes
Done installing UCVM! Thank you for installing UCVM. Log out log back %ucvm_query -H == After Install Completes == *which ucvm_query’ to see if your environment is setup *source conf/ucvm_env.sh script from either install directory or source directory *ucvm_query -H *installed_models.py <pre> (base) [maechlin@discovery2 ~]$ installed_models.py [b'1d', b'bbp1d', b'cvms', b'cvmh', b'cencal', b'cvmsi', b'albacore', b'cvms5', b'cca', b'cs173', b'cs173h']
- Now, you can delete src directory if you do not plan to review, modify UCVM source code, or debug UCVM software. Deleting it saves about 150GB of disk storage if you were doing a complete install of all models.
- goto <install>/tests directory: CARC: /project/maechlin_162/ucvm_bin/tests
- ./run-testing
/project/maechlin_162/ucvm_bin/tests (base) [maechlin@discovery2 tests]$ ./run-testing Test: UCVM lib initialization PASS Test: UCVM lib add model 1D PASS Test: UCVM lib query 1D PASS Test: UCVM lib get model label 1D PASS Test: UCVM lib setparam querymode geo-depth 1D PASS Test: UCVM lib setparam querymode geo-elev 1D PASS Test: UCVM lib model version 1D PASS Test: UCVM lib add model USGS CenCal PASS Test: UCVM lib add model SCEC CVM-H PASS Test: UCVM lib add model SCEC CVM-S PASS Test: UCVM lib add model SCEC CVM-SI PASS Test: UCVM lib add model CVMS5 PASS Test: UCVM lib add model CCA PASS Test: UCVM lib add model CS173 WARNING: Could not load model into memory. Reading the model from the hard disk may result in slow performance. PASS Test: UCVM lib add model CS173H WARNING: Could not load model into memory. Reading the model from the hard disk may result in slow performance. PASS Test: ucvm library 1d model w/ large grid PASS Runnning test test_ssh_generate [SUCCESS] Runnning test test_vs30_query [SUCCESS] Running examples_api query_1d_gtl [SUCCESS] Running examples_programs_basin z2500 basin_query_cvmh [SUCCESS] Running examples_programs_basin z1000 basin_query_cvmh [SUCCESS] Running examples_programs_basin basin_query_mpi_cvms5 [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvms [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvms5 [SUCCESS] Running examples_programs_basin basin_query_mpi_complete_cencal_cvmsi [SUCCESS] Running examples_programs_ucvm ucvm_query_cvmh [SUCCESS] Running examples_programs_ucvm ucvm_query_cencal_cvms5 [SUCCESS] Running examples_programs_ucvm ucvm_query_cencal_cvms [SUCCESS] Running examples_programs_ucvm2etree ucvm2etree_cvmh [SUCCESS] Running examples_programs_ucvm2mesh ucvm2mesh_cvmh [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_cvmh [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_cvmsi [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_layer_cvmsi [SUCCESS] Running examples_programs_mesh ucvm2mesh_mpi_layer_cvms [SUCCESS]
Testing Notes
- Testing on CARC discovery:
- Download onto /project filesystem - 20 minutes
- Compiled on headnode - 20 minutes
- Tests run on headnode - 20 minutes
Test Additions
Confirm each commmand line parameter
(base) [maechlin@discovery2 ~]$ ucvm_query -H Usage: ucvm_query [-m models<:ifunc>] [-p user_map] [-c coordtype] [-f config] [-z zmin,zmax] [-b] < file.in Flags: -h This help message. -H Detail help message. -m Comma delimited list of crustal/GTL models to query in order of preference. GTL models may optionally be suffixed with ':ifunc' to specify interpolation function. -c Z coordinate mode: geo-depth (gd, default), geo-elev (ge). -f Configuration file. Default is ./ucvm.conf. -p User-defined map to use for elevation and vs30 data. -v Display model version information only. -z Optional depth range for gtl/crust interpolation. -b Optional output in json format -l Optional input lon,lat,Z(depth/elevation) Input format is: lon lat Z Output format is: lon lat Z surf vs30 crustal cr_vp cr_vs cr_rho gtl gtl_vp gtl_vs gtl_rho cmb_algo cmb_vp cmb_vs cmb_rho Notes: - If running interactively, type Cntl-D to end input coord list. Version: 19.4.0 Installed Resources: 1d : crustal model bbp1d : crustal model cmuetree : crustal model 1dgtl : gtl elygtl : gtl cvms : crustal model cvmh : crustal model cencal : crustal model cvmsi : crustal model albacore : crustal model cvms5 : crustal model cca : crustal model cs173 : crustal model cs173h : crustal model linear : ifunc ely : ifunc ucvm : map yong : map model_etree : model i/f model_patch : model i/f map_etree : map i/f
Running UCVM on Discovery
This link provides detailed information about the USC CARC computing environment and information that might be needed to run UCVM, particularly large-scale UCVM queries, using USC CARC discovery.usc.edu system.