Difference between revisions of "UCVM install on Frontera"

From SCECpedia
Jump to navigationJump to search
Line 7: Line 7:
 
Modules to load,
 
Modules to load,
  
−
  module load gcc/13.2.0
+
<pre>
−
  module load hdf5/1.14.4
+
module load gcc/14.2.0
−
  module load sqlite/3.43.0
+
 
−
  #conflicted with gcc
+
module load phdf5/1.14.4
−
  #module load python3/3.9.2
+
module load sqlite/3.46.1
−
  module load automake
+
module load proj/9.3.1
−
  module load autoconf
+
 
−
  module load mkl
+
module load python3/3.11.8
 +
module load automake
 +
module load autoconf
 +
module load blis
 +
module load nvpl
 +
</pre>
 +
 
  
 
Because SFCVM/Geomodelgrids needs newer gcc module and python3 and that is not available on Frontera offering at this moment, so need to install anaconda for the python3 and some additional python pkgs.
 
Because SFCVM/Geomodelgrids needs newer gcc module and python3 and that is not available on Frontera offering at this moment, so need to install anaconda for the python3 and some additional python pkgs.
  
−
  Currently Loaded Modules:
+
<pre>
−
    1) git/2.24.1     4) hwloc/1.11.12   7) gcc/13.2.0   10) sqlite/3.43.0    13) mkl/23.1.0
+
Currently Loaded Modules:
−
    2) autotools/1.2   5) xalt/2.10.34    8) impi/21.9.0  11) automake/1.16.5
+
  1) ucc/1.3.0      5) TACC            9) phdf5/1.14.4   13) automake/1.16.5
−
    3) cmake/3.24.2    6) TACC            9) hdf5/1.14.4 12) autoconf/2.72
+
   2) ucx/1.17.0     6) tacc_tips/1.5  10) sqlite/3.46.1  14) autoconf/2.72
−
  Inactive Modules:
+
  3) cmake/3.31.5   7) gcc/14.2.0    11) proj/9.3.1      15) blis/1.0
−
    1) python3
+
  4) xalt/3.1      8) openmpi/5.0.5  12) python3/3.11.8 16) nvpl/24.7
 +
</pre>
  
 
=== script for setting up the UCVM/SW4 environment (setup-env)===
 
=== script for setting up the UCVM/SW4 environment (setup-env)===

Revision as of 05:21, 15 April 2025

UCVM with post-Geomodelgrid software stack

Frontera environment

This is for newer UCVM software stack with geomodelgrids support.

Modules to load, 

module load gcc/14.2.0

module load phdf5/1.14.4
module load sqlite/3.46.1
module load proj/9.3.1

module load python3/3.11.8
module load automake
module load autoconf
module load blis
module load nvpl


Because SFCVM/Geomodelgrids needs newer gcc module and python3 and that is not available on Frontera offering at this moment, so need to install anaconda for the python3 and some additional python pkgs. 

Currently Loaded Modules:
  1) ucc/1.3.0      5) TACC            9) phdf5/1.14.4    13) automake/1.16.5
  2) ucx/1.17.0     6) tacc_tips/1.5  10) sqlite/3.46.1   14) autoconf/2.72
  3) cmake/3.31.5   7) gcc/14.2.0     11) proj/9.3.1      15) blis/1.0
  4) xalt/3.1       8) openmpi/5.0.5  12) python3/3.11.8  16) nvpl/24.7

script for setting up the UCVM/SW4 environment (setup-env)

Note: should change setting for MY_TOP to match your directory setup 

#!/bin/bash

export MY_TOP=/home1/07508/mei/scratch1

export TOP_UCVM_TARGET=$MY_TOP/TARGET_UCVM_FRONTERA
export UCVM_SRC_PATH=$TOP_UCVM_TARGET/src
export UCVM_INSTALL_PATH=$TOP_UCVM_TARGET/ucvm_install

export ANACONDA3_TOP_DIR=$MY_TOP/anaconda3

# for python3.12
match="python3.12"
if [[ $PATH != *"$match"* ]]; then    
   export PATH=$ANACONDA3_TOP_DIR/bin:$PATH
   export PYTHONPATH=$ANACONDA3_TOP_DIR/lib/$match/site-packages
   echo "adding for "$match
   echo $PYTHONPATH
fi

export SW4_TOP_DIR=$MY_TOP/sw4

Script to build up anaconda

This should be called once only. 

#!/bin/bash
#
# install only if conda has not install yet
# 

source ~/setup_env

if [ -d $ANACONDA3_TOP_DIR ]; then
  echo "FOUND anaconda top"
  exit
fi


wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O anaconda.sh 
bash anaconda.sh -b -f -p $ANACONDA3_TOP_DIR
rm -f anaconda.sh
conda init bash

/bin/python -m pip install scipy h5py numpy pandas pybind11

Script to build up UCVM

#!/bin/bash

source ~/setup-env

rm -rf $TOP_UCVM_TARGET 
mkdir $TOP_UCVM_TARGET

cd $TOP_UCVM_TARGET
git clone https://github.com/SCECcode/ucvm.git -b testFrontera src


cd $UCVM_SRC_PATH/largefiles
./get_largefiles.py -m cvmsi,sfcvm

cd $UCVM_SRC_PATH/largefiles; ./check_largefiles_md5.py
cd $UCVM_SRC_PATH/largefiles; ./stage_largefiles.py

cd $UCVM_SRC_PATH
export PYTHONUNBUFFERED=TRUE
./ucvm_setup.py -d -a -p $UCVM_INSTALL_PATH &> ucvm_setup_install.log

cd $UCVM_SRC_PATH; make check

echo "..EXITING.."
exit

Script to build up sw4

source ~/setup-env

rm -rf $SW4_TOP_DIR
mkdir $SW4_TOP_DIR

cd $SW4_TOP_DIR
git clone https://github.com/meihuisu/sw4.git -b ucvm_cmd src 

cd $SW4_TOP_DIR/src
cd configs
ln -s make.ucvm_cmd.frontera  make.inc
cd $SW4_TOP_DIR/src
make

echo "..EXITING.."
exit

make.ucvm_cmd.frontera 

## FOR TACC Frontera

FC = gfortran
CXX = mpicxx

proj=yes
hdf5=yes
SW4ROOT = ${UCVM_INSTALL_PATH}/lib/proj
SQL3ROOT = ${UCVM_INSTALL_PATH}/lib/sqlite
HDF5ROOT = ${UCVM_INSTALL_PATH}/lib/hdf5

EXTRA_CXX_FLAGS  = -fopenmp -I$MKLROOT/include
EXTRA_FORT_FLAGS =
EXTRA_LINK_FLAGS = -L${UCVM_INSTALL_PATH}/lib/openssl/lib -lcrypto -lssl -Wl,-L${MKLROOT}/lib/intel64 -lmkl_in
tel_lp64 -lmkl_core -lmkl_gnu_thread -lpthread -lm -ldl -L${GCC_LIB} -lgfortran                               
                            
##MKLROOT /opt/intel/oneapi/mkl/2023.1.0

UCVM with pre-Geomodelgrid software stack

Frontera environment

Does not need any additional environment setup

Default Modules :

Currently Loaded Modules: 

 1) intel/19.1.1   3) git/2.24.1      5) python3/3.7.0   7) pmix/3.1.4      9) xalt/2.10.34
 2) impi/19.0.9    4) autotools/1.2   6) cmake/3.24.2    8) hwloc/1.11.12  10) TACC

Script to build UCVM

Copy the script below to your home directory.

Replace <<YOUR_BUILD_LOCATION>> in the script to point to your big scratch location(either scratch1 or scratch2) 


#!/bin/bash

export MY_TOP=<<YOUR_BUILD_LOCATION>>

export TOP_UCVM_TARGET=$MY_TOP/TARGET_UCVM
export UCVM_SRC_PATH=$TOP_UCVM_TARGET/ucvm
export UCVM_INSTALL_PATH=$TOP_UCVM_TARGET/ucvm_install

rm -rf $TOP_UCVM_TARGET
mkdir $TOP_UCVM_TARGET

cd $TOP_UCVM_TARGET
git clone https://github.com/SCECcode/ucvm.git -b withSCPBR


# all available velocity models in this UCVM branch
#
#cvmh,cencal,cca,cs173,cs173h,albacore,cvlsu,ivlsu,wfcvm
#cvms,cvms5,cvmsi
#cvmhibbn,cvmhlabn,cvmhrbn,cvmhsbbn,cvmhsbcbn,cvmhsgbn,cvmhsmbn,cvmhstbn,cvmhvbn
#sjcz
#

cd $UCVM_SRC_PATH/largefiles
# select the models to build/install
./get_largefiles.py -m cvmsi,cvms5,cvmh,cvms,cencal

cd $UCVM_SRC_PATH/largefiles; ./check_largefiles_md5.py
cd $UCVM_SRC_PATH/largefiles; ./stage_largefiles.py

cd $UCVM_SRC_PATH
./ucvm_setup.py -d -a -p $UCVM_INSTALL_PATH &> ucvm_setup_install.log

cd $UCVM_SRC_PATH; make check

echo "..EXITING.."
exit


Run the script to build your UCVM installation, 

chmod +x script
source ./script

Test your UCVM installation

setup your UCVM enabled environment, 

source <<YOUR_BUILD_LOCATION>>/TARGET_UCVM/ucvm_install/conf/ucvm_env.sh

Your $UCVM_INSTAlL_PATH is now set to "<<YOUR_BUILD_LOCATION>>/TARGET_UCVM/ucvm_install"
and your $UCVM_SRC_PATH is set to "<<YOUR_BUILD_LOCATION>>/TARGET_UCVM/ucvm"

To check if your ucvm environment is setup properly, 

which ucvm_query

Following command will show what is available in this UCVM installation, 

ucvm_query -H

and it should return, 

Usage: ucvm_query [-m models<:ifunc>] [-p user_map] [-c coordtype] [-f config] [-z zmin,zmax] [-b] < file.in

Flags:
	-h This help message.
	-H Detail help message.
	-m Comma delimited list of crustal/GTL models to query in order
	   of preference. GTL models may optionally be suffixed with ':ifunc'
	   to specify interpolation function.
	-c Z coordinate mode: geo-depth (gd, default), geo-elev (ge).
	-f Configuration file. Default is ./ucvm.conf.
	-p User-defined map to use for elevation and vs30 data.
	-v Display model version information only.
	-z Optional depth range for gtl/crust interpolation.

	-b Optional output in json format

	-l Optional input lat,lon,Z(depth/elevation)

	-L Optional interpolation floor limit vs,vp,density(meter in depth mode)

Input format is:
	lon lat Z

Output format is:
	lon lat Z surf vs30 crustal cr_vp cr_vs cr_rho gtl gtl_vp gtl_vs gtl_rho cmb_algo cmb_vp cmb_vs cmb_rho

Notes:
	- If running interactively, type Cntl-D to end input coord list.

Version: 22.7.0

Installed Resources:
          1d : crustal model
       bbp1d : crustal model
    cmuetree : crustal model
       1dgtl : gtl
      elygtl : gtl
        cvmh : crustal model
      cencal : crustal model
       cvms5 : crustal model
       cvmsi : crustal model
        cvms : crustal model
         ely : ifunc
       taper : ifunc
        ucvm : map
        yong : map
 model_etree : model i/f
 model_patch : model i/f
   map_etree : map i/fand it should return,


Using a test file from UCVM regression suite, 

-118.0 34.0 0.0
-118.0 34.0 50.0
-118.0 34.0 100.0
-118.0 34.0 500.0
-118.0 34.0 1000.0

Calling ucvm_query to query cvmsi model and interpolate it with the taper algorithm, 

ucvm_query -m cvmsi,elygtl:taper -f $UCVM_INSTALL_PATH/conf/ucvm.conf < $UCVM_INSTALL_PATH/tests/inputs/test_latlons.txt

should return, 

Using Geo Depth coordinates as default mode.
 -118.0000    34.0000      0.000    284.191    580.043      cvmsi   3035.433   1719.851   2402.684     elygtl      0.000    580.043      0.000      taper   1700.000    500.000   1974.976
 -118.0000    34.0000     50.000    284.191    580.043      cvmsi   3035.433   1719.851   2402.684     elygtl      0.000    580.043      0.000      taper   1700.000    548.000   2128.620
 -118.0000    34.0000    100.000    284.191    580.043      cvmsi   3035.433   1719.851   2402.684     elygtl      0.000    580.043      0.000      taper   1700.000    603.470   2130.773
 -118.0000    34.0000    500.000    284.191    580.043      cvmsi   3035.433   1719.851   2402.684     elygtl      0.000    580.043      0.000      taper   2701.194   1475.595   2354.105
 -118.0000    34.0000   1000.000    284.191    580.043      cvmsi   3330.891   1945.586   2443.042       none      0.000      0.000      0.000      crust   3330.891   1945.586   2443.042

Related Links

Retrieved from "https://strike.scec.org/scecwiki/index.php?title=UCVM_install_on_Frontera&oldid=30251"