OLCF Summit
From SCECpedia
Jump to navigationJump to searchThe following notes are for large mesh extraction on OLCF Summit.
Contents
Configuration for large mesh extraction using UCVM
Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below:
Currently Loaded Modules: 1) lsf-tools/2.0 5) nsight-compute/2021.2.1 9) imagemagick/7.0.8-7 13) fftw/3.3.10 2) hsi/5.0.2.p5 6) nsight-systems/2021.3.1.54 10) python/3.6.10-anaconda3 3) xalt/1.2.1 7) cuda/11.0.3 11) gcc/9.1.0 4) DefApps 8) subversion/1.14.1 12) spectrum-mpi/10.4.0.3-20210112
Currently Loaded Modules: 1) lsf-tools/2.0 3) xalt/1.2.1 5) gcc/9.1.0 7) python/3.8-anaconda3 2) hsi/5.0.2.p5 4) DefApps 6) spectrum-mpi/10.4.0.3-20210112 8) fftw/3.3.10
50 mesh configuration
[pmaech@login5.summit ucvm_for_yifeng]$ more *.conf
# List of CVMs to query ucvmlist=cvmsi,elygtl:ely # UCVM conf file ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf # Gridding cell centered or vertex gridtype=CENTER # Spacing of cells spacing=50.0 # Projection proj=+proj=utm +datum=NAD27 +zone=11 rot=-27.5082989431 x0=-119.261301 y0=33.834327 z0=0.0 # Number of cells along each dim nx=6000 ny=2744 nz=1600 # Partitioning of grid among processors px=40 py=7 pz=4 # Vs/Vp minimum vp_min=710. vs_min=500. # Mesh and grid files, format meshfile=cvmsi50+elygtl.media gridfile=cvmsi50+elygtl.grid meshtype=IJK-12 # Location of scratch dir scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng
job submit script
#!/bin/bash #BSUB -P geo112 #BSUB -W 2:00 #BSUB -q debug #BSUB -nnodes 28 #BSUB -J cvmsi50Mesh #BSUB -oo cvmsi50Mesh.o #BSUB -oe cvmsi50Mesh.e #source ./ucvm_env.sh cd $LS_SUBCWD jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf date
