OLCF Summit
From SCECpedia
Jump to navigationJump to searchThe following notes are for large mesh extraction on OLCF Summit.
Contents
Check available allocation
Command line for
- showusage
Configuration for large mesh extraction using UCVM
Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below:
Currently Loaded Modules: 1) lsf-tools/2.0 5) nsight-compute/2021.2.1 9) imagemagick/7.0.8-7 13) fftw/3.3.10 2) hsi/5.0.2.p5 6) nsight-systems/2021.3.1.54 10) python/3.6.10-anaconda3 3) xalt/1.2.1 7) cuda/11.0.3 11) gcc/9.1.0 4) DefApps 8) subversion/1.14.1 12) spectrum-mpi/10.4.0.3-20210112
Currently Loaded Modules: 1) lsf-tools/2.0 3) xalt/1.2.1 5) gcc/9.1.0 7) python/3.8-anaconda3 2) hsi/5.0.2.p5 4) DefApps 6) spectrum-mpi/10.4.0.3-20210112 8) fftw/3.3.10
50 mesh configuration
[pmaech@login5.summit ucvm_for_yifeng]$ more *.conf
# List of CVMs to query ucvmlist=cvmsi,elygtl:ely # UCVM conf file ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf # Gridding cell centered or vertex gridtype=CENTER # Spacing of cells spacing=50.0 # Projection proj=+proj=utm +datum=NAD27 +zone=11 rot=-27.5082989431 x0=-119.261301 y0=33.834327 z0=0.0 # Number of cells along each dim nx=6000 ny=2744 nz=1600 # Partitioning of grid among processors px=40 py=7 pz=4 # Vs/Vp minimum vp_min=710. vs_min=500. # Mesh and grid files, format meshfile=cvmsi50+elygtl.media gridfile=cvmsi50+elygtl.grid meshtype=IJK-12 # Location of scratch dir scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng
job submit script
#!/bin/bash #BSUB -P geo112 #BSUB -W 2:00 #BSUB -q debug #BSUB -nnodes 28 #BSUB -J cvmsi50Mesh #BSUB -oo cvmsi50Mesh.o #BSUB -oe cvmsi50Mesh.e #source ./ucvm_env.sh cd $LS_SUBCWD jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf date
