Difference between revisions of "OLCF Summit"

From SCECpedia
Jump to navigationJump to search
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
The following notes are for large mesh extraction on OLCF Summit.
 +
 +
== Check available allocation ==
 +
Command line for
 +
*showusage
 +
 +
== Configuration for large mesh extraction using UCVM ==
 +
Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below:
 +
<pre>
 +
Currently Loaded Modules:
 +
  1) lsf-tools/2.0  5) nsight-compute/2021.2.1      9) imagemagick/7.0.8-7            13) fftw/3.3.10
 +
  2) hsi/5.0.2.p5    6) nsight-systems/2021.3.1.54  10) python/3.6.10-anaconda3
 +
  3) xalt/1.2.1      7) cuda/11.0.3                11) gcc/9.1.0
 +
  4) DefApps        8) subversion/1.14.1          12) spectrum-mpi/10.4.0.3-20210112
 +
</pre>
 +
 +
<pre>
 +
Currently Loaded Modules:
 +
  1) lsf-tools/2.0  3) xalt/1.2.1  5) gcc/9.1.0                        7) python/3.8-anaconda3
 +
  2) hsi/5.0.2.p5    4) DefApps      6) spectrum-mpi/10.4.0.3-20210112  8) fftw/3.3.10
 +
</pre>
 +
 +
== 50 mesh configuration ==
 +
[pmaech@login5.summit ucvm_for_yifeng]$ more *.conf
 +
<pre>
 +
# List of CVMs to query
 +
ucvmlist=cvmsi,elygtl:ely
 +
# UCVM conf file
 +
ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf
 +
# Gridding cell centered or vertex
 +
gridtype=CENTER
 +
# Spacing of cells
 +
spacing=50.0
 +
# Projection
 +
proj=+proj=utm +datum=NAD27 +zone=11
 +
rot=-27.5082989431
 +
x0=-119.261301
 +
y0=33.834327
 +
z0=0.0
 +
# Number of cells along each dim
 +
nx=6000
 +
ny=2744
 +
nz=1600
 +
# Partitioning of grid among processors
 +
px=40
 +
py=7
 +
pz=4
 +
# Vs/Vp minimum
 +
vp_min=710.
 +
vs_min=500.
 +
# Mesh and grid files, format
 +
meshfile=cvmsi50+elygtl.media
 +
gridfile=cvmsi50+elygtl.grid
 +
meshtype=IJK-12
 +
# Location of scratch dir
 +
scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng
 +
</pre>
 +
 +
== job submit script ==
 +
<pre>
 +
#!/bin/bash
 +
#BSUB -P geo112
 +
#BSUB -W 2:00
 +
#BSUB -q debug
 +
#BSUB -nnodes 28
 +
#BSUB -J cvmsi50Mesh
 +
#BSUB -oo cvmsi50Mesh.o
 +
#BSUB -oe cvmsi50Mesh.e
 +
 +
#source ./ucvm_env.sh
 +
 +
cd $LS_SUBCWD
 +
jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf
 +
date
 +
</pre>
 +
 
== jsrun configuration tools ==
 
== jsrun configuration tools ==
 
*https://code.ornl.gov/t4p/Hello_jsrun
 
*https://code.ornl.gov/t4p/Hello_jsrun
 
*https://jobstepviewer.olcf.ornl.gov/
 
*https://jobstepviewer.olcf.ornl.gov/
 +
 +
== Related Entries ==
 +
*[[CME]]

Latest revision as of 19:09, 12 April 2023

The following notes are for large mesh extraction on OLCF Summit.

Check available allocation

Command line for

  • showusage

Configuration for large mesh extraction using UCVM

Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below:

Currently Loaded Modules:
  1) lsf-tools/2.0   5) nsight-compute/2021.2.1      9) imagemagick/7.0.8-7             13) fftw/3.3.10
  2) hsi/5.0.2.p5    6) nsight-systems/2021.3.1.54  10) python/3.6.10-anaconda3
  3) xalt/1.2.1      7) cuda/11.0.3                 11) gcc/9.1.0
  4) DefApps         8) subversion/1.14.1           12) spectrum-mpi/10.4.0.3-20210112
Currently Loaded Modules:
  1) lsf-tools/2.0   3) xalt/1.2.1   5) gcc/9.1.0                        7) python/3.8-anaconda3
  2) hsi/5.0.2.p5    4) DefApps      6) spectrum-mpi/10.4.0.3-20210112   8) fftw/3.3.10

50 mesh configuration

[pmaech@login5.summit ucvm_for_yifeng]$ more *.conf

# List of CVMs to query
ucvmlist=cvmsi,elygtl:ely
# UCVM conf file
ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf
# Gridding cell centered or vertex
gridtype=CENTER
# Spacing of cells
spacing=50.0
# Projection
proj=+proj=utm +datum=NAD27 +zone=11
rot=-27.5082989431
x0=-119.261301
y0=33.834327
z0=0.0
# Number of cells along each dim
nx=6000
ny=2744
nz=1600
# Partitioning of grid among processors
px=40
py=7
pz=4
# Vs/Vp minimum
vp_min=710.
vs_min=500.
# Mesh and grid files, format
meshfile=cvmsi50+elygtl.media
gridfile=cvmsi50+elygtl.grid
meshtype=IJK-12
# Location of scratch dir
scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng

job submit script

#!/bin/bash
#BSUB -P geo112
#BSUB -W 2:00
#BSUB -q debug
#BSUB -nnodes 28
#BSUB -J cvmsi50Mesh
#BSUB -oo cvmsi50Mesh.o
#BSUB -oe cvmsi50Mesh.e

#source ./ucvm_env.sh

cd $LS_SUBCWD
jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf
date

jsrun configuration tools

Related Entries