Difference between revisions of "OLCF Summit"
From SCECpedia
Jump to navigationJump to search| (2 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | The following notes are for large mesh extraction on OLCF Summit. | ||
| + | |||
| + | == Check available allocation == | ||
| + | Command line for | ||
| + | *showusage | ||
| + | |||
| + | == Configuration for large mesh extraction using UCVM == | ||
| + | Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below: | ||
| + | <pre> | ||
| + | Currently Loaded Modules: | ||
| + | 1) lsf-tools/2.0 5) nsight-compute/2021.2.1 9) imagemagick/7.0.8-7 13) fftw/3.3.10 | ||
| + | 2) hsi/5.0.2.p5 6) nsight-systems/2021.3.1.54 10) python/3.6.10-anaconda3 | ||
| + | 3) xalt/1.2.1 7) cuda/11.0.3 11) gcc/9.1.0 | ||
| + | 4) DefApps 8) subversion/1.14.1 12) spectrum-mpi/10.4.0.3-20210112 | ||
| + | </pre> | ||
| + | |||
| + | <pre> | ||
| + | Currently Loaded Modules: | ||
| + | 1) lsf-tools/2.0 3) xalt/1.2.1 5) gcc/9.1.0 7) python/3.8-anaconda3 | ||
| + | 2) hsi/5.0.2.p5 4) DefApps 6) spectrum-mpi/10.4.0.3-20210112 8) fftw/3.3.10 | ||
| + | </pre> | ||
| + | |||
| + | == 50 mesh configuration == | ||
| + | [pmaech@login5.summit ucvm_for_yifeng]$ more *.conf | ||
| + | <pre> | ||
| + | # List of CVMs to query | ||
| + | ucvmlist=cvmsi,elygtl:ely | ||
| + | # UCVM conf file | ||
| + | ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf | ||
| + | # Gridding cell centered or vertex | ||
| + | gridtype=CENTER | ||
| + | # Spacing of cells | ||
| + | spacing=50.0 | ||
| + | # Projection | ||
| + | proj=+proj=utm +datum=NAD27 +zone=11 | ||
| + | rot=-27.5082989431 | ||
| + | x0=-119.261301 | ||
| + | y0=33.834327 | ||
| + | z0=0.0 | ||
| + | # Number of cells along each dim | ||
| + | nx=6000 | ||
| + | ny=2744 | ||
| + | nz=1600 | ||
| + | # Partitioning of grid among processors | ||
| + | px=40 | ||
| + | py=7 | ||
| + | pz=4 | ||
| + | # Vs/Vp minimum | ||
| + | vp_min=710. | ||
| + | vs_min=500. | ||
| + | # Mesh and grid files, format | ||
| + | meshfile=cvmsi50+elygtl.media | ||
| + | gridfile=cvmsi50+elygtl.grid | ||
| + | meshtype=IJK-12 | ||
| + | # Location of scratch dir | ||
| + | scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng | ||
| + | </pre> | ||
| + | |||
| + | == job submit script == | ||
| + | <pre> | ||
| + | #!/bin/bash | ||
| + | #BSUB -P geo112 | ||
| + | #BSUB -W 2:00 | ||
| + | #BSUB -q debug | ||
| + | #BSUB -nnodes 28 | ||
| + | #BSUB -J cvmsi50Mesh | ||
| + | #BSUB -oo cvmsi50Mesh.o | ||
| + | #BSUB -oe cvmsi50Mesh.e | ||
| + | |||
| + | #source ./ucvm_env.sh | ||
| + | |||
| + | cd $LS_SUBCWD | ||
| + | jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf | ||
| + | date | ||
| + | </pre> | ||
| + | |||
== jsrun configuration tools == | == jsrun configuration tools == | ||
*https://code.ornl.gov/t4p/Hello_jsrun | *https://code.ornl.gov/t4p/Hello_jsrun | ||
*https://jobstepviewer.olcf.ornl.gov/ | *https://jobstepviewer.olcf.ornl.gov/ | ||
| + | |||
| + | == Related Entries == | ||
| + | *[[CME]] | ||
Latest revision as of 19:09, 12 April 2023
The following notes are for large mesh extraction on OLCF Summit.
Contents
Check available allocation
Command line for
- showusage
Configuration for large mesh extraction using UCVM
Two Summit users were able to build and run the UCVM 22.7 with taper. Their module lists are below:
Currently Loaded Modules: 1) lsf-tools/2.0 5) nsight-compute/2021.2.1 9) imagemagick/7.0.8-7 13) fftw/3.3.10 2) hsi/5.0.2.p5 6) nsight-systems/2021.3.1.54 10) python/3.6.10-anaconda3 3) xalt/1.2.1 7) cuda/11.0.3 11) gcc/9.1.0 4) DefApps 8) subversion/1.14.1 12) spectrum-mpi/10.4.0.3-20210112
Currently Loaded Modules: 1) lsf-tools/2.0 3) xalt/1.2.1 5) gcc/9.1.0 7) python/3.8-anaconda3 2) hsi/5.0.2.p5 4) DefApps 6) spectrum-mpi/10.4.0.3-20210112 8) fftw/3.3.10
50 mesh configuration
[pmaech@login5.summit ucvm_for_yifeng]$ more *.conf
# List of CVMs to query ucvmlist=cvmsi,elygtl:ely # UCVM conf file ucvmconf=/gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/conf/ucvm.conf # Gridding cell centered or vertex gridtype=CENTER # Spacing of cells spacing=50.0 # Projection proj=+proj=utm +datum=NAD27 +zone=11 rot=-27.5082989431 x0=-119.261301 y0=33.834327 z0=0.0 # Number of cells along each dim nx=6000 ny=2744 nz=1600 # Partitioning of grid among processors px=40 py=7 pz=4 # Vs/Vp minimum vp_min=710. vs_min=500. # Mesh and grid files, format meshfile=cvmsi50+elygtl.media gridfile=cvmsi50+elygtl.grid meshtype=IJK-12 # Location of scratch dir scratch=/gpfs/alpine/scratch/callag/geo112/ucvm_for_yifeng
job submit script
#!/bin/bash #BSUB -P geo112 #BSUB -W 2:00 #BSUB -q debug #BSUB -nnodes 28 #BSUB -J cvmsi50Mesh #BSUB -oo cvmsi50Mesh.o #BSUB -oe cvmsi50Mesh.e #source ./ucvm_env.sh cd $LS_SUBCWD jsrun -n 1120 -r 40 -a 1 -c 1 /gpfs/alpine/proj-shared/geo112/CyberShake/software/UCVM/ucvm-22.7.0/bin/ucvm2mesh_mpi -f cvmsi50.conf date
