Difference between revisions of "Ucvm2mesh-mpi"

From SCECpedia
Jump to navigationJump to search
 
(9 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
We are working on ucvm2mesh-mpi software to confirm it works in ucvmc
 
We are working on ucvm2mesh-mpi software to confirm it works in ucvmc
  
−
== Questions that we will resolve ==
+
== UCVM2mesh-mpi Questions ==
−
* ucvm2mesh-mpi documentation (look for configuraiton file)
+
# Locate ucvm2mesh-mpi documentation (look for configuration file)
−
* How can we set origin at center of region (rather than at lower left point)
+
## We determined there are some manual pages of ucvm2mesh and ucvm2mesh-mpi and posted example configuration files on ucvmc wiki
−
* What direction is the X axis and Y axis (confirm x is horizontal before rotation)
+
# Do the ucvm2mesh and ucvm2mesh-mpi produce the same results?
−
* Confirm that positive rotation is counter clockwise
+
## We have a test case that shows the outputs are identical
−
* Describe differences in output format mesh file versus grid file
+
#What direction is the X axis and Y axis (confirm x is horizontal before rotation)
−
* describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
+
## We have test cases that show the origin is expected to be the south west corner, and increasing X is in roughly east, assuming no rotation.
−
* Describe output file formats and locations
+
# Confirm that positive rotation is counter clockwise
 +
## We have a test case that shows this is true
 +
#Describe differences in output format mesh file versus grid file
 +
## The grid file is the x and y values, and the associated lat/lon values. The media file is one of four formats
 +
# Describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
 +
## The formats define number of bytes. Two of the formats include Qp and Qs values. ijk-32 includes the ijk values in the file
 +
#How can we set origin at center of region (rather than at lower left point)
 +
 
 +
== Installation and test process ==
 +
Notes on the installation and test process are here:
 +
*[[UCVMC MPI Testing]]
  
 
== small scale ucvm queries ==
 
== small scale ucvm queries ==
Line 19: Line 29:
 
* with ucvm2mesh
 
* with ucvm2mesh
 
* with ucvm2mesh-mpi
 
* with ucvm2mesh-mpi
 +
 +
== Example Usage ==
 +
ucvm2mesh.conf file
 +
<pre>
 +
# List of CVMs to query
 +
ucvmlist=cvmh,bbp1d
 +
 +
# UCVM conf file
 +
ucvmconf=/auto/scec-00/maechlin/t1/conf/ucvm.conf
 +
 +
# Gridding cell centered or vertex
 +
gridtype=CENTER
 +
 +
# Spacing of cells
 +
spacing=2000.0
 +
 +
# Projection
 +
proj=+proj=utm +datum=WGS84 +zone=11
 +
rot=-40.0
 +
x0=-122.3
 +
y0=34.7835
 +
z0=0.0
 +
 +
# Number of cells along each dim
 +
nx=384
 +
ny=248
 +
nz=25
 +
 +
# Partitioning of grid among processors
 +
px=2
 +
py=2
 +
pz=5
 +
 +
# Vs/Vp minimum
 +
vp_min=0
 +
vs_min=0
 +
 +
# Mesh and grid files, format
 +
meshfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.media
 +
gridfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.grid
 +
meshtype=IJK-12
 +
 +
# Location of scratch dir
 +
scratch=/scratch
 +
 +
</pre>
 +
 +
ucvm2mesh pbs file
 +
<pre>
 +
 +
-bash-4.2$ more *.pbs
 +
#!/bin/bash
 +
#
 +
# This is the pbs script ucvm2mesh
 +
#
 +
#PBS -l nodes=4:ppn=5
 +
#PBS -l walltime=00:10:00
 +
#PBS -m ae
 +
#PBS -M maechlin@usc.edu
 +
##PBS -q main
 +
#PBS -o $PBS_JOBID.out
 +
#PBS -e $PBS_JOBID.err
 +
cd $PBS_O_WORKDIR
 +
np=$(cat $PBS_NODEFILE | wc -l)
 +
cat $PBS_NODEFILE
 +
 +
mpirun -np $np -machinefile $PBS_NODEFILE /auto/scec-00/maechlin/t1/bin/ucvm2mesh-mpi -f /auto/scec-00/maechlin/ucvm_mpi/ucvm2mesh_2
 +
0cores.conf
 +
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.media $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.media
 +
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.grid  $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.grid
 +
exit
 +
</pre>
 +
 +
== Example: La Habra ==
 +
 +
*[[UCVMC La Habra mesh generation]]
 +
 +
== Dumpgrid.py Utility ==
 +
 +
* dump_grid.py utility reads in grid file and dumps the i j values, and the lat/lon values
 +
<pre>
 +
./dumpgrid.py mesh_bbp1d_ijk12_2000m.grid 384 248
 +
</pre>
 +
produces
 +
<pre>
 +
380, 247 : -112.506316, 33.825386
 +
381, 247 : -112.490418, 33.813224
 +
382, 247 : -112.474526, 33.801059
 +
383, 247 : -112.458637, 33.788893
 +
</pre>
  
 
== Links to Relevant Information ==
 
== Links to Relevant Information ==
 
*[http://scec.usc.edu/scecpedia/La_Habra_Simulation_Region La Habra Region Definition]
 
*[http://scec.usc.edu/scecpedia/La_Habra_Simulation_Region La Habra Region Definition]
 
* UCVM2Mesh documentation
 
* UCVM2Mesh documentation
 +
*[https://github.com/SCECcode/UCVMC/wiki/Manual-Pages ucvm2mesh manual pages]
 +
*[https://github.com/SCECcode/UCVMC/wiki UCVMC Wiki Page]
  
 
== Related Entries ==
 
== Related Entries ==
 
*[[UCVM]]
 
*[[UCVM]]

Latest revision as of 20:15, 4 June 2018

We are working on ucvm2mesh-mpi software to confirm it works in ucvmc

UCVM2mesh-mpi Questions

  1. Locate ucvm2mesh-mpi documentation (look for configuration file)
    1. We determined there are some manual pages of ucvm2mesh and ucvm2mesh-mpi and posted example configuration files on ucvmc wiki
  2. Do the ucvm2mesh and ucvm2mesh-mpi produce the same results?
    1. We have a test case that shows the outputs are identical
  3. What direction is the X axis and Y axis (confirm x is horizontal before rotation)
    1. We have test cases that show the origin is expected to be the south west corner, and increasing X is in roughly east, assuming no rotation.
  4. Confirm that positive rotation is counter clockwise
    1. We have a test case that shows this is true
  5. Describe differences in output format mesh file versus grid file
    1. The grid file is the x and y values, and the associated lat/lon values. The media file is one of four formats
  6. Describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
    1. The formats define number of bytes. Two of the formats include Qp and Qs values. ijk-32 includes the ijk values in the file
  7. How can we set origin at center of region (rather than at lower left point)

Installation and test process

Notes on the installation and test process are here:

small scale ucvm queries

  • Define small scale sample problem
  • 20 km x 10km x 10k volume
  • 1km spacing
  • with rotation
  • without rotation
  • with ucvm2mesh
  • with ucvm2mesh-mpi

Example Usage

ucvm2mesh.conf file 

# List of CVMs to query
ucvmlist=cvmh,bbp1d

# UCVM conf file
ucvmconf=/auto/scec-00/maechlin/t1/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=2000.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=11
rot=-40.0
x0=-122.3
y0=34.7835
z0=0.0

# Number of cells along each dim
nx=384
ny=248
nz=25

# Partitioning of grid among processors
px=2
py=2
pz=5

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.media
gridfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.grid
meshtype=IJK-12

# Location of scratch dir
scratch=/scratch

ucvm2mesh pbs file 


-bash-4.2$ more *.pbs
#!/bin/bash
#
# This is the pbs script ucvm2mesh 
#
#PBS -l nodes=4:ppn=5
#PBS -l walltime=00:10:00
#PBS -m ae
#PBS -M maechlin@usc.edu
##PBS -q main
#PBS -o $PBS_JOBID.out
#PBS -e $PBS_JOBID.err
cd $PBS_O_WORKDIR
np=$(cat $PBS_NODEFILE | wc -l)
cat $PBS_NODEFILE

mpirun -np $np -machinefile $PBS_NODEFILE /auto/scec-00/maechlin/t1/bin/ucvm2mesh-mpi -f /auto/scec-00/maechlin/ucvm_mpi/ucvm2mesh_2
0cores.conf
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.media $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.media
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.grid  $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.grid
exit

Example: La Habra

Dumpgrid.py Utility

  • dump_grid.py utility reads in grid file and dumps the i j values, and the lat/lon values
./dumpgrid.py mesh_bbp1d_ijk12_2000m.grid 384 248

produces 

380, 247 : -112.506316, 33.825386
381, 247 : -112.490418, 33.813224
382, 247 : -112.474526, 33.801059
383, 247 : -112.458637, 33.788893

Links to Relevant Information

Related Entries

Retrieved from "https://strike.scec.org/scecwiki/index.php?title=Ucvm2mesh-mpi&oldid=20930"