Difference between revisions of "Ucvm2mesh-mpi"

From SCECpedia
Jump to navigationJump to search
 
(6 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
We are working on ucvm2mesh-mpi software to confirm it works in ucvmc
 
We are working on ucvm2mesh-mpi software to confirm it works in ucvmc
  
== Questions that we will resolve ==
+
== UCVM2mesh-mpi Questions ==
* ucvm2mesh-mpi documentation (look for configuraiton file)
+
# Locate ucvm2mesh-mpi documentation (look for configuration file)
* How can we set origin at center of region (rather than at lower left point)
+
## We determined there are some manual pages of ucvm2mesh and ucvm2mesh-mpi and posted example configuration files on ucvmc wiki
* What direction is the X axis and Y axis (confirm x is horizontal before rotation)
+
# Do the ucvm2mesh and ucvm2mesh-mpi produce the same results?
* Confirm that positive rotation is counter clockwise
+
## We have a test case that shows the outputs are identical
* Describe differences in output format mesh file versus grid file
+
#What direction is the X axis and Y axis (confirm x is horizontal before rotation)
* describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
+
## We have test cases that show the origin is expected to be the south west corner, and increasing X is in roughly east, assuming no rotation.
* Describe output file formats and locations
+
# Confirm that positive rotation is counter clockwise
 +
## We have a test case that shows this is true
 +
#Describe differences in output format mesh file versus grid file
 +
## The grid file is the x and y values, and the associated lat/lon values. The media file is one of four formats
 +
# Describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
 +
## The formats define number of bytes. Two of the formats include Qp and Qs values. ijk-32 includes the ijk values in the file
 +
#How can we set origin at center of region (rather than at lower left point)
 +
 
 +
== Installation and test process ==
 +
Notes on the installation and test process are here:
 +
*[[UCVMC MPI Testing]]
  
 
== small scale ucvm queries ==
 
== small scale ucvm queries ==
Line 21: Line 31:
  
 
== Example Usage ==
 
== Example Usage ==
 +
ucvm2mesh.conf file
 
<pre>
 
<pre>
 
 
# List of CVMs to query
 
# List of CVMs to query
 
ucvmlist=cvmh,bbp1d
 
ucvmlist=cvmh,bbp1d
Line 65: Line 75:
  
 
</pre>
 
</pre>
 +
 +
ucvm2mesh pbs file
 +
<pre>
 +
 +
-bash-4.2$ more *.pbs
 +
#!/bin/bash
 +
#
 +
# This is the pbs script ucvm2mesh
 +
#
 +
#PBS -l nodes=4:ppn=5
 +
#PBS -l walltime=00:10:00
 +
#PBS -m ae
 +
#PBS -M maechlin@usc.edu
 +
##PBS -q main
 +
#PBS -o $PBS_JOBID.out
 +
#PBS -e $PBS_JOBID.err
 +
cd $PBS_O_WORKDIR
 +
np=$(cat $PBS_NODEFILE | wc -l)
 +
cat $PBS_NODEFILE
 +
 +
mpirun -np $np -machinefile $PBS_NODEFILE /auto/scec-00/maechlin/t1/bin/ucvm2mesh-mpi -f /auto/scec-00/maechlin/ucvm_mpi/ucvm2mesh_2
 +
0cores.conf
 +
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.media $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.media
 +
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.grid  $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.grid
 +
exit
 +
</pre>
 +
 +
== Example: La Habra ==
 +
 +
*[[UCVMC La Habra mesh generation]]
  
 
== Dumpgrid.py Utility ==
 
== Dumpgrid.py Utility ==
  
 
* dump_grid.py utility reads in grid file and dumps the i j values, and the lat/lon values
 
* dump_grid.py utility reads in grid file and dumps the i j values, and the lat/lon values
<
 
 
<pre>
 
<pre>
 
./dumpgrid.py mesh_bbp1d_ijk12_2000m.grid 384 248
 
./dumpgrid.py mesh_bbp1d_ijk12_2000m.grid 384 248
Line 80: Line 119:
 
383, 247 : -112.458637, 33.788893
 
383, 247 : -112.458637, 33.788893
 
</pre>
 
</pre>
 +
 
== Links to Relevant Information ==
 
== Links to Relevant Information ==
 
*[http://scec.usc.edu/scecpedia/La_Habra_Simulation_Region La Habra Region Definition]
 
*[http://scec.usc.edu/scecpedia/La_Habra_Simulation_Region La Habra Region Definition]

Latest revision as of 20:15, 4 June 2018

We are working on ucvm2mesh-mpi software to confirm it works in ucvmc

UCVM2mesh-mpi Questions

  1. Locate ucvm2mesh-mpi documentation (look for configuration file)
    1. We determined there are some manual pages of ucvm2mesh and ucvm2mesh-mpi and posted example configuration files on ucvmc wiki
  2. Do the ucvm2mesh and ucvm2mesh-mpi produce the same results?
    1. We have a test case that shows the outputs are identical
  3. What direction is the X axis and Y axis (confirm x is horizontal before rotation)
    1. We have test cases that show the origin is expected to be the south west corner, and increasing X is in roughly east, assuming no rotation.
  4. Confirm that positive rotation is counter clockwise
    1. We have a test case that shows this is true
  5. Describe differences in output format mesh file versus grid file
    1. The grid file is the x and y values, and the associated lat/lon values. The media file is one of four formats
  6. Describe differences in output format for ijk-12, ijk-16, ijk-32 and SORD
    1. The formats define number of bytes. Two of the formats include Qp and Qs values. ijk-32 includes the ijk values in the file
  7. How can we set origin at center of region (rather than at lower left point)

Installation and test process

Notes on the installation and test process are here:

small scale ucvm queries

  • Define small scale sample problem
  • 20 km x 10km x 10k volume
  • 1km spacing
  • with rotation
  • without rotation
  • with ucvm2mesh
  • with ucvm2mesh-mpi

Example Usage

ucvm2mesh.conf file

# List of CVMs to query
ucvmlist=cvmh,bbp1d

# UCVM conf file
ucvmconf=/auto/scec-00/maechlin/t1/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=2000.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=11
rot=-40.0
x0=-122.3
y0=34.7835
z0=0.0

# Number of cells along each dim
nx=384
ny=248
nz=25

# Partitioning of grid among processors
px=2
py=2
pz=5

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.media
gridfile=/auto/scec-00/maechlin/ucvm_mpi/mesh_cvmhbbp1d_ijk12_2000m.grid
meshtype=IJK-12

# Location of scratch dir
scratch=/scratch

ucvm2mesh pbs file


-bash-4.2$ more *.pbs
#!/bin/bash
#
# This is the pbs script ucvm2mesh 
#
#PBS -l nodes=4:ppn=5
#PBS -l walltime=00:10:00
#PBS -m ae
#PBS -M maechlin@usc.edu
##PBS -q main
#PBS -o $PBS_JOBID.out
#PBS -e $PBS_JOBID.err
cd $PBS_O_WORKDIR
np=$(cat $PBS_NODEFILE | wc -l)
cat $PBS_NODEFILE

mpirun -np $np -machinefile $PBS_NODEFILE /auto/scec-00/maechlin/t1/bin/ucvm2mesh-mpi -f /auto/scec-00/maechlin/ucvm_mpi/ucvm2mesh_2
0cores.conf
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.media $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.media
#mv /scratch/maechlin/mesh_cvmh_ijk12_2000m.grid  $PBS_O_WORKDIR/mesh_cvmh_ijk12_200m.grid
exit

Example: La Habra

Dumpgrid.py Utility

  • dump_grid.py utility reads in grid file and dumps the i j values, and the lat/lon values
./dumpgrid.py mesh_bbp1d_ijk12_2000m.grid 384 248

produces

380, 247 : -112.506316, 33.825386
381, 247 : -112.490418, 33.813224
382, 247 : -112.474526, 33.801059
383, 247 : -112.458637, 33.788893

Links to Relevant Information

Related Entries