Difference between revisions of "UCVM create new model with ucvm2mesh"

From SCECpedia
Jump to navigationJump to search
Line 148: Line 148:
 
the Slurm script,
 
the Slurm script,
 
<pre>
 
<pre>
 +
#!/bin/bash
 +
 +
#SBATCH -J norcal          # Job name
 +
#SBATCH -o norcal.oo      # Name of stdout output file
 +
#SBATCH -e norcal.ee      # Name of stderr error file
 +
#SBATCH -N 50              # Total # of nodes
 +
#SBATCH -n 1000
 +
#SBATCH -t 2:00:00          # Run time (hh:mm:ss)
 +
#SBATCH --mail-user=mei@usc.edu
 +
#SBATCH --mail-type=all
 +
#SBATCH -p batch
 +
#SBATCH -A geo156
 +
#SBATCH --open-mode=truncate
 +
 +
source /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm_env.sh
 +
 +
cp ${UCVM_INSTALL_PATH}/bin/ucvm2mesh_mpi .
 +
 +
srun -N50 -n1000 ./ucvm2mesh_mpi -f ./norcal_mesh.conf
 
</pre>
 
</pre>
  
 
the mesh conf script,
 
the mesh conf script,
 
<pre>
 
<pre>
 +
# List of CVMs to query
 +
ucvmlist=sfcvm,cca,nc1d
 +
 +
# UCVM conf file
 +
ucvmconf=/ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf
 +
 +
# Gridding cell centered or vertex
 +
gridtype=CENTER
 +
 +
# Spacing of cells
 +
spacing=100.0
 +
 +
# Projection
 +
proj=+proj=utm +datum=WGS84 +zone=10
 +
rot=-53.9
 +
x0=-121.9
 +
y0=36.3
 +
z0=0.0
 +
 +
# Number of cells along each dim (3000*1400*520=2,184,000,000)
 +
# x is mostly n-s,y is mostly e-w
 +
nx=3000
 +
ny=1400
 +
nz=520
 +
 +
# Partitioning of grid among processors (request px*py*pz processes in mpi submit) (8*8*8=512)
 +
# 3000/10 = 300 , 1400/10 = 140, 520/10 = 52 (10*10*10)
 +
# 3000/8 = 375 , 1400/8 = 175, 520/8 = 65
 +
px=10
 +
py=10
 +
pz=10
 +
 +
# Vs/Vp minimum
 +
vp_min=0
 +
vs_min=0
 +
 +
# Mesh and grid files, format
 +
meshfile=norcal_mesh
 +
gridfile=norcal_mesh.grid
 +
meshtype=SORD
 +
 +
# Location of scratch dir
 +
scratch=./scratch
 
</pre>
 
</pre>
  
 
the output log,
 
the output log,
 
<pre>
 
<pre>
 +
[0] /lustre/orion/geo156/scratch/mei/test_sfcvm_mesh/./ucvm2mesh_mpi Version: 22.7.0
 +
[0] Running on 1000 cores
 +
[0] Using config file ./norcal_mesh.conf
 +
[0] Configuration:
 +
[0] UCVM Model List: sfcvm,cca,nc1d
 +
[0] UCVM Conf file: /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf
 +
[0] Gridtype: 0
 +
[0] Querymode: 0
 +
[0] Spacing: 100.000000
 +
[0] Projection: +proj=utm +datum=WGS84 +zone=10
 +
[0] Rotation Angle: -53.900000
 +
[0] Origin x0,y0,z0: -121.900000, 36.300000, 0.000000
 +
[0] Dimensions: 3000, 1400, 520
 +
[0] Proc Dimensions: 10, 10, 10
 +
[0] Vp Min: 0.000000, Vs Min: 0.000000
 +
[0] Mesh File: norcal_mesh
 +
[0] Grid File: norcal_mesh.grid
 +
[0] Mesh Type: 4
 +
[0] Scratch Dir: ./scratch
 +
[0] Initialization complete
 +
[0] Converting grid to latlong
 +
[0] Grid generation complete
 +
[0] Configuring UCVM
 +
[740] Allocating 42000 grid points
 +
[740] Partition dimensions: 300 x 140 x 52
 +
[740] I,J,K start: 0, 560, 364
 +
[740] I,J,K end: 300, 700, 416
 +
...
 +
[800] Extracted 2184000 points
 +
[860] Extracted 2184000 points
 +
[820] Extracted 2184000 points
 +
[0] Max Vp: 8451.263672 at
 +
[0] i,j,k : 1636, 739, 475
 +
[0] Max Vs: 5028.872559 at
 +
[0] i,j,k : 2528, 1134, 305
 +
[0] Max Rho: 3456.249756 at
 +
[0] i,j,k : 2528, 1134, 305
 +
[0] Min Vp: 718.195862 at
 +
[0] i,j,k : 0, 433, 0
 +
[0] Min Vs: 84.268837 at
 +
[0] i,j,k : 0, 433, 0
 +
[0] Min Rho: 1600.008789 at
 +
[0] i,j,k : 0, 429, 0
 +
[0] Min Ratio: 1.460094 at
 +
[0] i,j,k : 2987, 610, 0
 
</pre>
 
</pre>
  
Line 179: Line 286:
 
</pre>
 
</pre>
  
−
convert to 4 corners in geo,
+
convert to 4 corners to umm,
 +
code is in cs242/src/latlon2utm
 +
input format
 +
<pre>
 +
  0, lon, lat
 +
  1, utm_e, utm_n
 +
</pre>
 +
 
 
<pre>
 
<pre>
 +
lon(-121.899223)lat(36.299894) => utm_e(598928.315878),utm_n(4017582.107443)
 +
lon(-120.013908)lat(34.083659) => utm_e(775627.918078),utm_n(3775266.509192)
 +
lon(-120.618201)lat(37.024137) => utm_e(711965.823647),utm_n(4100010.510239)
 +
lon(-118.753975)lat(34.789165) => utm_e(888665.421967),utm_n(3857694.945671)
 
</pre>
 
</pre>
  

Revision as of 06:46, 6 May 2024

mesh data

Select a region of interest with existing cvm models,

Generate vp/vs/rho data files using ucvm2mesh

config file for ucvm2mesh 

 ucvm2mesh -f southbay_sfcvm.conf > southbay_sfcvm.out

Origin is at the left top, and going down in fast-Y axis in writing out the binary data 

# List of CVMs to query
ucvmlist=sfcvm

# UCVM conf file
ucvmconf=/var/www/html/UCVM_web_sfcvm_mei/web/model/UCVM_TARGET/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=20.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=10
rot=-39.9
x0=-121.8384
y0=37.2533
z0=0.0

# Number of cells along each dim
nx=140
ny=70
nz=60

# Partitioning of grid among processors
px=2
py=2
pz=2

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=southbay_sfcvm.media
gridfile=southbay_sfcvm.grid
meshtype=SORD

# Location of scratch dir
scratch=./scratch

Output: 

[0] Using config file southbay_sfcvm.conf
[0] Configuration:
	[0] UCVM Model List: sfcvm
	[0] UCVM Conf file: /var/www/html/UCVM_web_sfcvm_mei/web/model/UCVM_TARGET/conf/ucvm.conf
	[0] Gridtype: 0
	[0] Querymode: 0
	[0] Spacing: 20.000000
	[0] Projection: +proj=utm +datum=WGS84 +zone=10
		[0] Rotation Angle: -39.900000
		[0] Origin x0,y0,z0: -121.838400, 37.253300, 0.000000
		[0] Dimensions: 140, 70, 60
	[0] Proc Dimensions: 2, 2, 2
	[0] Vp Min: 0.000000, Vs Min: 0.000000
	[0] Mesh File: southbay_sfcvm.media
	[0] Grid File: southbay_sfcvm.grid
	[0] Mesh Type: 4
	[0] Scratch Dir: ./scratch
Setting up UCVM
Generating 2D grid
Converting grid to latlong
Grid generation complete
Allocating 9800 grid points
Mesh dimensions: 140 x 70 x 60
Reading grid points
Grid 4 corners:
  -121.838241 37.253310 (0)
  -121.814448 37.237001 (139)
  -121.828113 37.262752 (9660)
  -121.804320 37.246441 (9799)
Extracted slice 0 (9800 pnts) in 248.13 ms, 39495.742188 pps
...

setup config for ucvm model

NOTE: nx and ny needs to be swapped from the ucvm2mesh's parameters 

# UTM Zone
utm_zone = 10

# Model directory, for 6 iterations
model_dir = cs242
model_data_path  = https://g-c662a6.a78b8.36fe.data.globus.org/ucvm/models

# Number of cells in x, y, and z.
nx = 140
ny = 70
nz = 60

interpolation = on

# Depth, in meters
depth = 1200
depth_interval = 20

# Corners, in UTM projection.
#-121.838241,37.253310,0
#-121.814448,37.237001,0
#-121.828113,37.262752,0
#-121.804320,37.246441,0
#
#lon(-121.838241)lat(37.253310) => utm_e(603123.693099),utm_n(4123409.463672)
#lon(-121.814448)lat(37.237001) => utm_e(605256.447077),utm_n(4121626.242860)
#lon(-121.828113)lat(37.262752) => utm_e(604008.906310),utm_n(4124468.090587)
#lon(-121.804320)lat(37.246441) => utm_e(606141.579119),utm_n(4122684.881493)

top_left_corner_e = 603123.693099
top_left_corner_n = 4123409.463672

bottom_left_corner_e    = 605256.447077
bottom_left_corner_n    = 4121626.242860

top_right_corner_e      = 604008.906310
top_right_corner_n      = 4124468.090587

bottom_right_corner_e   = 606141.579119
bottom_right_corner_n   = 4122684.881493

# Data Layout Parameters 
seek_axis = fast-Y
seek_direction = top-down


An example: using ucvm2mesh_mpi at Frontier

the Slurm script, 

#!/bin/bash

#SBATCH -J norcal          # Job name
#SBATCH -o norcal.oo       # Name of stdout output file
#SBATCH -e norcal.ee       # Name of stderr error file
#SBATCH -N 50              # Total # of nodes 
#SBATCH -n 1000
#SBATCH -t 2:00:00           # Run time (hh:mm:ss)
#SBATCH --mail-user=mei@usc.edu
#SBATCH --mail-type=all
#SBATCH -p batch
#SBATCH -A geo156
#SBATCH --open-mode=truncate

source /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm_env.sh

cp ${UCVM_INSTALL_PATH}/bin/ucvm2mesh_mpi .

srun -N50 -n1000 ./ucvm2mesh_mpi -f ./norcal_mesh.conf

the mesh conf script, 

# List of CVMs to query
ucvmlist=sfcvm,cca,nc1d

# UCVM conf file
ucvmconf=/ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=100.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=10
rot=-53.9
x0=-121.9
y0=36.3
z0=0.0

# Number of cells along each dim (3000*1400*520=2,184,000,000)
# x is mostly n-s,y is mostly e-w
nx=3000
ny=1400
nz=520

# Partitioning of grid among processors (request px*py*pz processes in mpi submit) (8*8*8=512)
# 3000/10 = 300 , 1400/10 = 140, 520/10 = 52 (10*10*10)
# 3000/8 = 375 , 1400/8 = 175, 520/8 = 65
px=10
py=10
pz=10

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=norcal_mesh
gridfile=norcal_mesh.grid
meshtype=SORD

# Location of scratch dir
scratch=./scratch

the output log, 

[0] /lustre/orion/geo156/scratch/mei/test_sfcvm_mesh/./ucvm2mesh_mpi Version: 22.7.0
[0] Running on 1000 cores
[0] Using config file ./norcal_mesh.conf
[0] Configuration:
	[0] UCVM Model List: sfcvm,cca,nc1d
	[0] UCVM Conf file: /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf
	[0] Gridtype: 0
	[0] Querymode: 0
	[0] Spacing: 100.000000
	[0] Projection: +proj=utm +datum=WGS84 +zone=10
		[0] Rotation Angle: -53.900000
		[0] Origin x0,y0,z0: -121.900000, 36.300000, 0.000000
		[0] Dimensions: 3000, 1400, 520
	[0] Proc Dimensions: 10, 10, 10
	[0] Vp Min: 0.000000, Vs Min: 0.000000
	[0] Mesh File: norcal_mesh
	[0] Grid File: norcal_mesh.grid
	[0] Mesh Type: 4
	[0] Scratch Dir: ./scratch
[0] Initialization complete
[0] Converting grid to latlong
[0] Grid generation complete
[0] Configuring UCVM
[740] Allocating 42000 grid points
[740] Partition dimensions: 300 x 140 x 52
[740] I,J,K start: 0, 560, 364
[740] I,J,K end: 300, 700, 416
...
[800] Extracted 2184000 points
[860] Extracted 2184000 points
[820] Extracted 2184000 points
[0] Max Vp: 8451.263672 at
[0]	i,j,k : 1636, 739, 475
[0] Max Vs: 5028.872559 at
[0]	i,j,k : 2528, 1134, 305
[0] Max Rho: 3456.249756 at
[0]	i,j,k : 2528, 1134, 305
[0] Min Vp: 718.195862 at
[0]	i,j,k : 0, 433, 0
[0] Min Vs: 84.268837 at
[0]	i,j,k : 0, 433, 0
[0] Min Rho: 1600.008789 at
[0]	i,j,k : 0, 429, 0
[0] Min Ratio: 1.460094 at
[0]	i,j,k : 2987, 610, 0

Next: process for 4 corners

in $UCVM_SRC_PATH/src/ucvm2mesh 

make grid4corners

run, 

grid4corners norcal_mesh.grid 3000 1400

output, 

Grid 4 corners:
  -121.899223 36.299894 (0)
  -120.013908 34.083659 (2999)
  -120.618201 37.024137 (4197000)
  -118.753975 34.789165 (4199999)

convert to 4 corners to umm, code is in cs242/src/latlon2utm input format 

   0, lon, lat
   1, utm_e, utm_n

lon(-121.899223)lat(36.299894) => utm_e(598928.315878),utm_n(4017582.107443)
lon(-120.013908)lat(34.083659) => utm_e(775627.918078),utm_n(3775266.509192)
lon(-120.618201)lat(37.024137) => utm_e(711965.823647),utm_n(4100010.510239)
lon(-118.753975)lat(34.789165) => utm_e(888665.421967),utm_n(3857694.945671)

Compose model data/config, 



Pack up a model tar ball,








Retrieved from "https://strike.scec.org/scecwiki/index.php?title=UCVM_create_new_model_with_ucvm2mesh&oldid=28605"