View source for UCVM create new model with ucvm2mesh

== goal ==

How to add a new model into UCVM using ucvm2mesh

== background ==

ucvm2mesh generates mesh with upper-left corner as origin with seeking_axis in fast-Y mode going from
upper left to lower-left corners. This is the same as using the ucvm web tool to generate a mesh.

The confusion part is nx is number of steps in the seeking_axis direction and so this example actually looks like

                                70 o
                                *
                                 140  o
                                    o
 
       

the generic model plugin code is assuming the mesh origin is at the lower-left corner and going from
lower-left to upper left corner as fast-Y mode. 

Although Cypershake uses upper-left corner as origin during its processing, the mesh it created to be included
into UCVM has an additional translation such that it origin is at the lower-left corner

Steps:

    create a mesh using ucvm2mesh
    translate data file 
    import into UCVM with the generic model plugin code

== generate mesh data ==

Select a region of interest with existing cvm models,

Generate vp/vs/rho data files using ucvm2mesh

config file for ucvm2mesh

  ucvm2mesh -f southbay_sfcvm.conf > southbay_sfcvm.out 

Origin is at the upper-, and going down in fast-Y axis in writing out
the binary data

<pre>
# List of CVMs to query
ucvmlist=sfcvm

# UCVM conf file
ucvmconf=/var/www/html/UCVM_web_sfcvm_mei/web/model/UCVM_TARGET/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=20.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=10
rot=-39.9
x0=-121.8384
y0=37.2533
z0=0.0

# Number of cells along each dim
nx=140
ny=70
nz=60

# Partitioning of grid among processors
px=2
py=2
pz=2

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=southbay_sfcvm
gridfile=southbay_sfcvm.grid
meshtype=SORD

# Location of scratch dir
scratch=./scratch
</pre>

Output:

<pre>
[0] Using config file southbay_sfcvm.conf
[0] Configuration:
	[0] UCVM Model List: sfcvm
	[0] UCVM Conf file: /var/www/html/UCVM_web_sfcvm_mei/web/model/UCVM_TARGET/conf/ucvm.conf
	[0] Gridtype: 0
	[0] Querymode: 0
	[0] Spacing: 20.000000
	[0] Projection: +proj=utm +datum=WGS84 +zone=10
		[0] Rotation Angle: -39.900000
		[0] Origin x0,y0,z0: -121.838400, 37.253300, 0.000000
		[0] Dimensions: 140, 70, 60
	[0] Proc Dimensions: 2, 2, 2
	[0] Vp Min: 0.000000, Vs Min: 0.000000
	[0] Mesh File: southbay_sfcvm.media
	[0] Grid File: southbay_sfcvm.grid
	[0] Mesh Type: 4
	[0] Scratch Dir: ./scratch
Setting up UCVM
Generating 2D grid
Converting grid to latlong
Grid generation complete
Allocating 9800 grid points
Mesh dimensions: 140 x 70 x 60
Reading grid points
Grid 4 corners:
  -121.838241 37.253310 (0)
  -121.814448 37.237001 (139)
  -121.828113 37.262752 (9660)
  -121.804320 37.246441 (9799)
Extracted slice 0 (9800 pnts) in 248.13 ms, 39495.742188 pps
...
</pre>

== extract 4 corners ==

using ucvm2mesh,  4 latlon corners are available from the stderr output

for any other means,

UCVM's grid4corner on the grid file and dimension will return the corners

<pre>
./grid4corners southbay_sfcvm.grid 140 70
</pre>

returns

<pre>
Grid 4 corners:
  -121.838241 37.253310 (0)
  -121.814448 37.237001 (139)
  -121.828113 37.262752 (9660)
  -121.804320 37.246441 (9799)
</pre>

the location of the 4 corners are

 upper-left
 lower-left
 upper-right
 lower-right

== 4 corners in utm ==

prepare input file

<pre>
0  -121.838241 37.253310 
0  -121.814448 37.237001 
0  -121.828113 37.262752 
0  -121.804320 37.246441 
</pre>

using a utm converter in cs242/src, 

<pre>
latlon2utm < in
</pre>

and output
<pre>
lon(-121.838241)lat(37.253310) => utm_e(70891.768031),utm_n(4133755.798077)
lon(-121.814448)lat(37.237001) => utm_e(72911.428832),utm_n(4131836.847914)
lon(-121.828113)lat(37.262752) => utm_e(71844.378668),utm_n(4134758.273205)
lon(-121.804320)lat(37.246441) => utm_e(73863.908699),utm_n(4132839.336733)
</pre>

can double check by extract the utm_e and utm_n out from the output, prepend with 1 and call latlon2utm and
the result should match the original latlon data

== setup config for ucvm model ==


NOTE:  nx and ny needs to be swapped from the ucvm2mesh's parameters

<pre>
# UTM Zone
utm_zone = 10

# Model directory, for 6 iterations
model_dir = cs242
model_data_path  = https://g-c662a6.a78b8.36fe.data.globus.org/ucvm/models

# Number of cells in x, y, and z.
nx = 140
ny = 70
nz = 60

interpolation = on

# Depth, in meters
depth = 1200
depth_interval = 20

# Corners, in UTM projection.
#-121.838241,37.253310,0
#-121.814448,37.237001,0
#-121.828113,37.262752,0
#-121.804320,37.246441,0
#
#lon(-121.838241)lat(37.253310) => utm_e(603123.693099),utm_n(4123409.463672)
#lon(-121.814448)lat(37.237001) => utm_e(605256.447077),utm_n(4121626.242860)
#lon(-121.828113)lat(37.262752) => utm_e(604008.906310),utm_n(4124468.090587)
#lon(-121.804320)lat(37.246441) => utm_e(606141.579119),utm_n(4122684.881493)

top_left_corner_e = 603123.693099
top_left_corner_n = 4123409.463672

bottom_left_corner_e    = 605256.447077
bottom_left_corner_n    = 4121626.242860

top_right_corner_e      = 604008.906310
top_right_corner_n      = 4124468.090587

bottom_right_corner_e   = 606141.579119
bottom_right_corner_n   = 4122684.881493

# Data Layout Parameters 
seek_axis = fast-Y
seek_direction = top-down
</pre>

== An example: using ucvm2mesh_mpi at Frontier ==

the Slurm script,
<pre>
#!/bin/bash

#SBATCH -J norcal          # Job name
#SBATCH -o norcal.oo       # Name of stdout output file
#SBATCH -e norcal.ee       # Name of stderr error file
#SBATCH -N 50              # Total # of nodes 
#SBATCH -n 1000
#SBATCH -t 2:00:00           # Run time (hh:mm:ss)
#SBATCH --mail-user=mei@usc.edu
#SBATCH --mail-type=all
#SBATCH -p batch
#SBATCH -A geo156
#SBATCH --open-mode=truncate

source /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm_env.sh

cp ${UCVM_INSTALL_PATH}/bin/ucvm2mesh_mpi .

srun -N50 -n1000 ./ucvm2mesh_mpi -f ./norcal_mesh.conf
</pre>

the mesh conf script,
<pre>
# List of CVMs to query
ucvmlist=sfcvm,cca,nc1d

# UCVM conf file
ucvmconf=/ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf

# Gridding cell centered or vertex
gridtype=CENTER

# Spacing of cells
spacing=100.0

# Projection
proj=+proj=utm +datum=WGS84 +zone=10
rot=-53.9
x0=-121.9
y0=36.3
z0=0.0

# Number of cells along each dim (3000*1400*520=2,184,000,000)
# x is mostly n-s,y is mostly e-w
nx=3000
ny=1400
nz=520

# Partitioning of grid among processors (request px*py*pz processes in mpi submit) (8*8*8=512)
# 3000/10 = 300 , 1400/10 = 140, 520/10 = 52 (10*10*10)
# 3000/8 = 375 , 1400/8 = 175, 520/8 = 65
px=10
py=10
pz=10

# Vs/Vp minimum
vp_min=0
vs_min=0

# Mesh and grid files, format
meshfile=norcal_mesh
gridfile=norcal_mesh.grid
meshtype=SORD

# Location of scratch dir
scratch=./scratch
</pre>

the output log,
<pre>
[0] /lustre/orion/geo156/scratch/mei/test_sfcvm_mesh/./ucvm2mesh_mpi Version: 22.7.0
[0] Running on 1000 cores
[0] Using config file ./norcal_mesh.conf
[0] Configuration:
	[0] UCVM Model List: sfcvm,cca,nc1d
	[0] UCVM Conf file: /ccs/home/mei/scratch/TARGET_UCVM_SFCVM_mei/ucvm_install/conf/ucvm.conf
	[0] Gridtype: 0
	[0] Querymode: 0
	[0] Spacing: 100.000000
	[0] Projection: +proj=utm +datum=WGS84 +zone=10
		[0] Rotation Angle: -53.900000
		[0] Origin x0,y0,z0: -121.900000, 36.300000, 0.000000
		[0] Dimensions: 3000, 1400, 520
	[0] Proc Dimensions: 10, 10, 10
	[0] Vp Min: 0.000000, Vs Min: 0.000000
	[0] Mesh File: norcal_mesh
	[0] Grid File: norcal_mesh.grid
	[0] Mesh Type: 4
	[0] Scratch Dir: ./scratch
[0] Initialization complete
[0] Converting grid to latlong
[0] Grid generation complete
[0] Configuring UCVM
[740] Allocating 42000 grid points
[740] Partition dimensions: 300 x 140 x 52
[740] I,J,K start: 0, 560, 364
[740] I,J,K end: 300, 700, 416
...
[800] Extracted 2184000 points
[860] Extracted 2184000 points
[820] Extracted 2184000 points
[0] Max Vp: 8451.263672 at
[0]	i,j,k : 1636, 739, 475
[0] Max Vs: 5028.872559 at
[0]	i,j,k : 2528, 1134, 305
[0] Max Rho: 3456.249756 at
[0]	i,j,k : 2528, 1134, 305
[0] Min Vp: 718.195862 at
[0]	i,j,k : 0, 433, 0
[0] Min Vs: 84.268837 at
[0]	i,j,k : 0, 433, 0
[0] Min Rho: 1600.008789 at
[0]	i,j,k : 0, 429, 0
[0] Min Ratio: 1.460094 at
[0]	i,j,k : 2987, 610, 0
</pre>

Next: process for 4 corners

in $UCVM_SRC_PATH/src/ucvm2mesh

<pre>
make grid4corners
</pre>

run,
<pre>
grid4corners norcal_mesh.grid 3000 1400
</pre>
output,
<pre>
Grid 4 corners:
  -121.899223 36.299894 (0)
  -120.013908 34.083659 (2999)
  -120.618201 37.024137 (4197000)
  -118.753975 34.789165 (4199999)
</pre>

convert to 4 corners to umm,
code is in cs242/src/latlon2utm
using
<pre>    
    snprintf(_projstr, 64, "+proj=utm +zone=%d +datum=NAD27 +units=m +no_defs", 10);
    if (!(_geo2utm = proj_create_crs_to_crs(PJ_DEFAULT_CTX, "EPSG:4326", _projstr, NULL))) {
        fprintf(stderr,"%s\n",(char  *)proj_context_errno_string(PJ_DEFAULT_CTX, proj_context_errno(PJ_DEFAULT_CTX)));
        return (1);
    }
</pre>

input format :
   0, lon, lat
   1, utm_e, utm_n

<pre>
0 -121.899223 36.299894
0 -120.013908 34.083659
0 -120.618201 37.024137
0 -118.753975 34.789165
</pre>

<pre>
lon(-121.899223)lat(36.299894) => utm_e(598928.315878),utm_n(4017582.107443)
lon(-120.013908)lat(34.083659) => utm_e(775627.918078),utm_n(3775266.509192)
lon(-120.618201)lat(37.024137) => utm_e(711965.823647),utm_n(4100010.510239)
lon(-118.753975)lat(34.789165) => utm_e(888665.421967),utm_n(3857694.945671)
</pre>

Compose model data/config,
<pre>

#
# Phil's norcal_ucvm2mesh
#
# UTM Zone -- sfcvm,cca,nc1d
utm_zone = 10

# Model directory, for 6 iterations
model_dir = cs242
model_data_path  = https://g-c662a6.a78b8.36fe.data.globus.org/ucvm/models

#proj=+proj=utm +datum=WGS84 +zone=10
#rot=-53.9
#x0=-121.9
#y0=36.3
#z0=0.0

# Number of cells in x, y, and z.
#         tr
#      tl 
#
#            br
#         bl  
#
#  x is from top-left to bottom-left 
nx = 3000
ny = 1400 
nz = 520

interpolation = on

# Depth, in meters
depth = 52000
depth_interval =  100

#Grid 4 corners:
#  -121.899223 36.299894 (0)
#  -120.013908 34.083659 (2999)
#  -120.618201 37.024137 (4197000)
#  -118.753975 34.789165 (4199999)
#
#lon(-121.899223)lat(36.299894) => utm_e(598928.315878),utm_n(4017582.107443)
#lon(-120.013908)lat(34.083659) => utm_e(775627.918078),utm_n(3775266.509192)
#lon(-120.618201)lat(37.024137) => utm_e(711965.823647),utm_n(4100010.510239)
#lon(-118.753975)lat(34.789165) => utm_e(888665.421967),utm_n(3857694.945671)

top_left_corner_e = 598928.315878
top_left_corner_n = 4017582.107443
bottom_left_corner_e = 775627.918078
bottom_left_corner_n = 3775266.509192
top_right_corner_e = 711965.823647
top_right_corner_n = 4100010.510239
bottom_right_corner_e = 888665.421967
bottom_right_corner_n = 3857694.94567

# Data Layout Parameters 
seek_axis = fast-Y
seek_direction = top-down

</pre>

Pack up a model tar ball,
<pre>
</pre>