Difference between revisions of "Product Backlog - Mei"
From SCECpedia
Jump to navigationJump to search| Line 4: | Line 4: | ||
#CFM,gather and define the functions needed for the leaflet CFM viewers and study api and start building up the viewer utilities | #CFM,gather and define the functions needed for the leaflet CFM viewers and study api and start building up the viewer utilities | ||
#CFM, update CFM db/tb schema after science group fix the excel page and resolve typo/conflicts/missing data | #CFM, update CFM db/tb schema after science group fix the excel page and resolve typo/conflicts/missing data | ||
| − | #learn the tools used in https://ds.iris.edu/ds/products/emc-desktoptools/. paraview for desktop | + | #UCVMC,learn the tools used in https://ds.iris.edu/ds/products/emc-desktoptools/. paraview for desktop |
| − | #refine sop & doc steps for generating ucvm.e | + | #UCVMC,refine sop & doc steps for generating ucvm.e |
| − | #tied up loose ends in UCVMC18.5 and test | + | #UCVMC,tied up loose ends in UCVMC18.5 and test |
#need to make UCVMC work with python3(Scott said bw is stern about it), mainly plotting and config/build scripts | #need to make UCVMC work with python3(Scott said bw is stern about it), mainly plotting and config/build scripts | ||
| − | #Discuss mesh plotting Scripts with Scott Cal. | + | #UCVMC,Discuss mesh plotting Scripts with Scott Cal. |
DONE | DONE | ||
| Line 15: | Line 15: | ||
#CFM, translate shapesfiles from release tarball into postgis db sql script and ingest into backend db | #CFM, translate shapesfiles from release tarball into postgis db sql script and ingest into backend db | ||
#CFM, explore and code a leaflet viewer with canned geoJSON data | #CFM, explore and code a leaflet viewer with canned geoJSON data | ||
| − | #CFM, code php/java scripts/html page for queries with user input:query by region (with on the fly region list extracted from backend db via php), query by a range of strike value(with on the fly min/max strike range, extracted from the backend db via php), query by keyword(full text search from postgres db on object name, | + | #CFM, code php/java scripts/html page for queries with user input:query by region (with on the fly region list extracted from backend db via php), query by a range of strike value(with on the fly min/max strike range, extracted from the backend db via php), query by keyword(full text search from postgres db on object name, can be expanded in the future per request); query by latlon (within 0.0001 precision, a small roi box is created and used postgis to make overlap check) |
| − | can be expanded in the future per request); query by latlon (within | ||
| − | 0.0001 precision, a small roi box is created and used postgis to | ||
| − | make overlap check) | ||
#CFM, preprocess the excel from CFM5_release_2017, define db/tb schema and ingest the information into postgres db. | #CFM, preprocess the excel from CFM5_release_2017, define db/tb schema and ingest the information into postgres db. | ||
#UCVMC,generate the tiled v1000 data that Kevin needed for https://github.com/kevinmilner/cybershake-analysis/tree/master/study_18_8/hazard_maps#study-188-hazard-maps, they are left with https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Bay_Area_Velocity_Model | #UCVMC,generate the tiled v1000 data that Kevin needed for https://github.com/kevinmilner/cybershake-analysis/tree/master/study_18_8/hazard_maps#study-188-hazard-maps, they are left with https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Bay_Area_Velocity_Model | ||
#CFM,study requirements for work on SCEC Community Fault Model,start with postgres db, postgis extension, php service, javascript, leaflet, tile service from open street map. | #CFM,study requirements for work on SCEC Community Fault Model,start with postgres db, postgis extension, php service, javascript, leaflet, tile service from open street map. | ||
| − | #define the json blob for material properties used in plotting scripts, define import and export to file format, rpretell@ucdavis.edu, https://scec.usc.edu/scecpedia/LA_Vertical_Profiles#Result | + | #UCVMC,define the json blob for material properties used in plotting scripts, define import and export to file format, rpretell@ucdavis.edu, https://scec.usc.edu/scecpedia/LA_Vertical_Profiles#Result |
| − | #look into updating Wills' map in UCVMC, https://scec.usc.edu/scecpedia/Wills_Map, ucvm.e | + | #UCVMC,look into updating Wills' map in UCVMC, https://scec.usc.edu/scecpedia/Wills_Map, ucvm.e |
| − | #work to make Rob Grave's mesh generating code to work for him (tweak Rob's directory and Scott's https://source.usc.edu/svn/cybershake/import/trunk/UCVM) | + | #UCVMC,work to make Rob Grave's mesh generating code to work for him (tweak Rob's directory and Scott's https://source.usc.edu/svn/cybershake/import/trunk/UCVM) |
| − | #doc how to process .bin file generated from plotting script or ucvm basin_query call, https://scec.usc.edu/scecpedia/UCVMC_How_to_process_bin_data | + | #UCVMC,doc how to process .bin file generated from plotting script or ucvm basin_query call, https://scec.usc.edu/scecpedia/UCVMC_How_to_process_bin_data |
| − | #translate pbs/hpc/ucvm2mesh to slurm calls | + | #UCVMC,translate pbs/hpc/ucvm2mesh to slurm calls |
#(No work done)Generate large (100M+ point) AWP mesh using ucvmc and ucvmp and compare(Scott said ucvmp can not generate big mesh since it does not have mpi flavor to it) | #(No work done)Generate large (100M+ point) AWP mesh using ucvmc and ucvmp and compare(Scott said ucvmp can not generate big mesh since it does not have mpi flavor to it) | ||
| − | #Bay Area velocity model's z2.5 plots for Kevin -- different tiling order of cca/cencal/cvms5 , https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Result | + | #UCVMC,Bay Area velocity model's z2.5 plots for Kevin -- different tiling order of cca/cencal/cvms5 , https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Result |
| − | #updated cca06 crossing data for Kevin (fuzzy edge problem compare to UCVM python version, a bug found by Scott in CCA06 C version), nwo includes two sets, one without GTL and the other with GTL, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result | + | #UCVMC,updated cca06 crossing data for Kevin (fuzzy edge problem compare to UCVM python version, a bug found by Scott in CCA06 C version), nwo includes two sets, one without GTL and the other with GTL, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result |
#Doc how La Habra mesh is made with ucvm2mesh_mpi_layer, https://scec.usc.edu/scecpedia/UCVMC_La_Habra_mesh_generation#ucvm2mesh_mpi_layer | #Doc how La Habra mesh is made with ucvm2mesh_mpi_layer, https://scec.usc.edu/scecpedia/UCVMC_La_Habra_mesh_generation#ucvm2mesh_mpi_layer | ||
| − | #Generated various plots highlight CS17.3-H, https://scec.usc.edu/scecpedia/UCVMC_CS17.3-H_plots#Plots | + | #UCVMC,Generated various plots highlight CS17.3-H, https://scec.usc.edu/scecpedia/UCVMC_CS17.3-H_plots#Plots |
| − | #Registered new Central California Basin Model from Havard. CS17.3-H with ELY GTL | + | #UCVMC,Registered new Central California Basin Model from Havard. CS17.3-H with ELY GTL |
| − | #built a useable UCVMC installation on Titan, /lustre/atlas/proj-shared/geo112/ucvm_18_5/ | + | #UCVMC,built a useable UCVMC installation on Titan, /lustre/atlas/proj-shared/geo112/ucvm_18_5/ |
| − | #made UCVMC statically linkable and the plugin models worked as in dynamically linked UCVMC | + | #UCVMC,made UCVMC statically linkable and the plugin models worked as in dynamically linked UCVMC |
| − | #reworked logic in ucvm2mesh-mpi to have a more 'feasible' compute node requirement for very large mesh like La Habra, ucvm2mesh_mpi_layer, remember to strip result directory when running at blue waters. lfs setstripe -c 4 /path/to/working/directory | + | #UCVMC,reworked logic in ucvm2mesh-mpi to have a more 'feasible' compute node requirement for very large mesh like La Habra, ucvm2mesh_mpi_layer, remember to strip result directory when running at blue waters. lfs setstripe -c 4 /path/to/working/directory |
| − | #extract UCVMC cs173 info for Scott's gridpoints (off alittle but okayed by Scott) | + | #UCVMC,extract UCVMC cs173 info for Scott's gridpoints (off alittle but okayed by Scott) |
| − | #reworked basin_query tests in slurm on hpc | + | #UCVMC,reworked basin_query tests in slurm on hpc |
| − | #created a pull request to official UCVM-C, fix up each models that is in github -- merged into official UCVMC's dev branch | + | #UCVMC,created a pull request to official UCVM-C, fix up each models that is in github -- merged into official UCVMC's dev branch |
| − | #Ran tests tiling cencal+cca+cvms5+bbp1d, the 1d part gets surface gtl | + | #UCVMC,Ran tests tiling cencal+cca+cvms5+bbp1d, the 1d part gets surface gtl |
| − | #Tiled cencal+cca(with gtl)+cvms5(with gtl)+1d tested and verified | + | #UCVMC,Tiled cencal+cca(with gtl)+cvms5(with gtl)+1d tested and verified |
| − | #GTL for CCA06- verified by generating identical plots to CyberShake_Study_18.5 | + | #UCVMC,GTL for CCA06- verified by generating identical plots to CyberShake_Study_18.5 |
| − | #Fixed plugin/dynamic loading of modules, Scott's interleaving bug, fixes for cvmsi is also needed for the plugin module. There was the missed case for left-over points in chunking/processing code, fixed that. Also the code was not setup to take more than one dynamic library | + | #UCVMC,Fixed plugin/dynamic loading of modules, Scott's interleaving bug, fixes for cvmsi is also needed for the plugin module. There was the missed case for left-over points in chunking/processing code, fixed that. Also the code was not setup to take more than one dynamic library |
| − | #Setup to run my latest UCVMC on blue water - query works but plotting scripts do not work with BW's installed default pthread library(too old, conflicting with matplotlib's pyplot from Anaconda2) | + | #UCVMC,Setup to run my latest UCVMC on blue water - query works but plotting scripts do not work with BW's installed default pthread library(too old, conflicting with matplotlib's pyplot from Anaconda2) |
| − | #Fixed segfaults flagged from a working ucvmc installation on local mac (fixed cs173_final(), cca_final()), (mac gdb installation hints: https://gist.github.com/gravitylow/fb595186ce6068537a6e9da6d8b5b96d) | + | #UCVM,Fixed segfaults flagged from a working ucvmc installation on local mac (fixed cs173_final(), cca_final()), (mac gdb installation hints: https://gist.github.com/gravitylow/fb595186ce6068537a6e9da6d8b5b96d) |
| − | #Ran basin_query_mpi on hpc with SLURM | + | #UCVMC,Ran basin_query_mpi on hpc with SLURM |
| − | #Doc projection fix on scecpedia, https://scec.usc.edu/scecpedia/Final_Projection_Review_Result | + | #UCVMC,Doc projection fix on scecpedia, https://scec.usc.edu/scecpedia/Final_Projection_Review_Result |
| − | #Add the fixed config file into CCA's github, also edits from below(pending moving to scec space): https://github.com/meihuisu/CCA.git | + | #UCVMC,Add the fixed config file into CCA's github, also edits from below(pending moving to scec space): https://github.com/meihuisu/CCA.git |
| − | #Copy the CCA06 data on scec-01/Intensity to hypocenter, change the dependency to hypocenter in the make-data script | + | #UCVMC,Copy the CCA06 data on scec-01/Intensity to hypocenter, change the dependency to hypocenter in the make-data script |
| − | #adding elygtl to CCA06 | + | #UCVMC,adding elygtl to CCA06 |
| − | #new cs173 github location(pending moving to scec space):https://github.com/meihuisu/cs173.git | + | #UCVMC,new cs173 github location(pending moving to scec space):https://github.com/meihuisu/cs173.git |
| − | #Register Scott NorCal model into UCVM
- CS173, compare CS173 plots from ScottC with my version | + | #UCVMC,Register Scott NorCal model into UCVM
- CS173, compare CS173 plots from ScottC with my version |
| − | #Doc CCA06 Crossing review, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result | + | #UCVMC,Doc CCA06 Crossing review, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result |
| − | #Update basin_query_mpi to return list of crossing per grid point, also return one crossing output file for each algorithm. | + | #UCVMC,Update basin_query_mpi to return list of crossing per grid point, also return one crossing output file for each algorithm. |
| − | #To check projection fix, trace processing from en-jui data files to configuration files | + | #UCVMC,To check projection fix, trace processing from en-jui data files to configuration files |
| − | #Cross section plots showing 0-10km depth through CCA06, showing velocity inversions, starting at one end of region at 5km spacing to other end of region. **No need, posted inversion plot in scecpedia** | + | #UCVMC,Cross section plots showing 0-10km depth through CCA06, showing velocity inversions, starting at one end of region at 5km spacing to other end of region. **No need, posted inversion plot in scecpedia** |
== Related Entries == | == Related Entries == | ||
*[[CME Software Staff]] | *[[CME Software Staff]] | ||
*[[CME Project]] | *[[CME Project]] | ||
Revision as of 17:38, 19 October 2018
Mei-Hui Su's Project/Product Backlog
Project/Product Backlog
- CFM,gather and define the functions needed for the leaflet CFM viewers and study api and start building up the viewer utilities
- CFM, update CFM db/tb schema after science group fix the excel page and resolve typo/conflicts/missing data
- UCVMC,learn the tools used in https://ds.iris.edu/ds/products/emc-desktoptools/. paraview for desktop
- UCVMC,refine sop & doc steps for generating ucvm.e
- UCVMC,tied up loose ends in UCVMC18.5 and test
- need to make UCVMC work with python3(Scott said bw is stern about it), mainly plotting and config/build scripts
- UCVMC,Discuss mesh plotting Scripts with Scott Cal.
DONE
- CFM, add the linkup in js for the leaflet viewer and postgis' geoJSON geometry trace info
- CFM, add php/js to extract geoJSON for target fault object during query and track them to be sent to the viewer
- CFM, translate shapesfiles from release tarball into postgis db sql script and ingest into backend db
- CFM, explore and code a leaflet viewer with canned geoJSON data
- CFM, code php/java scripts/html page for queries with user input:query by region (with on the fly region list extracted from backend db via php), query by a range of strike value(with on the fly min/max strike range, extracted from the backend db via php), query by keyword(full text search from postgres db on object name, can be expanded in the future per request); query by latlon (within 0.0001 precision, a small roi box is created and used postgis to make overlap check)
- CFM, preprocess the excel from CFM5_release_2017, define db/tb schema and ingest the information into postgres db.
- UCVMC,generate the tiled v1000 data that Kevin needed for https://github.com/kevinmilner/cybershake-analysis/tree/master/study_18_8/hazard_maps#study-188-hazard-maps, they are left with https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Bay_Area_Velocity_Model
- CFM,study requirements for work on SCEC Community Fault Model,start with postgres db, postgis extension, php service, javascript, leaflet, tile service from open street map.
- UCVMC,define the json blob for material properties used in plotting scripts, define import and export to file format, rpretell@ucdavis.edu, https://scec.usc.edu/scecpedia/LA_Vertical_Profiles#Result
- UCVMC,look into updating Wills' map in UCVMC, https://scec.usc.edu/scecpedia/Wills_Map, ucvm.e
- UCVMC,work to make Rob Grave's mesh generating code to work for him (tweak Rob's directory and Scott's https://source.usc.edu/svn/cybershake/import/trunk/UCVM)
- UCVMC,doc how to process .bin file generated from plotting script or ucvm basin_query call, https://scec.usc.edu/scecpedia/UCVMC_How_to_process_bin_data
- UCVMC,translate pbs/hpc/ucvm2mesh to slurm calls
- (No work done)Generate large (100M+ point) AWP mesh using ucvmc and ucvmp and compare(Scott said ucvmp can not generate big mesh since it does not have mpi flavor to it)
- UCVMC,Bay Area velocity model's z2.5 plots for Kevin -- different tiling order of cca/cencal/cvms5 , https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Result
- UCVMC,updated cca06 crossing data for Kevin (fuzzy edge problem compare to UCVM python version, a bug found by Scott in CCA06 C version), nwo includes two sets, one without GTL and the other with GTL, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result
- Doc how La Habra mesh is made with ucvm2mesh_mpi_layer, https://scec.usc.edu/scecpedia/UCVMC_La_Habra_mesh_generation#ucvm2mesh_mpi_layer
- UCVMC,Generated various plots highlight CS17.3-H, https://scec.usc.edu/scecpedia/UCVMC_CS17.3-H_plots#Plots
- UCVMC,Registered new Central California Basin Model from Havard. CS17.3-H with ELY GTL
- UCVMC,built a useable UCVMC installation on Titan, /lustre/atlas/proj-shared/geo112/ucvm_18_5/
- UCVMC,made UCVMC statically linkable and the plugin models worked as in dynamically linked UCVMC
- UCVMC,reworked logic in ucvm2mesh-mpi to have a more 'feasible' compute node requirement for very large mesh like La Habra, ucvm2mesh_mpi_layer, remember to strip result directory when running at blue waters. lfs setstripe -c 4 /path/to/working/directory
- UCVMC,extract UCVMC cs173 info for Scott's gridpoints (off alittle but okayed by Scott)
- UCVMC,reworked basin_query tests in slurm on hpc
- UCVMC,created a pull request to official UCVM-C, fix up each models that is in github -- merged into official UCVMC's dev branch
- UCVMC,Ran tests tiling cencal+cca+cvms5+bbp1d, the 1d part gets surface gtl
- UCVMC,Tiled cencal+cca(with gtl)+cvms5(with gtl)+1d tested and verified
- UCVMC,GTL for CCA06- verified by generating identical plots to CyberShake_Study_18.5
- UCVMC,Fixed plugin/dynamic loading of modules, Scott's interleaving bug, fixes for cvmsi is also needed for the plugin module. There was the missed case for left-over points in chunking/processing code, fixed that. Also the code was not setup to take more than one dynamic library
- UCVMC,Setup to run my latest UCVMC on blue water - query works but plotting scripts do not work with BW's installed default pthread library(too old, conflicting with matplotlib's pyplot from Anaconda2)
- UCVM,Fixed segfaults flagged from a working ucvmc installation on local mac (fixed cs173_final(), cca_final()), (mac gdb installation hints: https://gist.github.com/gravitylow/fb595186ce6068537a6e9da6d8b5b96d)
- UCVMC,Ran basin_query_mpi on hpc with SLURM
- UCVMC,Doc projection fix on scecpedia, https://scec.usc.edu/scecpedia/Final_Projection_Review_Result
- UCVMC,Add the fixed config file into CCA's github, also edits from below(pending moving to scec space): https://github.com/meihuisu/CCA.git
- UCVMC,Copy the CCA06 data on scec-01/Intensity to hypocenter, change the dependency to hypocenter in the make-data script
- UCVMC,adding elygtl to CCA06
- UCVMC,new cs173 github location(pending moving to scec space):https://github.com/meihuisu/cs173.git
- UCVMC,Register Scott NorCal model into UCVM - CS173, compare CS173 plots from ScottC with my version
- UCVMC,Doc CCA06 Crossing review, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result
- UCVMC,Update basin_query_mpi to return list of crossing per grid point, also return one crossing output file for each algorithm.
- UCVMC,To check projection fix, trace processing from en-jui data files to configuration files
- UCVMC,Cross section plots showing 0-10km depth through CCA06, showing velocity inversions, starting at one end of region at 5km spacing to other end of region. **No need, posted inversion plot in scecpedia**
