Difference between revisions of "Product Backlog - Mei"

From SCECpedia
Jump to navigationJump to search
Line 2: Line 2:
  
 
== Project/Product Backlog ==
 
== Project/Product Backlog ==
#UCVM, change largefiles location to Globus online collection at CARC (/project/scec_608/public/ucvm/v21_10/model)
+
#CPD, new web dashboard
 +
#CSM, new web dashboard
 +
#CGM, adding new INSAR tracks
 +
#UCVM, continue adding ci to more cvms
 +
#UCVM, move CVMH and CENCAL to be pluggable models
 +
#UCVM, upgrade CENCAL to newer data
 +
#CFM, add separate models, CFM6.0 preferred/alternative/rupture set in addition to CFM5.3
 
#UCVM, study LSU model blending code
 
#UCVM, study LSU model blending code
#UCVM, add 9 new basin models from Andreas/Harvard group
 
#UCVM, add ci to CVMH in GitHub
 
#UCVM, work on getting release for upcoming UCVM release, retest on CARC
 
#UCVM, adding new basin models from Andreas
 
#UCVM, link Github runner from ucvm_plotting to UCVM
 
 
#UCVM, find replacement(pyGMT) to matplotlib used in the plotting scripts because matplotlib's Basemap is not being supported in python3
 
#UCVM, find replacement(pyGMT) to matplotlib used in the plotting scripts because matplotlib's Basemap is not being supported in python3
 
#UCVMC, setup remote GitHub runner for UCVM's GitHub actions
 
#UCVMC, setup remote GitHub runner for UCVM's GitHub actions
Line 31: Line 32:
  
 
DONE
 
DONE
 +
#UCVM, add tapering interpolation method
 +
#UCVM, add San Jacinto Fault Zone velocity model(sjfz)
 +
#UCVM, change largefiles location to Globus online collection at CARC (/project/scec_608/public/ucvm)
 +
#UCVM, add 9 new basin models from Andreas/Harvard group
 +
#UCVM, add ci to cvms models in GitHub
 +
#UCVM, work on getting release for upcoming UCVM release, retest on CARC
 +
#UCVM, link Github runner from ucvm_plotting to UCVM
 
#UCVM, test out docker setup for python2/ucvm/ucvm_plotting
 
#UCVM, test out docker setup for python2/ucvm/ucvm_plotting
 
#UCVM, rework get_largefiles.py to pick up path/config info from setup.list instead of current hardwired method.
 
#UCVM, rework get_largefiles.py to pick up path/config info from setup.list instead of current hardwired method.

Revision as of 07:33, 2 December 2022

Mei-Hui Su's Project/Product Backlog

Project/Product Backlog

  1. CPD, new web dashboard
  2. CSM, new web dashboard
  3. CGM, adding new INSAR tracks
  4. UCVM, continue adding ci to more cvms
  5. UCVM, move CVMH and CENCAL to be pluggable models
  6. UCVM, upgrade CENCAL to newer data
  7. CFM, add separate models, CFM6.0 preferred/alternative/rupture set in addition to CFM5.3
  8. UCVM, study LSU model blending code
  9. UCVM, find replacement(pyGMT) to matplotlib used in the plotting scripts because matplotlib's Basemap is not being supported in python3
  10. UCVMC, setup remote GitHub runner for UCVM's GitHub actions
  11. UCVMC, survey different kinds of unittest/regrssion testing platforms in C/bash/python
  12. UCVMC, explore Valgrind as memory leak tool and perhaps coverage tools
  13. UCVMC, explore gcov, and prof as coverage tool
  14. UCVMC, need to make UCVMC work with python3, mainly plotting and config/build scripts - with docker setup
  15. CTM, study the CTM python code from Liz Hearn
  16. UCVMC, add watermark on ucvm_web's viewer part (Look at Edric's leaflet overlay for CGM web)
  17. WEB, status review for CFM, UCVM, GFM
  18. UCVMC, design/test out continuous integration testing framework on github for ucvmc.
  19. UCVMC, prepare Plotly usage example to the group
  20. UCVMC, internal GTL for cvms5/cca is different from external elygtl:ely, https://scec.usc.edu/scecpedia/UCVM_cca/cvms5_comparing_builtin-gtl_vs_elygtl:ely
  21. CFM, update CFM db/tb schema after science group fix the excel page ands typo/conflicts/missing data
  22. UCVMC, config dumping tool
  23. UCVMC,learn the tools used in https://ds.iris.edu/ds/products/emc-desktoptools/. paraview for desktop
  24. UCVMC,refine sop & doc steps for generating ucvm.e

LOST

  1. GFM+CRM, add Moho search option
  2. UCVMC, user reports UCVM used for simulation SPECFEM3D seems to be causing segfaults (ctape@alaska.edu)>> ?? no segfault ??https://scec.usc.edu/scecpedia/UCVM_elevation_vs_depth,_model_boundary

DONE

  1. UCVM, add tapering interpolation method
  2. UCVM, add San Jacinto Fault Zone velocity model(sjfz)
  3. UCVM, change largefiles location to Globus online collection at CARC (/project/scec_608/public/ucvm)
  4. UCVM, add 9 new basin models from Andreas/Harvard group
  5. UCVM, add ci to cvms models in GitHub
  6. UCVM, work on getting release for upcoming UCVM release, retest on CARC
  7. UCVM, link Github runner from ucvm_plotting to UCVM
  8. UCVM, test out docker setup for python2/ucvm/ucvm_plotting
  9. UCVM, rework get_largefiles.py to pick up path/config info from setup.list instead of current hardwired method.
  10. CFM, extend the REST API to allow ingesting of 'external' gocad tsurf file and displaying it on plot3D
  11. CFM, add 'sharing link' option to plot3D, implement command option/REST API, to CFM viewer.
  12. CGM, add surface plot for INSAR's velocity data given a boundary
  13. CGM, add 2d view of INSAR data, add time series plot for INSAR data (calling python pkg)
  14. CGM, add GNSS time series data hosted at MIT with plotly plot
  15. CGM, flesh out GNSS data access, add INSAR data
  16. CGM, trimmed out CFM part of code out of CGM_view 1.0, rework some search logic
  17. CFM, add seismicity data to 3dPlot vtk viewer as point scatter plots
  18. CFM, add seismicity data to map base leaflet as pixi overlay - Hauksson, Ross, and significant faults layers
  19. CM, update UCVM part of code with new UCVM+ucvm_plotting code bases
  20. UCVM, reactivated and updated the Utah velocity model, WH
  21. UCVM, add 2 new basin velocity models from LSU, (SSIP Imperial Valley, SSIP Coachella Valley)
  22. CFM, release CFM viewer with CFM5.3, plus many updates to viewer functions, ie. fault search filtering, strike and dip search option
  23. UCVM, split the plotting out to different github repository to simplify the UCVM's dependency python3. --https://github.com/SCECcode/ucvm_plotting
  24. USC, test out USC discovery/endeavour slurm/sbatch/srun/mpirun with UCVM's mpi code
  25. UCVM, Elnaz TACC/ucvm2etree_extract_MPI was having problem. It turns out global variable for zrange needs to be initalized when using intel mpi compiler on Frontera.
  26. GFM, add 3d viewer to GFM viewer
  27. CRM/CXM viewer tool to do shp file -> Geojson conversion with postgis -> convert data for CTM, CRM
  28. CM+UCVM, create CM viewer with integrated 2d mapview with CFM/GFM/CTM, and ucvm viewer capability
  29. GFM+CRM, update GFM with CRM and a new viewer setup
  30. GFM, process the shp CRM_polygons_Plesch_Mar_7_2019 from Liz Hearn (CRM pqsql schema)
  31. CFM, expand 3d viewer to include surface california shoreline+border as reference orientation
  32. CFM, expand 3d viewer to include surface trace/blind for faults
  33. CFM, expand 3d viewer to have a orientation axes marker
  34. CFM, expand 3d viewer to allow user selectable color picker for each 3D fault representation
  35. CFM, expand 3d viewer to include toggle option for surface/wireframe
  36. UCVMC, extract geometry geoJson for the 2D area from CRM_polygons_Plesch_Mar_7_2019 and create an optional layer over UCVMC viewer
  37. CFM, VTK.js has some updates, need to bring that in, look into vtkOrientationMarkerWidget for CFM_web
  38. UCVMC, test GitHub actions and runner for continuous integration setup for ucvm build
  39. CFM, on fixing the mixing blindness in a fault - array of blinds at OBJECT_tb
  40. CFM, extract geometry geoJson of faults from CFM5.2 into a standalone external file and code ingest leaflet groupLayer that can be reused with any leaflet based scec viewers
  41. UCVMC, review various places where vs30/Wallis is used in ucvm/models
  42. UCVMC, adding Cencal & CCA06 to Moho's ucvm_web
  43. UCVMC, trace the UCVM VS30 values that differ from Rob Grave's VS30 list. interpolation vs nearest point:https://scec.usc.edu/scecpedia/UCVM_VS30_etree_map_(Wills_2015)_UCVM%27s_interpolation#vs30_etree_map
  44. UCVMC, add a new model for UCVMC, Albacore, Southern California Offshore velocity model, kohler@caltech.edu, Daniel.bowden@erdw.ethz.ch
  45. UCVMC, a list of elevations for elevation-based profile script in generated meta file
  46. UCVMC, branch cleanup on UCVMC -- only Master, forGFM, withSVM, forPython3 left
  47. UCVMC, verify UCVM's CVMS4.26 profile plots matches with Billy's averaging profile plot with EnJui's CVM4SI26_slices. Also, tested with PO Chan's set of CVMS4.26 dataset.
  48. UCVMC, make a mini dockerized UCVM image with cvms, cvms5, and cvmh1511 git@github.com:SCECcode/ucvm_docker.git
  49. UCVMC, needs some Z25/Z10 data for some models with GTL https://scec.usc.edu/scecpedia/UCVM_z1,z2.5_for_CCA06_with_GTL,_CVM-S4.26.M01
  50. UCVMC, viewer that allow user to query for UCVMC material property and initiate the plotting capability in UCVMC/utilities(git@github.com:SCECcode/UCVM_web.git)
  51. UCVMC, add Elevation query mode to ucvm2mesh, ucvm2mesh_mpi, ucvm2mesh_mpi_layer, (currently on branch -forGFM). User needs to include cvmh as part of cvmlist and let UCVM's model tiling to set the property values for in-air, in-water properly,ucvmlist=cvmsi,cvmh. Note: if region is outside of cvmh area, then no data would exists for those regions. In configuration file, specify querymode=ELEVATION
  52. CFM, add simple 3d viewer (using vtk.js) to selected faults
  53. SVM, Elnas has requested generation of 3 new etrees, https://scec.usc.edu/scecpedia/UCVM_svm_etree and new data is at https://scec.usc.edu/scecpedia/UCVM_svmgtl_and_elygtl
  54. GFM, viewer that allow user to query for GFM material property + plotting capability(https://github.com/SCECcode/GFM_web.git)
  55. GFM, geological features decorated model -- update vx-lite with new g model files
  56. CFM, study web page accessibility, https://www.w3.org/TR/wai-aria-practices, study ui write-up, https://www.smashingmagazine.com/06/designing-perfect-accordion-checklist/
  57. CFM, prep the core CFM code for the new CFM data release (Edric has been updating and maintaining UI per user's requests) -- CFM5.2-Fault_ID_preferred_final.xlsx, CFM52_db, nonblind, blind native ts files
  58. UCVMC, SVM GTL from Caltech (https://scec.usc.edu/scecpedia/UCVM_svmgtl_and_elygtl) and (https://scec.usc.edu/scecpedia/UCVM_svmgtl)
  59. UCVMC,review and update the GitHub UCVMC's wiki pages
  60. UCVMC,tied up loose ends and tests for UCVM V19.4,add examples directory for program and plots examples
  61. UCVMC, generate etree for Garner Valley, CA for elnaz@caltech.edu, https://scec.usc.edu/scecpedia/UCVM_etree_for_Garner_Valley
  62. UCVMC, add plot_elevation_profile.py and plot_elevation_cross_section.py,(ctape@alaska.edu) https://scec.usc.edu/scecpedia/UCVM_elevation_vs_depth,_model_boundary
  63. CFM, document CFM dashboard in github and transfer to SCEC space
  64. CFM, streamline the preprocessing of CFM database -- xlsx file to Postgres db
  65. UCVMC, generate plots for USGS CVM doc -- http://hypocenter.usc.edu/research/ucvmc/USGS-CVM-plot.tar.gz
  66. CFM, add analytic tracking to the dashboard -- Edric/google
  67. CFM, add latlon search option with direct location selection on the map area
  68. CFM, add highlighting of faults on the viewer part in addition to be from the table
  69. CFM, add group download(zip) of multiple faults geo data and meta data
  70. UCVMC, final set of plot/data for jplet@cpp.edu, 400+ 1d plots in http://hypocenter.usc.edu/research/ucvmc_result/pomona4.tar.gz, cross section plots in http://hypocenter.usc.edu/research/ucvmc_result/pomona3.tar.gz
  71. CFM, update cfm dashboard(add dip search, add 'key' trigger to keywordTxt, change latlon search to 4 inputs)
  72. CFM, add download function to CFM dashboard (on popup window)
  73. CFM, stash the CFM5.2 dataset at asperity's /var/www/html/CFM5_preferred/native, 500m, 1000m
  74. CFM, refine end-to-end setup of new CFM dashboard to server/client test environment
  75. CFM, setup initial CFM db host over at AWS, http://asperity.scec.org/CFM/web/cfm_view.html
  76. UCVMC, first set of plot/data for jplet@cpp.edu. https://scec.usc.edu/scecpedia/UCVMC_basin_depth_study,_poly_tech,_Pomona
  77. UCVMC, support jpolet@cpp.edu's group with cross_section and depth_profile processing (added metadata/data output to depth_profile.py) https://scec.usc.edu/scecpedia/UCVMC_how_to_plot_cross_section_and_depth_profile
  78. CFM, gather and define the functions needed for the leaflet CFM viewers and study leaflet api and start building up the viewer utilities
  79. UCVMC, update 1d grid reference data for UCVMC acceptance test
  80. CFM, add the linkup in js for the leaflet viewer and postgis' geoJSON geometry trace info
  81. CFM, add php/js to extract geoJSON for target fault object during query and track them to be sent to the viewer
  82. CFM, translate shapesfiles from release tarball into postgis db sql script and ingest into backend db
  83. CFM, explore and code a leaflet viewer with canned geoJSON data
  84. CFM, code php/java scripts/html page for queries with user input:query by region (with on the fly region list extracted from backend db via php), query by a range of strike value(with on the fly min/max strike range, extracted from the backend db via php), query by keyword(full text search from postgres db on object name, can be expanded in the future per request); query by latlon (within 0.0001 precision, a small roi box is created and used postgis to make overlap check)
  85. CFM, preprocess the excel from CFM5_release_2017, define db/tb schema and ingest the information into postgres db.
  86. UCVMC,generate the tiled v1000 data that Kevin needed for https://github.com/kevinmilner/cybershake-analysis/tree/master/study_18_8/hazard_maps#study-188-hazard-maps, they are left with https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Bay_Area_Velocity_Model
  87. CFM,study requirements for work on SCEC Community Fault Model,start with postgres db, postgis extension, php service, javascript, leaflet, tile service from open street map.
  88. UCVMC,define the json blob for material properties used in plotting scripts, define import and export to file format, rpretell@ucdavis.edu, https://scec.usc.edu/scecpedia/LA_Vertical_Profiles#Result
  89. UCVMC,look into updating Wills' map in UCVMC, https://scec.usc.edu/scecpedia/Wills_Map, ucvm.e
  90. UCVMC,work to make Rob Grave's mesh generating code to work for him (tweak Rob's directory and Scott's https://source.usc.edu/svn/cybershake/import/trunk/UCVM)
  91. UCVMC,doc how to process .bin file generated from plotting script or ucvm basin_query call, https://scec.usc.edu/scecpedia/UCVMC_How_to_process_bin_data
  92. UCVMC,translate pbs/hpc/ucvm2mesh to slurm calls
  93. (No work done)Generate large (100M+ point) AWP mesh using ucvmc and ucvmp and compare(Scott said ucvmp can not generate big mesh since it does not have mpi flavor to it)
  94. UCVMC,Bay Area velocity model's z2.5 plots for Kevin -- different tiling order of cca/cencal/cvms5 , https://scec.usc.edu/scecpedia/Bay_Area_Velocity_Model_Z2.5_data#Result
  95. UCVMC,updated cca06 crossing data for Kevin (fuzzy edge problem compare to UCVM python version, a bug found by Scott in CCA06 C version), nwo includes two sets, one without GTL and the other with GTL, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result
  96. Doc how La Habra mesh is made with ucvm2mesh_mpi_layer, https://scec.usc.edu/scecpedia/UCVMC_La_Habra_mesh_generation#ucvm2mesh_mpi_layer
  97. UCVMC,Generated various plots highlight CS17.3-H, https://scec.usc.edu/scecpedia/UCVMC_CS17.3-H_plots#Plots
  98. UCVMC,Registered new Central California Basin Model from Havard. CS17.3-H with ELY GTL
  99. UCVMC,built a useable UCVMC installation on Titan, /lustre/atlas/proj-shared/geo112/ucvm_18_5/
  100. UCVMC,made UCVMC statically linkable and the plugin models worked as in dynamically linked UCVMC
  101. UCVMC,reworked logic in ucvm2mesh-mpi to have a more 'feasible' compute node requirement for very large mesh like La Habra, ucvm2mesh_mpi_layer, remember to strip result directory when running at blue waters. lfs setstripe -c 4 /path/to/working/directory
  102. UCVMC,extract UCVMC cs173 info for Scott's gridpoints (off alittle but okayed by Scott)
  103. UCVMC,reworked basin_query tests in slurm on hpc
  104. UCVMC,created a pull request to official UCVM-C, fix up each models that is in github -- merged into official UCVMC's dev branch
  105. UCVMC,Ran tests tiling cencal+cca+cvms5+bbp1d, the 1d part gets surface gtl
  106. UCVMC,Tiled cencal+cca(with gtl)+cvms5(with gtl)+1d tested and verified
  107. UCVMC,GTL for CCA06- verified by generating identical plots to CyberShake_Study_18.5
  108. UCVMC,Fixed plugin/dynamic loading of modules, Scott's interleaving bug, fixes for cvmsi is also needed for the plugin module. There was the missed case for left-over points in chunking/processing code, fixed that. Also the code was not setup to take more than one dynamic library
  109. UCVMC,Setup to run my latest UCVMC on blue water - query works but plotting scripts do not work with BW's installed default pthread library(too old, conflicting with matplotlib's pyplot from Anaconda2)
  110. UCVM,Fixed segfaults flagged from a working ucvmc installation on local mac (fixed cs173_final(), cca_final()), (mac gdb installation hints: https://gist.github.com/gravitylow/fb595186ce6068537a6e9da6d8b5b96d)
  111. UCVMC,Ran basin_query_mpi on hpc with SLURM
  112. UCVMC,Doc projection fix on scecpedia, https://scec.usc.edu/scecpedia/Final_Projection_Review_Result
  113. UCVMC,Add the fixed config file into CCA's github, also edits from below(pending moving to scec space): https://github.com/meihuisu/CCA.git
  114. UCVMC,Copy the CCA06 data on scec-01/Intensity to hypocenter, change the dependency to hypocenter in the make-data script
  115. UCVMC,adding elygtl to CCA06
  116. UCVMC,new cs173 github location(pending moving to scec space):https://github.com/meihuisu/cs173.git
  117. UCVMC,Register Scott NorCal model into UCVM
 - CS173, compare CS173 plots from ScottC with my version
  118. UCVMC,Doc CCA06 Crossing review, https://scec.usc.edu/scecpedia/Final_CCA06_Crossings_Result
  119. UCVMC,Update basin_query_mpi to return list of crossing per grid point, also return one crossing output file for each algorithm.
  120. UCVMC,To check projection fix, trace processing from en-jui data files to configuration files
  121. UCVMC,Cross section plots showing 0-10km depth through CCA06, showing velocity inversions, starting at one end of region at 5km spacing to other end of region. **No need, posted inversion plot in scecpedia**

Related Entries