Difference between revisions of "BBP Release Steps"

From SCECpedia
Jump to navigationJump to search
Line 1: Line 1:
 
= Steps For Releasing Broadband Platform =
 
= Steps For Releasing Broadband Platform =
  
−
These are the steps needed for making a Broadband Platform release.
+
There are a number of steps needed for making a Broadband Platform release. The main steps are:
 +
 
 +
* Code Freeze
 +
* Science Review
 +
* Update Tests
 +
* Preparing Documentation
 +
* Preparing Release Candidate
 +
* Create Final Release
 +
* Release Announcement
  
 
=== Code Freeze ===
 
=== Code Freeze ===
  
−
The first step for a BBP release is to collect the latest codes from all science teams. Ideally, each science team will have reviewed their results and signed off on their method before the code freeze. This will avoid surprises and delays later in the release process.
+
The first step for a BBP release is to collect the latest codes from all science teams. Ideally, each science team will have reviewed their results and signed off on their method before the code freeze. This should be done to avoid surprises and delays later in the release process.
  
 
* Announce code freeze date
 
* Announce code freeze date
−
* Collect code and parameter changes
+
* Collect code updates and parameter changes from each group
−
* Run simulations as needed
+
* Run simulations as needed to validate changes
 
** Single realizations
 
** Single realizations
−
*** Useful to make sure codes work
+
*** Useful to make sure codes work, saves tmpdata files in case something goes wrong and needs debugging
 
** Cluster testing
 
** Cluster testing
 
*** Useful to explore different realizations, create validation sets
 
*** Useful to explore different realizations, create validation sets
Line 21: Line 29:
 
These are the steps for a science review of the BBP:
 
These are the steps for a science review of the BBP:
  
−
* Make sure cluster has latest versions of everything, use 'diff -r' to make sure source distribution, bbp_gf, and bbp_val are in sync
+
* Make sure cluster has latest versions of everything
−
* rsync needed packages in bbp_gf to /staging
+
** Use 'diff -r' to make sure source distribution, bbp_gf, and bbp_val are in sync
 +
* rsync needed region packages in bbp_gf to /staging
 
* Run Part-A and Part-B full validations for each method
 
* Run Part-A and Part-B full validations for each method
−
** Can use XXXX_a.sh and XXXX_b.sh scripts (where XXXX is each method)
+
** Use XXXX_a.sh and XXXX_b.sh scripts (where XXXX is each method)
−
*** Create and submits to cluster all validation events for that method
+
*** Creates and submits to cluster all validation events for that method
 
* Run Multi-segment Part-A events not included in XXXX_a.sh scripts
 
* Run Multi-segment Part-A events not included in XXXX_a.sh scripts
−
** GP and SDSU Landers is an example, each segment needs to be submitted and then merged with bbp_merge_multisegment_validation.py tool
+
** Landers
 +
*** Song and Irikura Recipe Method 1: use single cluster simulations
 +
*** GP and SDSU: each segment needs to be submitted separately and then merged with bbp_merge_multisegment_validation.py tool
 
* Plot results
 
* Plot results
−
** plot_XXX_a.sh and plot_XXX_b.sh scripts will generate the combined GoF plots automatically
+
** plot_part_a_XXX.sh and plot_part_b_XXX.sh scripts will generate the combined GoF plots automatically
 
** combine_map_gof_gen.py and combine_dist_gof_gen.py to generate map and distance GoF plots
 
** combine_map_gof_gen.py and combine_dist_gof_gen.py to generate map and distance GoF plots
 
** Run create_convergence_plots.sh to create convergence plots for the Part-A validation events
 
** Run create_convergence_plots.sh to create convergence plots for the Part-A validation events
Line 43: Line 54:
 
**** Tables
 
**** Tables
 
** Copy Part-A GoFs, Part-B GMPE plots, Map GoFs, Distance GoFs, Convergence Plots to their respective directories
 
** Copy Part-A GoFs, Part-B GMPE plots, Map GoFs, Distance GoFs, Convergence Plots to their respective directories
 +
* Create tables
 +
** Use combine_bbp_data.py tool, once per method, to collect data from all Part-A events
 +
** Use report_bbp_data.py tool, also once per method, to process data file generated above and create table
 +
** Use create_dreger_fig3.py tool, again once per method, with the files produced in the step above to generate plots
 +
*** Copy dreger fig 3 plots to 'Summary' folder created above
 +
** Use create_combined_table.py tool, only once, to combine tables produced by report_bbp_data into a super table
 +
*** Order of files to use: UCSB, EXSIM, GP, SDSU, SONG, IRIKURA1, IRIKURA2
 +
** Import table into Excel
 +
*** Fix formatting, add colors, create comparison table by calculating differences between current and previous releases
 +
*** Copy final Excel file (supertable) to the 'Tables' folder created above
 +
* Send all data produced above to science team for review and approval
 +
 +
=== Update Tests ===
 +
 +
It is important to update the BBP tests. Here are the steps involved:
 +
 +
* Update version number in comps/version.txt
 +
* Run gen_accept_tests.py tool in tests directory
 +
** Will update xml files for acceptance tests
 +
*** Important in case of module API changes as acceptance tests use pre-computed XML files
 +
* Run Unit and Acceptance Tests once
 +
** If codes/parameters were updated, it is likely some tests will fail since reference files will not match
 +
* Update modified reference files with currently produced files
 +
* Run Unit and Acceptance Tests again
 +
** Save screen output as it will be used in the documentation steps below
 +
 +
=== Preparing Documentation ===
 +
 +
There are a few steps needed to get the documentation ready:
 +
 +
* Check out local copy of BBP GitHub wiki
 +
* Update documentation on the Wiki
 +
** Plots go to the images directory
 +
** Pdf files go to the pdfs directory
 +
* Update documentation
 +
 +
=== Preparing Release Candidate ===
 +
 +
These are the steps needed for a release candidate:
 +
 +
 +
 +
=== Create Final Release ===
 +
 +
=== Release Announcement ===

Revision as of 20:59, 4 June 2019

Steps For Releasing Broadband Platform

There are a number of steps needed for making a Broadband Platform release. The main steps are:

  • Code Freeze
  • Science Review
  • Update Tests
  • Preparing Documentation
  • Preparing Release Candidate
  • Create Final Release
  • Release Announcement

Code Freeze

The first step for a BBP release is to collect the latest codes from all science teams. Ideally, each science team will have reviewed their results and signed off on their method before the code freeze. This should be done to avoid surprises and delays later in the release process.

  • Announce code freeze date
  • Collect code updates and parameter changes from each group
  • Run simulations as needed to validate changes
    • Single realizations
      • Useful to make sure codes work, saves tmpdata files in case something goes wrong and needs debugging
    • Cluster testing
      • Useful to explore different realizations, create validation sets
  • Share results with science teams
  • Obtain approval from each science team

Science Review

These are the steps for a science review of the BBP:

  • Make sure cluster has latest versions of everything
    • Use 'diff -r' to make sure source distribution, bbp_gf, and bbp_val are in sync
  • rsync needed region packages in bbp_gf to /staging
  • Run Part-A and Part-B full validations for each method
    • Use XXXX_a.sh and XXXX_b.sh scripts (where XXXX is each method)
      • Creates and submits to cluster all validation events for that method
  • Run Multi-segment Part-A events not included in XXXX_a.sh scripts
    • Landers
      • Song and Irikura Recipe Method 1: use single cluster simulations
      • GP and SDSU: each segment needs to be submitted separately and then merged with bbp_merge_multisegment_validation.py tool
  • Plot results
    • plot_part_a_XXX.sh and plot_part_b_XXX.sh scripts will generate the combined GoF plots automatically
    • combine_map_gof_gen.py and combine_dist_gof_gen.py to generate map and distance GoF plots
    • Run create_convergence_plots.sh to create convergence plots for the Part-A validation events
    • Create release directory structure in hypocenter, on the BBP server, it will be:
      • /home/hypocenter-01/bbp/BBP_MMMYYYY_Plots
      • Inside, create the following folders:
        • Converge
        • Distance
        • Maps
        • Part-A
        • Part-B
        • Summary
        • Tables
    • Copy Part-A GoFs, Part-B GMPE plots, Map GoFs, Distance GoFs, Convergence Plots to their respective directories
  • Create tables
    • Use combine_bbp_data.py tool, once per method, to collect data from all Part-A events
    • Use report_bbp_data.py tool, also once per method, to process data file generated above and create table
    • Use create_dreger_fig3.py tool, again once per method, with the files produced in the step above to generate plots
      • Copy dreger fig 3 plots to 'Summary' folder created above
    • Use create_combined_table.py tool, only once, to combine tables produced by report_bbp_data into a super table
      • Order of files to use: UCSB, EXSIM, GP, SDSU, SONG, IRIKURA1, IRIKURA2
    • Import table into Excel
      • Fix formatting, add colors, create comparison table by calculating differences between current and previous releases
      • Copy final Excel file (supertable) to the 'Tables' folder created above
  • Send all data produced above to science team for review and approval

Update Tests

It is important to update the BBP tests. Here are the steps involved:

  • Update version number in comps/version.txt
  • Run gen_accept_tests.py tool in tests directory
    • Will update xml files for acceptance tests
      • Important in case of module API changes as acceptance tests use pre-computed XML files
  • Run Unit and Acceptance Tests once
    • If codes/parameters were updated, it is likely some tests will fail since reference files will not match
  • Update modified reference files with currently produced files
  • Run Unit and Acceptance Tests again
    • Save screen output as it will be used in the documentation steps below

Preparing Documentation

There are a few steps needed to get the documentation ready:

  • Check out local copy of BBP GitHub wiki
  • Update documentation on the Wiki
    • Plots go to the images directory
    • Pdf files go to the pdfs directory
  • Update documentation

Preparing Release Candidate

These are the steps needed for a release candidate:


Create Final Release

Release Announcement

Retrieved from "https://strike.scec.org/scecwiki/index.php?title=BBP_Release_Steps&oldid=22860"