Broadband Release Info
This page contains instructions for creating a release of the SCEC Broadband Platform software. A release consists of the main BBP source distribution, along with one or more region packages, containing GFs and region-specific parameters for each of the methods supported by the Platform, and optionally validation packages, which contain event specific data and can be used to run the Broadband Platform in validation mode.
Contents
Testing Before Release
The first step to release a version of the Broadband Platform is to run tests to make sure the Platform is working.
Unit Tests
The Platform contains a number of tests cases designed to test each individual software component of the platform. These are called Unit Tests, and can be executed with the following commands:
$ cd tests $ ./UnitTests.py
The tests take about 10 minutes to run, and after each test you should see an 'ok' message. Here's a sample output for a successful test run:
test_runprog (test_bband_utils.Test_bband_utils) ... ok test_runprog2 (test_bband_utils.Test_bband_utils) ... ok test_runprog3 (test_bband_utils.Test_bband_utils) ... ok test_execute_platform_bbp (test_python_code.Test_PythonCode) ... ok test_python_code_comps (test_python_code.Test_PythonCode) ... ok test_python_code_tests (test_python_code.Test_PythonCode) ... ok test_arias_duration (test_arias.Test_Arias) ... ok test_bbp2peer (test_bbp_format.Test_BBP_Format) ... ok test_exsim2bbp (test_bbp_format.Test_BBP_Format) ... ok test_peer2bbp (test_bbp_format.Test_BBP_Format) ... ok test_gensrf (test_genslip.TestGenslip) ... ok test_jbsim (test_jbsim.TestJbsim) ... ok test_hfsims (test_hfsims.Test_hfsims) ... ok test_wcc_siteamp (test_wcc_siteamp.Test_Wcc_siteamp) ... ok test_match (test_match.Test_Match) ... ok test_uc_fault_utils (test_uc_fault_utils.Test_UC_Fault_Utils) ... ok test_ucgen (test_ucrmg.Test_UCrmg) ... ok test_syn1d (test_syn1d.Test_syn1D) ... ok test_site_amp (test_uc_site.Test_UC_Site) ... ok test_bbtoolbox (test_bbtoolbox.Test_BBToolbox) ... ok test_amp_fac (test_amp_fac.Test_Amp_Fac) ... ok test_exsim (test_exsim.TestExsim) ... ok test_respect (test_respect.Test_Respect) ... ok test_rotd50 (test_rotd50.Test_RotD50) ... ok test_gof (test_gp_gof.Test_GP_Gof) ... ok test_mogof_10stat (test_sdsu_mogof.Test_SDSU_MOGof) ... ok ---------------------------------------------------------------------- Ran 26 tests in 592.858s OK
Acceptance Tests
The next step is to run the acceptance tests. These tests are complete end-to-end simulation runs, for both scenario and validation simulations using each of the available methods on the Broadband Platform.
In order to start the tests, just follow these steps:
$ cd tests $ ./AcceptTests.py
These tests will take just over 2 hours to run, and after each test completes, you should see an "ok" message. Here's the sample output of a successful test run:
Number of tests to run: 12 test_user-CSM (__main__.BBP2G_Acceptance_Tests) ... ok test_user-EXSIM (__main__.BBP2G_Acceptance_Tests) ... ok test_user-GP (__main__.BBP2G_Acceptance_Tests) ... ok test_user-SDSU (__main__.BBP2G_Acceptance_Tests) ... ok test_user-UCSB (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-CSM (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-EXSIM (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-GP (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-GP_seis (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-SDSU (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-SDSU_seis (__main__.BBP2G_Acceptance_Tests) ... ok test_valid-northridge-UCSB (__main__.BBP2G_Acceptance_Tests) ... ok ---------------------------------------------------------------------- Ran 12 tests in 7639.309s OK
Note that, all acceptance tests have their 'xml' workflow already created in the 'ref_data' directory of the BBP distribution. When modifications are made to the top-level workflow builder script, or to the interface between modules (e.g. adding/removing parameters a module uses), it is necessary to recreate these 'xml' workflow description files. This can be easily done with the 'comps/gen_accept_tests.py' script.
Full Validation Tests
The acceptance tests use a small number of stations to validate the codes inside the Broadband Platform. A more extensive test is to run a complete validation tests using a larger number of stations. The Broadband Platform is distributed with 2 validation packages that can be used to make sure everything is working properly. In order to run these validation simulations, users have to make sure they have all the needed dependencies installed on their computer. The Northridge event uses the LABasin velocity model package, and the LomaPrieta event uses the LOMAP velocity model package for Northern California. One important consideration is the time it takes to run some of these validation simulations. This is especially true for the GP, and SDSU methods, which may take up to 72 hours to run each of them. In these cases, it is recommended that the output of the platform is redirected to a log file:
$ cd comps $ ./run_bbp.py -l ~/northridge-gp-log.txt Welcome to the SCEC Broadband Platform. Please select the modules you want to run. Do you want to perform a validation run (y/n)? y Do you want to validate with: (1) Loma Prieta (2) Northridge ? 2 Choose a Method to use in a Broadband validation simulation: (1) GP (Graves & Pitarka) (2) GP Seis (GP precomp seismograms) (3) UCSB (4) SDSU (5) SDSU Seis (SDSU precomp seismograms) (6) EXSIM (7) Composite Source Model (CSM) (8) Irikura Recipe (Irikura) ? 1 Do you want to run a rupture generator (y/n)? y Do you want to provide a custom source file (y/n)? n Do you want to (1) run all validation stations (2) select a custom station list ? 1 Do you want to run the site response module (y/n)? y Do you want to plot velocity seismograms (y/n)? y Do you want to plot acceleration seismograms (y/n)? y Do you want to generate a GMPE comparison plot (y/n)? y Choose a Goodness of Fit (GOF) Module: (1) GP (2) SDSU (3) Both (4) None ? 1 ^Z [1]+ Stopped ./run_bbp.py -l ~/northridge-gp-log.txt $ bg [1]+ ./run_bbp.py -l ~/northridge-gp-log.txt &
Note that after selecting the GOF module, the simulation starts and the user puts the process in background by hitting "CTRL-Z" and then typing 'bg' at the shell.
Update Build Version
Starting with the Broadband Platform 13.6.1 release, the version of the main BBP source distribution and the versions of validation events and velocity model packages are independent. To update the version of the BBP source distribution, just edit the comps/version.txt file.
In order to update the version number of one or more velocity model/validation package, you will need to edit the corresponding configuration file and edit the version line:
VELMODEL_VERSION = 4-March-2014
Tag Version on SVN
After making sure the latest versions of all files are committed to svn, you should create a svn tag with the release version. The svn command to use is:
$ svn copy -m "Tagged Broadband version 14.3.0" . https://source.usc.edu/svn/broadband/tags/Version_14_3_0_2014-03-31
Generating MD5 Sums
Event validation packages and velocity model packages include a set of MD5 sums in order to help catch user modifications to the various input files provided to the Broadband Platform. In order to generate these MD5 sums before a BBP release, the user needs to use the following program:
$ cd checksums $ ./md5sum_check.py -g
Note that running the script without the '-g' option, will cause it to check the previously stored MD5 sum values against the MD5 sum of the actual files in each of the validation/velocity model packages. It will print any discrepancies to the terminal window, signaling a file modification.
Create Distribution Files
The BBP distribution includes a source distribution tar file, and a number of event validation and velocity model tar files.
Generating Source Distribution Tar File
To create the Broadband Platform source distribution tar file, you can use the 'create_source_dist.sh' script, located in the utils/dist directory of the Platform. The script will take an argument specifying the version number, and will create a tar file (without the .svn sub-directories) with the version number and date embedded in the filename. The script should run in the directory above where the "trunk" svn directory is.
$ ./trunk/utils/dist/create_source_dist.sh 14.3.0
The output file will be named like "bbp-14.3.0-23-apr-2014.tar.gz"
Generating Validation Events and Velocity Models Tar Files
In order to generate the tar files for all validation events and velocity model packages, you can use the 'make_packages_release.sh' script located in the utils/dist directory. The script assumes you have a directory somewhere (BBP_ROOT), where both 'bbp_val' and 'bbp_gf' subdirectories are located. The script takes a single parameter, the version number to add to all tar filenames. For example, the following commands:
$ cd $BBP_ROOT $ make_packages_release.sh 14.3.0
will generate all validation and velocity model packages for the 14.3.0 version of the platform:
gmpe-verification-14.3.0.tar.gz whittier-validation-14.3.0.tar.gz nr-validation-14.3.0.tar.gz nps-validation-14.3.0.tar.gz landers-validation-14.3.0.tar.gz lomap-validation-14.3.0.tar.gz niigata-validation-14.3.0.tar.gz tottori-validation-14.3.0.tar.gz rdl1k-validation-14.3.0.tar.gz saguenayhartzell1k-validation-14.3.0.tar.gz mineral-validation-14.3.0.tar.gz alum-rock-validation-14.3.0.tar.gz chino-hills-validation-14.3.0.tar.gz northridge-validation-14.3.0.tar.gz lomaprieta-validation-14.3.0.tar.gz labasin-velocity-model-14.3.0.tar.gz lomap-velocity-model-14.3.0.tar.gz mojave-velocity-model-14.3.0.tar.gz centraljapan-velocity-model-14.3.0.tar.gz westernjapan-velocity-model-14.3.0.tar.gz canada1000-velocity-model-14.3.0.tar.gz ceus1000-velocity-model-14.3.0.tar.gz
Posting Distribution Files
BBP distribution files should be posted on hypocenter.usc.edu. Assuming we are releasing version 14.3.0, you should log into hypocenter.usc.edu, and then:
$ cd /var/www/html/research/bbp/versions $ mkdir 14.3.0 $ cd 14.3.0 $ touch index.html (to disable directory listing!)
Copy the source distribution and all validation/velocity model packages to this directory (often scp is needed for this). On the machine where the source distribution/other packages were created:
$ cd <where BBP source distribution is> $ scp bbp-14.3.0-23-apr-2014.tar.gz user@hypocenter.usc.edu:/var/www/html/research/bbp/versions/14.3.0/. $ cd <where validation/velocity model packages are> $ scp *-14.3.0.tar.gz hypocenter.usc.edu:/var/www/html/research/bbp/versions/14.3.0/.
Back on hypocenter.usc.edu, you will need to generate the md5 sums for all files using the following for loop:
$ for file in `ls | grep tar.gz`; do md5sum $file > $file.md5; done;
