Difference between revisions of "Broadband User Guide v11.2.2"

From SCECpedia
Jump to navigationJump to search
(Created page with ''''Version 11.2.2.''' If you find errors in this document or sections to be unclear, please either fix them yourself or contact Sandarsh Kumar (sandarsh at usc.edu), Scott Calla…')
 
 
(13 intermediate revisions by 3 users not shown)
Line 14: Line 14:
 
Required:
 
Required:
  
*Python 2.7+ with
+
*Python v2.7 with
 
**PyGTK
 
**PyGTK
 
**Matplotlib
 
**Matplotlib
Line 20: Line 20:
 
**Pyproj
 
**Pyproj
  
*Intel compilers (64-bit) v12.x
+
*Intel compilers (64-bit) v12.0.4
*GNU compilers (gcc, gfortran) v4.5+
+
*GNU compilers (gcc, gfortran) v4.5.1
*GNU Fortran 77 v3.4+
+
*GNU Fortran 77 v3.4.6
 
+
*GMT (Generic Mapping Toolkit)
Optional:
+
*ImageMagick
 
 
*GMT tools (for plots)
 
  
 
== Setting Up Your Account ==
 
== Setting Up Your Account ==
Line 93: Line 91:
 
  setenv BBP /home/scec-00/kumar/bbp_2g
 
  setenv BBP /home/scec-00/kumar/bbp_2g
  
==
 
 
* PYTHONPATH
 
* PYTHONPATH
 
After you've obtained a copy of the project, you'll need to make sure the comps directory is on Python's path so Python can find all the project modules. If you're running a Bash shell, add the following line to your .bash_profile with your favorite text editor:
 
After you've obtained a copy of the project, you'll need to make sure the comps directory is on Python's path so Python can find all the project modules. If you're running a Bash shell, add the following line to your .bash_profile with your favorite text editor:
Line 110: Line 107:
 
For Bash Shell (in .bash_profile)
 
For Bash Shell (in .bash_profile)
 
   
 
   
  export PATH=/usr/scec/intel/cce/9.0/bin:/usr/scec/intel/fce/9.0/bin:$PATH
+
  export PATH=/usr/usc/intel/composerxe-2011.4.191/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:$PATH
  
 
For C-shell (in .cshrc)
 
For C-shell (in .cshrc)
 
   
 
   
  setenv PATH /usr/scec/intel/cce/9.0/bin:/usr/scec/intel/fce/9.0/bin:$PATH
+
  setenv PATH /usr/usc/intel/composerxe-2011.4.191/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:$PATH
  
 
When running elements of the platform over ssh, be sure to enable ssh forwarding (with the -X or -Y options).
 
When running elements of the platform over ssh, be sure to enable ssh forwarding (with the -X or -Y options).
Line 191: Line 188:
 
Once you have checked out the code, you need to build it. By default, every executable is compiled using the compiler recommended by the code developer. However, if you have limited compiler options or are building the codes on an untested system, you may need to specify non-standard alternative compilers, as described below.
 
Once you have checked out the code, you need to build it. By default, every executable is compiled using the compiler recommended by the code developer. However, if you have limited compiler options or are building the codes on an untested system, you may need to specify non-standard alternative compilers, as described below.
  
Before you can build the platform, you need to make sure that the Intel compilers are in your path. This is done automatically on intensity.usc.edu, but on other systems you can check by typing:
+
Before you can build the platform, you need to make sure that the Intel compilers are in your path. This is done automatically on broadband.usc.edu, but on other systems you can check by typing:
  
 
  $>which icc
 
  $>which icc
Line 225: Line 222:
  
 
== Advanced Users ==
 
== Advanced Users ==
Alternatively, if you would like access to the latest version of the platform, to get frequent but less thoroughly tested improvements, you can check out the platform from SCEC's Subversion version control repository on intensity.usc.edu. Only advanced users should follow this approach. If you are unsure, you should download the code as outlined in [[#Downloading the Platform|Downloading the Platform]] and skip this section.
+
Alternatively, if you would like access to the latest version of the platform, to get frequent but less thoroughly tested improvements, you can check out the platform from SCEC's Subversion version control repository on source.usc.edu. Only advanced users should follow this approach. If you are unsure, you should download the code as outlined in [[#Downloading the Platform|Downloading the Platform]] and skip this section.
  
 
You may need to request access from the SCEC system administrator, John Yu ([mailto:johnyu@usc.edu johnyu@usc.edu]). Make sure you have subversion in your path before beginning.
 
You may need to request access from the SCEC system administrator, John Yu ([mailto:johnyu@usc.edu johnyu@usc.edu]). Make sure you have subversion in your path before beginning.
Line 237: Line 234:
 
  $> svn checkout https://source.usc.edu/svn/broadband/trunk bbp_2g
 
  $> svn checkout https://source.usc.edu/svn/broadband/trunk bbp_2g
  
This will check out the project to your local machine. It contains the source code, tests, and some example files. You also need to obtain a copy of the Broadband Platform data files, containing the Green's tensors. They are too large to be stored in version control. You can obtain a copy from the [on intensity. If you are working on intensity as well, you can just create a symbolic link:
+
This will check out the project to your local machine. It contains the source code, tests, and some example files. You also need to obtain a copy of the Broadband Platform data files, containing the Green's tensors. They are too large to be stored in version control. You can obtain a copy from SCEC software distribution server. If you are working on SCEC development server, you might be able to create a symbolic link to data folder. Please contact [[#User_Support|User Support]] to get a location of the data folder on the SCEC development server. If you are not working on SCEC development server, or don't have an SCEC Linux login account, you can get a archive (.tgz) file of the Green's functions from the same download site you obtained this guide.
  
$> ln -s /home/scec-00/scottcal/bband/bbp_2g_gf <path to your gf directory>
+
If you know where a read-accessible installation of the green functions data directories you can create a link to that location. This can save disk space in some cases.
  
If you are not working on intensity, or don't have an intensity username, you can get a .tgz file of the Green's functions from the same download site you obtained this guide.
+
ln -s /home/scec-00/scottcal/bband/bbp_2g_gf <path to your gf directory>
  
 
Periodically you should check for updates to the code. To do so, go to your bbp_2g directory and type
 
Periodically you should check for updates to the code. To do so, go to your bbp_2g directory and type
Line 292: Line 289:
  
 
  $> cd tests
 
  $> cd tests
  $> ./AcceptTests.py
+
  $> ./AcceptTests.py &> accept.out &
  
Since they take so long to run, you may want to background them instead and redirect their output:
+
Since the tests take so long to run (twenty hours), you should run them as a background process, and redirect their output to a log file, as shown in the command above.
  
$> ./AcceptTests.py &> accept.out &
+
You can check the progress by tailing the log file.
 +
 
 +
  $> tail accept.out
 +
 
 +
When all the acceptance tests are completed, the AcceptTests.py script will either print "OK" or how many tests failed.  
  
When they're complete, the console will either print "OK" or how many tests failed. Acceptance test failures indicate that the modules are not integrated correctly. Like with the unit tests, verify that you have the ref_data directory.  If a certain acceptance test fails, you can get more information by consulting the acceptance test logs in bbp_2g/logs/acceptance_tests_logs/<test that failed>.log .  If you can't determine the reason for the failure, contact [[support]].
+
Acceptance test failures indicate that the modules are not integrated correctly. Like with the unit tests, verify that you have the ref_data directory.  If a certain acceptance test fails, you can get more information by consulting the acceptance test logs in bbp_2g/logs/acceptance_tests_logs/<test that failed>.log .  If you can't determine the reason for the failure, contact [[support]].
  
 
Since the acceptance tests can take a long time to run, you may wish to resume where you left off. This feature is supported automatically. If you do not wish to resume, delete the resume file in ref_data/accept_inputs/resume.txt. If there are any failure or errors while running the acceptance tests, consult the Troubleshooting section at the end of this user guide for know issues and their solutions.
 
Since the acceptance tests can take a long time to run, you may wish to resume where you left off. This feature is supported automatically. If you do not wish to resume, delete the resume file in ref_data/accept_inputs/resume.txt. If there are any failure or errors while running the acceptance tests, consult the Troubleshooting section at the end of this user guide for know issues and their solutions.
Line 783: Line 784:
 
If you run into bugs with the platform, you can open a trouble ticket at the [http://northridge.usc.edu/trac/broadband Broadband Trac site].
 
If you run into bugs with the platform, you can open a trouble ticket at the [http://northridge.usc.edu/trac/broadband Broadband Trac site].
  
Check to see if there has already been a ticket opened for the bug. If you are unable to get the platform to run, you can get direct user support by emailing [mailto:software@intensity.usc.edu software@intensity.usc.edu].
+
Check to see if there has already been a ticket opened for the bug. If you are unable to get the platform to run, you can get direct user support by emailing [mailto:software@scec.org software@scec.org].
  
 
= Appendix A: Possible Module Permutations =
 
= Appendix A: Possible Module Permutations =
Line 1,483: Line 1,484:
 
'''Solution:''' Ensure that the path Intel 64-bit compilers are listed in you PATH variable and being picked up by running 'which icc' and 'which ifort'. If you are trying to build the platform on SCEC development servers like Intensity, verify your path variable has:
 
'''Solution:''' Ensure that the path Intel 64-bit compilers are listed in you PATH variable and being picked up by running 'which icc' and 'which ifort'. If you are trying to build the platform on SCEC development servers like Intensity, verify your path variable has:
 
<pre>
 
<pre>
  PATH=...:usr/scec/intel/cc/9.0/bin:/usr/scec/intel/fc/9.0/bin
+
  PATH=...:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/bin/intel64
 
</pre>
 
</pre>
  
Line 1,647: Line 1,648:
  
 
'''Solution:''' The stochastic nature of the high frequency simulations is considered, and a constant random seed as input for our tests. This is the same constant random seed used to generate the reference results included in the release. This ensures that the time-series generated with unit tests agree with reference results. However, the high frequency codes are more sensitive to changes in compilers and machines due to the algorithm, so small differences that cause the test to fail are generally acceptable. The user is however encouraged to plot the two time-series and visually inspect them to verify they look similar. We are trying to come up with a better comparison for these tests.
 
'''Solution:''' The stochastic nature of the high frequency simulations is considered, and a constant random seed as input for our tests. This is the same constant random seed used to generate the reference results included in the release. This ensures that the time-series generated with unit tests agree with reference results. However, the high frequency codes are more sensitive to changes in compilers and machines due to the algorithm, so small differences that cause the test to fail are generally acceptable. The user is however encouraged to plot the two time-series and visually inspect them to verify they look similar. We are trying to come up with a better comparison for these tests.
 +
 +
= Appendix C: Technical Notes =
 +
 +
== Running Individual Acceptance Tests ==
 +
 +
The acceptance test script ./tests/AcceptTest.py attempts to run all of the defined tests when executed. However, sometimes it is useful to run an acceptance test in isolation for troubleshooting purposes:
 +
 +
* Identify the test you want to run. Eg: user-URS-UCSB-SDSU-SDSU
 +
* Move to the ./comps directory and execute that test's XML workflow with run_bbp_2G.py. Using the above example:
 +
<pre>
 +
$ ./run_bbp_2G.py -x ../ref_data/accept_inputs/user-URS-UCSB-SDSU-SDSU.xml
 +
</pre>
 +
 +
Since you are running the test outside of AcceptTest.py, you will not see a PASS/FAIL message once the simulation completes. However, you can review the contents of the ./outdata/<SIMID> directory to verify that the correct seismograms and response spectra tables were produced.
 +
 +
== Manaully Setting Random Number Generator Seed Values ==
 +
 +
Several of the platform codebases employ random number generators to produce SRFs and seismograms. These random number generators are generally seeded with an initial value (typically an integer) which are hard-coded in the platform. Advanced users may desire to change these seed values.
 +
 +
Changing the seeds for seismogram synthesis varies depending on which codebase you want to use:
 +
 +
* URS: Edit ./comps/hfsims_cfg.py, Line 43 "self.RAND_NUM"
 +
* UCSB: Edit ./src/ucsb/Syn1DSCEC/syn1D_LAH.f, Line 183 "iseed", and recompile with "make"
 +
* SDSU:
 +
** User event: Edit ./bbp_2g_gf/sdsu/scattering_generic.dat (seed values annotated in file)
 +
** Validation event: Edit ./bbp_2g_gf/sdsu/<event>/scattering.dat

Latest revision as of 17:08, 15 July 2014

Version 11.2.2.

If you find errors in this document or sections to be unclear, please either fix them yourself or contact Sandarsh Kumar (sandarsh at usc.edu), Scott Callaghan (scottcal at usc.edu) or Philip Maechling (maechlin at usc.edu).

Contents

Installing the Second-Generation Broadband Platform

Installing the Broadband Platform involves obtaining a copy of the code and building the required executables. You can either download the platform from the Broadband web site (http://www.scec.org/research/broadband) or check the code out of SCEC's Subversion repository. Most users should download the platform.

System Dependencies

The current version of the Broadband Platform is designed to run on standard 64-bit Linux machines. Testing has been performed on SCEC's development servers running Fedora Core 10 (kernel version 2.6.27.41-170.2.117.fc10.x86_64). In this guide we outline how to install the platform into your own account on a Linux computer using the simplest approach.

Software Dependencies

The Broadband Platform has certain software dependencies.

Required:

  • Python v2.7 with
    • PyGTK
    • Matplotlib
    • Numpy
    • Pyproj
  • Intel compilers (64-bit) v12.0.4
  • GNU compilers (gcc, gfortran) v4.5.1
  • GNU Fortran 77 v3.4.6
  • GMT (Generic Mapping Toolkit)
  • ImageMagick

Setting Up Your Account

For simplicity of installation, we recommend users use a bash shell for the Broadband Platform account. It is possible to get the platform running using other shell's, but we will focus on a bash shell installation. The user environment is a common source of problems since certain environment variables must be defined for the platform to work correctly.

To check your account, make sure you can run basic commands like ls and cd before proceeding.

Downloading the Platform

Download 4 files from the Broadband website, the code (bbp_dist_<version>.tgz), the data (bbp_data_<version>.tgz), and their checksum files (bbp_dist_<version>.tgz.md5 and bbp_data_<version>.tgz.md5). The code file is about 100 MB, the data file about 3 GB. After you've downloaded the files to your local Linux system, the next step is to calculate the checksums yourself and compare them to the checksums you downloaded.

First, verify that the md5sum command is in your path:

$> which md5sum

You should get something like /usr/bin/md5sum. If you see the message 'no md5sum in...', contact your Linux system administrator and ask to have md5sum added to your path.

Once you can run the md5sum command, run:

$> md5sum -c bbp_dist_<version>.tgz.md5
$> md5sum -c bbp_data_<version>.tgz.md5

You should get the messages

bbp_dist_<version>.tgz.md5: OK
bbp_data_<version>.tgz.md5: OK

If you get FAILED instead, re-download the tgz files and try again. When it passes, that means the files were downloaded without error.

Once both files have passed the checksum test, untar the files.

$> tar xzvf bbp_dist_<version>.tgz file
$> tar xzvf bbp_data_<version>.tgz file

If multiple users are planning to use the platform on the same system, you only need one copy of the data files per machine. Each user will still need his or her own copy of the code files.

Alternatively, if you would like access to the latest version of the platform and get frequent but less thoroughly tested improvements, you can check out the platform from SCEC's Subversion repository. Only advanced users should take this approach, outlined in detail in the Advanced Users section.

User Account Setup

The Broadband platform installation is divided into two parts (1) input Green Functions (GF) data directory (9.3Gb), and (2) BBP home directory (366Mb).

First, the Green's function libraries are larger, but they are static. The broadband platform reads the data files, but does not change them.

Second, the remainder of the broadband platform is organized in the BBP home directory. The BBP home directory has a specific directory structure that includes the source code for the scientific programs, the python scripts that link the scientific programs, simulation input directory, temporary and log file directories, and the output data directory where all the platform results are written.

The BBP home directory will increase in size as you run the platform, because both output data and output log files are written to the home directory. Running the acceptance tests will produce nearly ( 10Gb) of input data (5.2Gb), output data (500Mb), temporary files (2.7Gb), and output log files (250Mb). Once these tests have passed much of this data can be removed. However, the BBP home directory should have at least 10Gb of disk space to insure the acceptance tests can be run when a particular broadband platform software distribution is first installed on a system.

Data Directory

Input data files called Greens Functions are distributed with the platform. These files are generated using specific velocity structures, so they are often considered region specific. We have GF for three regions including near Loma Prieta, near Landers, and near Northridge.

The data directory is static, and the data directories are read-only. A single copy can be installed in a shared disk, which can then be shared by multiple users.

Setting Environment Variables

The BBP source codes and scripts are organized under the broadband platform home directory. The broadband platform home directory is specified in a couple of places during Broadband Platform installation.

Internal to the Broadband platform software, all broadband platform files (except the Greens Function data files) are in sub-directories of the BBP home directory.

BBP Environment Variable

We recommend setting both an alias and a environment variable to the BBP home directory. This will help avoid typing it many times.

If you're running a Bash shell, add the following line to your .bash_profile with your favorite text editor:

export BBP=/home/scec-00/kumar/bbp_2g

If you're running a C-shell, add the following line to your .cshrc with your favorite text editor:

setenv BBP /home/scec-00/kumar/bbp_2g
  • PYTHONPATH

After you've obtained a copy of the project, you'll need to make sure the comps directory is on Python's path so Python can find all the project modules. If you're running a Bash shell, add the following line to your .bash_profile with your favorite text editor:

export PYTHONPATH=$BBP/comps:$PYTHONPATH

If you're running a C-shell, add the following line to your .cshrc with your favorite text editor:

setenv PYTHONPATH $BBP/comps:$PYTHONPATH
  • PATH

In order to successfully compile the project, you'll need to make sure the required compilers directories are in your PATH variable. Broadband requires Intel 64-bit compilers, icc and ifort and GNU Compilers, gcc, g77 and f77 to compile the scientific code.

If you are planning to run Broadband on SCEC Development servers, make sure you have the following directories in your PATH:

For Bash Shell (in .bash_profile)

export PATH=/usr/usc/intel/composerxe-2011.4.191/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:$PATH

For C-shell (in .cshrc)

setenv PATH /usr/usc/intel/composerxe-2011.4.191/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:$PATH

When running elements of the platform over ssh, be sure to enable ssh forwarding (with the -X or -Y options).

After modifying your login script above, log out and log back into the machine so the changes are reflected in your environment.

Edit Install_cfg.py with Installation Directory Paths

You need to tell the platform where it's installed by editing a single python file called "install_cfg.py". Edit the file bbp_2g/comps/install_cfg.py with your favorite text editor, and edit the lines:

self.A_INSTALL_ROOT = <bbp_2g directory>
self.A_GF_DIR = <bbp_2g_gf directory>

as one example:

self.A_INSTALL_ROOT = /home/scec-00/kumar/bbp_2g
self.A_GF_DIR = /home/scec-00/kumar/bbp_2g_gf

with the paths on your system to the bbp_2g directory (the source) and the bbp_2g_gf directory (the data) that you unzipped. For example, it could be /home/scottcal/broadband_platform/bbp_2g and /home/scottcal/broadband_platform/bbp_2g_gf.

Here is some information about these two directories that may be useful as you decide how, and where, to install these two parts of the Broadband Platform on your computer disk system. The data files, and therefore the <bbp_2g_gf directory> are nearly 3GB, but they are static and will not be modified and will not grow in size during use of the platform. The source directory is small to begin. However, this directory will increase as the platform is used, since the results produced by the platform will be stored here.

Directory Structure

The platform consists of two top-level directories, bbp_2g and bbp_2g_gf. bbp_2g contains the source code, executables, scripts, tests, input, working, and output directories. bbp_2g_gf contains the Green's Functions, input files for the validation events, and other required input files for the various code bases. Note that indata, logs, outdata, tmpdata, and xml are created when the platform is first run, so they will be missing when you first install the platform.

bbp_2g has the following directories:

  • checksums: Contains checksums for bbp_2g_gf files
  • comps: The Python scripts to run the platform
  • docs: Documentation for the platform
  • etc: Miscellaneous utility scripts
  • examples: Contains example input files
  • indata: An internal directory, used to stage input files
  • logs: Contains logs from BBP runs
  • outdata: Contains output files from a run
  • ref_data: Contains reference files for BBP tests
  • start: Put input files for an interactive run here
  • src: Source code for BBP modules
  • tests: Contains unit and acceptance tests
  • tmpdata: An internal directory, used during a run
  • xml: Contains XML files which describe simulations and can be used as input

bbp_2g_gf has the following directories:

  • compare: Contains observed seismograms for validation events
  • plot: Data files for GMT plots
  • sdsu, ucsb, urs: Contains Green's functions, velocity files, and other required inputs for the codebases.

In general, you will be interacting with the start directory for input files, comps to run the platform, tests to test the platform, and outdata to examine data products.

Adding aliases

You may find it helpful to add aliases, so you can quickly and easily move to different broadband directories with a single command. We recommend creating aliases for the home, start, and outdata directories.

If you are using the Bash shell, you can create aliases by adding the following lines to ~/.bash_profile:

alias bbp= 'cd <path to bbp_2g directory>'
alias start='cd <path to bbp_2g directory>/start'
alias outdata='cd <path to bbp_2g directory>/outdata'

If you're using a C shell, edit your ~/.cshrc and add:

alias bbp cd <path to bbp_2g directory>
alias start cd <path to bbp_2g directory>/start
alias outdata cd <path to bbp_2g directory>/outdata

Log out and log back in. You'll notice that now you can type the alias command as a shortcut to change directories:

$> pwd
/home/scec-00/scottcal
$> start
$> pwd
/home/scec-00/scottcal/bband/bbp_2g/start

This can be a useful way to navigate around the broadband platform directories.

Building the Platform

Once you have checked out the code, you need to build it. By default, every executable is compiled using the compiler recommended by the code developer. However, if you have limited compiler options or are building the codes on an untested system, you may need to specify non-standard alternative compilers, as described below.

Before you can build the platform, you need to make sure that the Intel compilers are in your path. This is done automatically on broadband.usc.edu, but on other systems you can check by typing:

$>which icc

If you get the message "no icc in ...", then you'll need to add the Intel compilers to your path. Once the Intel compilers are in your path, you can make the code by cd-ing to the bbp_2g/src directory and typing make:

$> cd src
$> make

It takes a minute or two to build the code. You may encounter build warnings; these are fine. However, if you get any build errors, this is a problem and should be investigated.

By default, every executable is compiled using the compiler recommended by the code developer. Depending on the system, some compilers may not be available to you. You can override the C and Fortran compilers used by editing

src/makefile

Uncomment USER_C and set FC and CC to the compilers you wish. For example:

Before:

#USER_C=1
FC=f77
CC=gcc

After (an example, you may choose different compilers):

USER_C=1
FC=gfortran
CC=gcc

Note that not all compiler combinations have been thoroughly tested. You may encounter build errors with untested compiler combinations. If you encounter any errors while building the platform, consult the Troubleshooting section at the end of this user guide for know issues and their solutions.

Once the platform has been successfully built, you can move on to running the tests to verify that all components are working correctly.

Advanced Users

Alternatively, if you would like access to the latest version of the platform, to get frequent but less thoroughly tested improvements, you can check out the platform from SCEC's Subversion version control repository on source.usc.edu. Only advanced users should follow this approach. If you are unsure, you should download the code as outlined in Downloading the Platform and skip this section.

You may need to request access from the SCEC system administrator, John Yu (johnyu@usc.edu). Make sure you have subversion in your path before beginning.

Once you have access, you'll need to decide if you want a stable tagged version, or the latest version in the repository. For a stable version, open a terminal window on the system of your choice and type:

$> svn checkout https://source.usc.edu/svn/broadband/tags/<stable version> bbp_2g

For the latest version, type:

$> svn checkout https://source.usc.edu/svn/broadband/trunk bbp_2g

This will check out the project to your local machine. It contains the source code, tests, and some example files. You also need to obtain a copy of the Broadband Platform data files, containing the Green's tensors. They are too large to be stored in version control. You can obtain a copy from SCEC software distribution server. If you are working on SCEC development server, you might be able to create a symbolic link to data folder. Please contact User Support to get a location of the data folder on the SCEC development server. If you are not working on SCEC development server, or don't have an SCEC Linux login account, you can get a archive (.tgz) file of the Green's functions from the same download site you obtained this guide.

If you know where a read-accessible installation of the green functions data directories you can create a link to that location. This can save disk space in some cases.

ln -s /home/scec-00/scottcal/bband/bbp_2g_gf <path to your gf directory>

Periodically you should check for updates to the code. To do so, go to your bbp_2g directory and type

$> svn update

This will pull down any code updates that have been made in the repository. After you update, make the code again (detailed in Building the Platform) so that any code changes are captured in the executables.

Running the Broadband Platform

We recommend users initiate all Broadband Platform runs from the "start" directory. The start directory is a collection point for input files for a particular simulation.

Assuming the user is starting at BBP home directory, the user should move to the start directory and then run the Checksum, Unit and Acceptance tests to confirm the Broadband Platform software installation is operating as expected.

$> cd start

Running Checksum, Unit, and Acceptance Tests

The broadband platform contains three kinds of tests. The checksum tests verify that the data files were copied correctly and you have all the files that are expected. Unit tests run each module using a set of input files, and compare the results against known outputs. They verify that each module has been built and is working correctly. Acceptance tests verify that the modules are working correctly together. They test the platform end-to-end using different combinations with known inputs and compare the results. All the possible module combinations for both user-defined and validation events are tested as checks against integration errors.

Begin with the checksum tests. Once they past, run the unit tests and finally the acceptance tests.

Running Checksum Tests

To run the checksum tests, change to the checksums directory and run md5sum_check.py.

From the BBP home directory:

$> cd checksums
$> ./md5sum_check.py

It takes about 5 minutes to compute the checksums. If a checksum disagrees, that means that the contents of the datafile were not what was expected. Try reinstalling the data files and try again. If you continue to get a checksum error, contact support.

Once the checksums agree, move on to the unit tests.

Running Unit Tests

To run the unit tests, change to the tests directory and run UnitTests.py.

From the BBP home directory:

$> cd tests
$> ./UnitTests.py

The unit tests take about 15 minutes to run. When they're complete, the console will either print "OK" or how many tests failed. If a test has failed, first check that you have built the executables.You can rerun just the specific test that failed (test_<module>.py). If the test is still failing, also verify that you have the ref_data directory, since it contains the input and reference files. If you're looking for more information about the failure, you can consult the Unit Tests log file in bbp_2g/logs/unit_test.log . If you can't determine the reason for the failure, contact support.

Once the unit tests all pass, proceed to the acceptance tests. If there are any failure or errors while running the unit tests, consult the Troubleshooting section at the end of this user guide for know issues and their solutions.

Running Acceptance Tests

Make sure the unit tests pass before moving on to the acceptance tests. To run the acceptance tests, change to the tests directory (if you're not there already) and run AcceptTests.py. The acceptance tests take a long time to run, around 20 hours

From the BBP home directory:

$> cd tests
$> ./AcceptTests.py &> accept.out &

Since the tests take so long to run (twenty hours), you should run them as a background process, and redirect their output to a log file, as shown in the command above.

You can check the progress by tailing the log file.

 $> tail accept.out

When all the acceptance tests are completed, the AcceptTests.py script will either print "OK" or how many tests failed.

Acceptance test failures indicate that the modules are not integrated correctly. Like with the unit tests, verify that you have the ref_data directory. If a certain acceptance test fails, you can get more information by consulting the acceptance test logs in bbp_2g/logs/acceptance_tests_logs/<test that failed>.log . If you can't determine the reason for the failure, contact support.

Since the acceptance tests can take a long time to run, you may wish to resume where you left off. This feature is supported automatically. If you do not wish to resume, delete the resume file in ref_data/accept_inputs/resume.txt. If there are any failure or errors while running the acceptance tests, consult the Troubleshooting section at the end of this user guide for know issues and their solutions.

Types of Broadband Platform Simulations

The platform supports two kinds of simulations, validation events and user-defined events. Validation simulations are performed using a historical event, and are directly compared to observed seismograms using goodness-of-fit. User-defined events are run using a rupture description provided by the user which may not necessarily be a historical earthquake.

When you run a simulation, the platform assigns an ID to it. This ID can be used to track the simulation and locate the output data products.

To supply input files to the platform, put them in the start directory. Extensions are important - the platform recognizes station lists (.stl), SRF files (.srf), and simple source descriptions (.src). If there are multiple files of a type, the platform will prompt the user to select one.

To perform a simulation, a user selects between different required and optional modules. Below we present a summary of the various modules.

Modules

The broadband platform consists of a series of modules. There are two main types of modules, science modules and utility modules. Science modules are those for which the platform has multiple implementations, provided by different coding research groups. Utility modules only have 1 implementation. A schematic of the available modules and their flow relationships is shown below:

Science Modules

All simulations must include a low-frequency, high-frequency, and site response science module. Simulations may include an optional rupture generation science module. Users may select between the following different implementations of each of these modules:

Rupture generation:URS, UCSB

Low-frequency:URS, UCSB

High-frequency:URS, UCSB, SDSU

Site response:URS, UCSB, SDSU

Utility Modules

A spectral response utility module is automatically run after the site response module. Additionally, users may select an optional goodness-of-fit utility module to run at the conclusion of the simulation. In the case of a user-defined event, users select a historical event or previously run simulation to use as the comparison for goodness-of-fit.

Platform Directory

A broadband platform distribution is organized in a specific directory structure that separates the input data sets, the python scripts, the simulation configuration files, log files, and output data files.

When running Broadband Platform simulations, standard procedure is to place the simulation input files in the start directory and invoke the Broadband platform scripts from there.

From the BBP home directory:

$> cd start $> run_bbp_2G.py

Simulation output results will be written into the bbp_home/output

Validation Simulations

To run a validation simulation, go to the comps directory and run run_bbp_2G.py. The platform will ask you a series of questions. Answer 'y' to "Do you want to perform a validation run?"

From the BBP home directory:

$> cd start
$> run_bbp_2G.py
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? y
Do you want to validate with:
Northridge (1)
Loma Prieta (2)
Landers (3)
?
...

No input files are required by the user. However, you may wish to run with a reduced station list to speed up the computations. You can put your own station list into the start directory (the format is described in section 5.3). Note that any stations which do not have observed seismograms will not be included in the automatically generated goodness-of-fit comparison.

In addition to the low-frequency modules which compute seismograms using 1D Green's Tensors, validation events can also be run using precomputed 3D seismograms to supply the low-frequency.

User-defined Simulations

To run a user-defined simulation, two input files are required, a rupture description and a station list. The rupture description can either be in SRF format or a simplified source description (the format is described in section 5.1). To run a user-defined simulation, run run_bbp_2G.py.

From the BBP home directory:

$> cd start
$> run_bbp_2G.py
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? y
Do you want to run a rupture generator (y/n)? 
...

You may then choose if you want to run a rupture generator. This is necessary if you're supplying a simple rupture description. If you're supplying an SRF file, you can skip this. Next you can select from different module implementations. Finally, you have the option to run goodness-of-fit against either observed seismograms or another simulation.

...
Do you want to run goodness of fit (y/n)? y
What reference files would you like to compare to? Enter a simulation ID or a scenario name: <comparison>
Enter a label to use for the comparison seismograms: <label>

The label will be used in GoF plots to distinguish one set of results from the other.

The user-defined events must be within one of the regions supported by the platform - Southern California, Northern California, or the Mojave. The platform will automatically select a region based on the event hypocenter. If the event falls outside of these regions, you'll get an error.

Logging

During the run, log files will be produced in logs/<simulation ID>/<simulation ID>.<module name>.log. If the platform fails, this is a good place to look to determine the error. Additionally, any fatal errors will be recorded in fatal_error.log.

Metadata capturing all the executable calls is located in tmpdata/<simulation ID>/metadata.txt for careful tracing of exactly what was called. Both the log files and metadata can be useful if troubleshooting an issue.

Data Products

The platform produces a variety of data products. All data products are located in outdata/<simulation ID>. On most Linux systems, you can show images using display:

$> display <PNG file>

Make sure you have X11 forwarding enabled.

Station map

To help visualize the stations in relationship to the fault, the platform produces a PNG file displaying station locations with red circles and the fault plane with a black line, on an image of California. You can find this file in outdata/<simulation ID>/station_map.png.

Seismograms

When running the platform, you have the option to output velocity and acceleration seismograms, for each station. Plots of these files can be found in outdata/<simulation ID>/<simulation ID>.<station>_<velocity or acceleration>_seis.png.

The raw seismogram data is available in outdata/<simulation ID>/<simulation ID>.<station>.vel.bbp (velocity) and outdata/<simulation ID>.<station>.acc.bbp (acceleration). Its format is described in section 5.4.

To compare seismogram data, you can run comps/plotit.py with two seismograms. It will plot them both for comparison. From the comps directory, run:

$> plotit.py <path to seismogram 1> <path to seismogram 2>

Response spectra

The respect code, run at the end of each simulation, calculates the response spectra for each station. The raw respect data is located at

From the BBP home directory:

outdata/<simulation ID>/<simulation ID>.<station>.rsp

in the format described in section 5.5.

Goodness-of-fit

If you run goodness-of-fit, several additional data products are produced. The goodness-of-fit comparison is performed by comparing the response spectra of a set of calculated seismograms to seismograms from another simulation or observed seismograms. For each station involved in the comparison, a plot comparing the response spectra can be found at outdata/<simulation ID>/<comparison label>_<simulation ID>_<station>_rsp.png. A plot showing the seismograms on top and bottom can be found at outdata/<comparison label>_<simulation ID>_<station>_overlay.png. The goodness-of-fit plot can be found at gof-<comparison label>-<simulation ID>_r0-<cutoff distance>.png.

Note that at least 3 stations must be run for goodness-of-fit to be valid. If fewer than 3 stations are run, no goodness of fit calculation will be performed.

Rupture files

When a user-defined event is simulated, the user has the option to run a rupture generator. This generator produces an SRF file, found in outdata/<simulation ID>/*.srf. This file can be put in the start directory and used in future runs. Additionally, the platform produces a plot of the cumulative slip on the fault surface, outdata/<simulation ID>/<SRF prefix>.png.

Platform Modes

The platform can be run in multiple modes. The default is interactive mode, in which the user is prompted to answer a series of questions. Once all the information has been gathered, the run begins.

For a large number of runs, or if the user is repeating a specific run, this can be tedious. The platform provides two other ways to describe a run, with an option file or an XML description.

An option file provides responses to all the questions that the platform poses. The format is described in section 5.6, but it's basically a text file, 1 entry per line, with support for comments. It can be fed to the platform using the -o option.

The platform will also accept XML files containing a full description of a run. The schema for these files is given in section 5.7. These files are also produced by the platform after every simulation, and placed in xml/<simulation ID>.xml. So if you want to rerun a simulation, you can point the platform to the XML file from that simulation using the -x option. Note that a new simulation ID will be assigned to the run, so there is no risk of overwriting previous simulation results.

Available Options

To get a list of the current available options, run run_bbp_2G.py with the -h flag.

$> run_bbp_2G.py -h
Usage: run_bbp_2G.py [options]
Options:
 -h, --help                        Show this help message and exit
 -x XML_FILE, --xml-file=XML_FILE  Run using XML description of workflow
 -s SIM_ID, --simID=SIM_ID         Force a simID
 -o OPTFILE, --option-file=OPTFILE File containing responses to interactive platform prompts
 -v, --version                     Broadband platform version
 -c, --compact                     Compact output results. Remove temporary files for simulations that complete successfully.
 -g, --generate-xml                Generates the XML description but does not run the platform
 -l LOG_FILE, --log=LOG_FILE       Store all output in a file

Cleanup

After a while, you may find that your indata, tmpdata, outdata, and logs directories are getting cluttered with files from past runs. If you wish to delete these old files, you can use the clean_directories.sh script in the etc directory.

From the BBP home directory:

$> cd etc
$> ./clean_directories <days>

Any simulation ID directories in the indata, tmpdata, outdata, and logs directories which were last accessed <days> days or longer ago will be deleted. If you have a few directories you want to keep, either copy them somewhere else, or copy them to a directory name that's not all digits and they won't be deleted.

Examples

Below are some examples that you can try using the sample files in the examples directory. Make sure all the tests pass before you try this. You should be in the comps directory when you start these examples:

From the BBP home directory:

$> cd comps

Sample Validation Run

You don't need to move any files for this. Notice that 3D seismograms are a low-frequency option, since you're running a validation event.

From the BBP home directory:

$> cd start
$> run_bbp_2G.py 
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? y
Do you want to validate with:
Northridge (1)
Loma Prieta (2)
Landers (3)
?1
Do you want to
run all validation stations (1)
select a station list (2)
?1
Choose a source of low frequency seismograms:
URS module (1)
URS 3D seismograms (2)
UCSB module (3)
SDSU module (4)
SDSU 3D seismograms (5)
?2
Choose a high frequency module:
URS (1)
UCSB (2)
SDSU (3)
?1
Choose a site response module:
URS (1)
UCSB (2)
SDSU (3)
?1
Do you want to plot velocity seismograms (y/n)? y
Do you want to plot acceleration seismograms (y/n)? y

This simulation takes about 10 minutes. Once it's complete the platform will tell you:

You can find results in <path to bbp_2g install>/outdata/<simulation ID> .

In that directory you will find:

  • Velocity seismograms (<simulation ID>.<station>.bbp)
  • Acceleration seismograms (<simulation ID>.<station>.acc.bbp)
  • Plots of velocity seismograms (<simulation ID>.<station>_velocity_seis.png)
  • Plots of acceleration seismograms (<simulation ID>.<station>_acceleration_seis.png)
  • Response spectra files (<simulation ID>.<station>.rsp)
  • Plots comparing simulated and observed seismograms (Northridge_<simulation ID>_<station>_overlay.png)
  • Plots comparing simulated and observed response spectra (Northridge_<simulation ID>_<station>_rsp.png)
  • Overall goodness-of-fit plots (gof-Northridge-<simulation ID>_r0-25.png)

Sample Validation run, custom stations

If the validation runs aren't performed with 3D low-frequency, it can take a long time, up to 10 minutes per station. Sometimes you might want to run with a reduced station list so the simulation will run faster.

Copy the files in example/valid_custom_stations into the start directory. Take a look at the format of the station file:

$> more valid_test_stat.stl 
#Required: lon, lat, station name, distance to fault plane, Vs30
#Optional: low freq corner, high freq corner
#Unused: station information
#lon lat stationdist Vs30 LF corner HF corner station information
-118.6417 34.5640 cast 20.47 450 0.120 23.0 CSMIP peer.berkeley.edu/nga
-118.4180 34.0628 lacn 22.82 278 0.140 23.0 CSMIP peer.berkeley.edu/nga
-118.8811 34.2886 moor 24.16 405 0.160 23.0 CSMIP peer.berkeley.edu/nga

Now, run the platform, using a station list:

From the BBP Home directory

$> cd start
$> run_bbp_2G.py 
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? y
Do you want to validate with:
Northridge (1)
Loma Prieta (2)
Landers (3)
?1
Do you want to
run all validation stations (1)
select a station list (2)
?2

If you have multiple station list files in your start directory, you may see the following prompt asking you to select one. If you only have one, the platform will select it automatically.

Found multiple BBP station list files in the start directory. Please select one:
nr_one_stat.stl (1)
valid_test_stat.stl (2)
?2
Choose a source of low frequency seismograms:
URS module (1)
URS 3D seismograms (2)
UCSB module (3)
SDSU module (4)
SDSU 3D seismograms (5)
?3
Choose a high frequency module:
URS (1)
UCSB (2)
SDSU (3)
?2
Choose a site response module:
URS (1)
UCSB (2)
SDSU (3)
?2
Do you want to plot velocity seismograms (y/n)? y
Do you want to plot acceleration seismograms (y/n)? y

Again, when the run completes in about 15 minutes you can find results in the output directory. You'll notice far fewer files, as only 3 stations were run instead of 133. The goodness-of-fit plots won't look very good - more stations are really needed to get an accurate plot.

Sample User-defined run with source description

Next let's try running a user-defined event. Copy examples/user_rup_gen/nr_one_stat.stl and examples/user_rup_gen/user_eq.src to the start directory. user_eq.src is a simple source description. Its format is outlined in section 5.1. Since this is a user-defined event, precomputed 3D seismograms aren't a valid option.

$> run_bbp_2G.py 
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? n
Do you want to run a rupture generator (y/n)? y
Rupture generators:
URS (1)
UCSB (2)
?1
Using region: Southern California
Choose a low frequency module:
URS (1)
UCSB (2)
?2
Found multiple BBP station list files in the start directory. Please select one:
nr_one_stat.stl (1)
valid_test_stat.stl (2)
?1
Choose a high frequency module:
URS (1)
UCSB (2)
SDSU (3)
?2
Choose a site response module:
URS (1)
UCSB (2)
SDSU (3)
?1
Do you want to plot velocity seismograms (y/n)? y
Do you want to plot acceleration seismograms (y/n)? y
Do you want to run goodness of fit (y/n)? n

Since this run only includes one station, it will run in about 5 minutes. In the output directory you'll notice there are no goodness-of-fit or files, since we didn't run goodness-of-fit. However, there is also a map file (station_map.png), showing the fault plane and the stations, and a plot of the rupture slip (user_eq.png). The SRF generated by the rupture generator is in user_eq.srf; this file could be used in future runs. The filenames of the rupture slip plot and SRF are taken from the rupture description filename.

Sample User-defined run with SRF file and goodness-of-fit

Finally, try a run with an SRF file and run goodness-of-fit on the results. Copy in examples/user_no_rup_gen/ucsb_modSRF_0.srf and examples/user_no_rup_gen/nr_three_stat.stl

$> run_bbp_2G.py 
Welcome to the SCEC Broadband Platform.
Please select the modules you want to run.
Do you want to perform a validation run (y/n)? n
Do you want to run a rupture generator (y/n)? n
Using region: Southern California
Choose a low frequency module:
URS (1)
UCSB (2)
?1
Found multiple BBP station list files in the start directory. Please select one:
nr_one_stat.stl (1)
valid_test_stat.stl (2)
nr_three_stat.stl (3)
?3
Choose a high frequency module:
URS (1)
UCSB (2)
SDSU (3)
?3
Choose a site response module:
URS (1)
UCSB (2)
SDSU (3)
?1
Do you want to plot velocity seismograms (y/n)? y
Do you want to plot acceleration seismograms (y/n)? y
Do you want to run goodness of fit (y/n)? y
What reference files would you like to compare to? Enter a simulation ID or a scenario name: Northridge
Enter a label to use for the comparison seismograms: nr

The reference files can either be a scenario event or a simulation ID previously run. The label is just for the figures when comparisons are plotted.

This run will take about 15 minutes, and the results can be found in the appropriate outdir. Again, the goodness of fit measures will look poor due to the few stations involved in the comparison. If you had fewer than 3 stations, no goodness-of-fit plots would be produced. However, response spectra comparison plots will still be generated.

File Formats

This section offers descriptions of various file formats in the broadband platform.

Simple source description

This is the format for the simple source description. It is not whitespace sensitive. The filename must end in .src for the platform to identify it as a source description.

MAGNITUDE = <magnitude>
FAULT_LENGTH = <length of fault>
DLEN = <DX, along length>
FAULT_WIDTH = <down-dip fault width>
DWID = <DY, along width>
DEPTH_TO_TOP = <depth of fault below surface>
STRIKE = <strike>
RAKE = <rake>
DIP = <dip>
LAT_TOP_CENTER = <latitude of top center point in fault plane>
LON_TOP_CENTER = <longitude of top center point in fault plane>
HYPO_ALONG_STK = <along strike location of hypocenter (0.0 = middle)>
HYPO_DOWN_DIP = <down dip location of hypocenter (0.0 = top)>
DT = <dt>
SEED = <random seed used in calculations>
CORNER_FREQ = <corner frequency to use>

SRF rupture description

Details of the Standard Rupture Format (SRF) are given in more detail in the accompanying document. The general fault surface is represented by a distribution of point sources, and it can support one or more planes. The platform produces SRFs as output from rupture generators, but a user can also supply an SRF file as input for user-defined simulations. SRF filenames must end with the suffix .srf for the platform to correctly identify them.

Station File List

The user may specify a list of stations, matching the following format:

#optional
%comment
# lines
<lon> <lat> <stat name> <distance to fault trace> <Vs30> <LF corner> <HF corner> <stat info>
...

The station list filename needs to end in .stl for the platform to recognize it as a station list.

BBP seismograms

All intermediate and output seismograms are produced in a 4-column text format, whether velocity or acceleration.

# optional
% comment
# lines
<timestamp> <North/South> <East/West> <Up/Down>
...

Time is in seconds and motion is in cm/s (velocity) or cm/s/s (acceleration) unless otherwise specified in the header. Velocity filenames end in .vel.bbp; accelerations in .acc.bbp.

Response spectra

The response spectra file is in 4-column text format.

# optional
% comment
# lines
<period (sec)> <North/South> <East/West> <Up/Down>
...

The platform samples response spectra at 112 periods, 0.01-20 sec and outputs pseudo spectral acceleration in units of g.

Option File

Option files contain responses to the prompts in the platform. You can use them with the -o option to run_bbp_2G.py. The responses have to match the prompts exactly. For example, if you craft an option file assuming there is only one station list in the start directory (and therefore the user isn't prompted to select one) but then it is run with multiple station lists in the start directory, this will cause the run to abort. Option files are recommended to perform a series of runs with different inputs or different modules. If you have a run you perform regularly, it's better practice to run the platform with the -g option and the option file to produce an XML description, which are more complete descriptions of the workflow.

#optional comments
%using either symbol
n #comments
y % can go
1 # here
...

Use with:

$> run_bbp_2G.py -o <option file>

XML workflow description schema

Each time the platform is invoked, an XML file is produced describing the workflow, obeying the following schema:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name= "BBP_Run_Specification" minOccurs= "1" maxOccurs= "1">
  <xs:attribute name="version" type = "xs:string" use = "required" />
  <xs:complexType>
    <xs:sequence>
      <xs:element name= "Validation_Run" minOccurs= "0" maxOccurs = "1">
        <xs:attribute name= "event" type= "xs:string" use= "required" />
        <xs:attribute name= "input_station_file" type = "xs:string" />
        <xs:attribute name= "subset" type = "xs:boolean" use = "required" />
      </xs:element>
      <xs:element name= "BBP_Modules">
        <xs:complexType>
          <xs:sequence>
            <xs:element name = "BBP_Module" maxOccurs = "unbounded" >
              <xs:complexType>
                <xs:sequence>
                  <xs:element name= "name" type="xs:string" />
                  <xs:element name= "staged_files" >
                    <xs:complexType>
                      <xs:sequence>
                        <xs:element name="file" type="xs:string" maxOccurs= "unbounded"/>
                      </xs:sequence>
                    </xs:complexType>
                  </xs:element>
                  <xs:element name= "arguments" >
                    <xs:complexType>
                      <xs:sequence>
                        <xs:element name="argument" type="xs:string" maxOccurs="unbounded">
                          <xs:attribute name= "type" type= "xs:string" user= "required" />
                        </xs:element>
                      </xs:sequence>
                    </xs:complexType>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:sequence>
  </xs:complexType>
</xs:element>

These files are difficult to construct by hand, but can be generated using the -g flag when running run_bbp_2G.py.

User Support

If you run into bugs with the platform, you can open a trouble ticket at the Broadband Trac site.

Check to see if there has already been a ticket opened for the bug. If you are unable to get the platform to run, you can get direct user support by emailing software@scec.org.

Appendix A: Possible Module Permutations

Validation Events

Permutation #
Low-frequency module
High-frequency module
Site response
1
URS 1D
URS
URS
2
URS 1D
URS
UCSB
3
URS 1D
URS
SDSU
4
URS 1D
UCSB
URS
5
URS 1D
UCSB
UCSB
6
URS 1D
UCSB
SDSU
7
URS 1D
SDSU
URS
8
URS 1D
SDSU
UCSB
9
URS 1D
SDSU
SDSU
10
URS 3D precomputed
URS
URS
11
URS 3D precomputed
URS
UCSB
12
URS 3D precomputed
URS
SDSU
13
URS 3D precomputed
UCSB
URS
14
URS 3D precomputed
UCSB
UCSB
15
URS 3D precomputed
UCSB
SDSU
16
URS 3D precomputed
SDSU
URS
17
URS 3D precomputed
SDSU
UCSB
18
URS 3D precomputed
SDSU
SDSU
19
UCSB 1D
URS
URS
20
UCSB 1D
URS
UCSB
21
UCSB 1D
URS
SDSU
22
UCSB 1D
UCSB
URS
23
UCSB 1D
UCSB
UCSB
24
UCSB 1D
UCSB
SDSU
25
UCSB 1D
SDSU
URS
26
UCSB 1D
SDSU
UCSB
27
UCSB 1D
SDSU
SDSU
28
SDSU 1D
URS
URS
29
SDSU 1D
URS
UCSB
30
SDSU 1D
URS
SDSU
31
SDSU 1D
UCSB
URS
32
SDSU 1D
UCSB
UCSB
33
SDSU 1D
UCSB
SDSU
34
SDSU 1D
SDSU
URS
35
SDSU 1D
SDSU
UCSB
36
SDSU 1D
SDSU
SDSU
37
SDSU 3D precomputed
URS
URS
38
SDSU 3D precomputed
URS
UCSB
39
SDSU 3D precomputed
URS
SDSU
40
SDSU 3D precomputed
UCSB
URS
41
SDSU 3D precomputed
UCSB
UCSB
42
SDSU 3D precomputed
UCSB
SDSU
43
SDSU 3D precomputed
SDSU
URS
44
SDSU 3D precomputed
SDSU
UCSB
45
SDSU 3D precomputed
SDSU
SDSU

User-Defined Events

Permutation #
Rupture generator
Low-frequency
High-frequency
Site response
1
URS
URS
URS
URS
2
URS
URS
URS
UCSB
3
URS
URS
URS
SDSU
4
URS
URS
UCSB
URS
5
URS
URS
UCSB
UCSB
6
URS
URS
UCSB
SDSU
7
URS
URS
SDSU
URS
8
URS
URS
SDSU
UCSB
9
URS
URS
SDSU
SDSU
10
URS
UCSB
URS
URS
11
URS
UCSB
URS
UCSB
12
URS
UCSB
URS
SDSU
13
URS
UCSB
UCSB
URS
14
URS
UCSB
UCSB
UCSB
15
URS
UCSB
UCSB
SDSU
16
URS
UCSB
SDSU
URS
17
URS
UCSB
SDSU
UCSB
18
URS
UCSB
SDSU
SDSU
19
UCSB
URS
URS
URS
20
UCSB
URS
URS
UCSB
21
UCSB
URS
URS
SDSU
22
UCSB
URS
UCSB
URS
23
UCSB
URS
UCSB
UCSB
24
UCSB
URS
UCSB
SDSU
25
UCSB
URS
SDSU
URS
26
UCSB
URS
SDSU
UCSB
27
UCSB
URS
SDSU
SDSU
28
UCSB
UCSB
URS
URS
29
UCSB
UCSB
URS
UCSB
30
UCSB
UCSB
URS
SDSU
31
UCSB
UCSB
UCSB
URS
32
UCSB
UCSB
UCSB
UCSB
33
UCSB
UCSB
UCSB
SDSU
34
UCSB
UCSB
SDSU
URS
35
UCSB
UCSB
SDSU
UCSB
36
UCSB
UCSB
SDSU
SDSU
37
User-supplied SRF
URS
URS
URS
38
User-supplied SRF
URS
URS
UCSB
39
User-supplied SRF
URS
URS
SDSU
40
User-supplied SRF
URS
UCSB
URS
41
User-supplied SRF
URS
UCSB
UCSB
42
User-supplied SRF
URS
UCSB
SDSU
43
User-supplied SRF
URS
SDSU
URS
44
User-supplied SRF
URS
SDSU
UCSB
45
User-supplied SRF
URS
SDSU
SDSU
46
User-supplied SRF
UCSB
URS
URS
47
User-supplied SRF
UCSB
URS
UCSB
48
User-supplied SRF
UCSB
URS
SDSU
49
User-supplied SRF
UCSB
UCSB
URS
50
User-supplied SRF
UCSB
UCSB
UCSB
51
User-supplied SRF
UCSB
UCSB
SDSU
52
User-supplied SRF
UCSB
SDSU
URS
53
User-supplied SRF
UCSB
SDSU
UCSB
54
User-supplied SRF
UCSB
SDSU
SDSU

Appendix B: Troubleshooting

If you experience trouble building the platform or successfully running test and simulations, try the following solutions.

Build Errors

The instruction for installing Broadband Platform are listed in Section 1: "Installing the Second-Generation Broadband Platform" of the User guide. If after following all steps listed in this section of the user guide the build fails, check if the failure is listed in this section and try the solution to fix the issue you are facing.

64-bit Intel Compiler Issue

If you try to compile with:

 % cd bbp_2g/src
 % make

and you get:

 ... 
 make[2]: Entering directory `.../bbp_2g/src/ucsb/getInFile'
 icc -o getfaultGlobal getfaultGlobal.c
 ld: cannot find -lgcc_s_32
 make[2]: *** [getfaultGlobal] Error 1
 make[2]: Leaving directory `.../bbp_2g/src/ucsb/getInFile'
 make[1]: *** [all] Error 2
 make[1]: Leaving directory `.../bbp_2g/src/ucsb'
 make: *** [all] Error 2

Cause: The Broadband Platform requires Intel 64-bit compilers to build successfully. In this case the 32-bit Intel compilers are being invoked as they are being picked up from the PATH environment variable. The error is due to a missing GNU-GCC 32-bit compatibility pack that has not been installed on your system. This is not required when 64-bit Intel compilers are used.

Solution: Ensure that the path Intel 64-bit compilers are listed in you PATH variable and being picked up by running 'which icc' and 'which ifort'. If you are trying to build the platform on SCEC development servers like Intensity, verify your path variable has:

 PATH=...:/usr/usc/intel/composerxe-2011.4.191/mpirt/bin/intel64:/usr/usc/intel/composerxe-2011.4.191/bin/intel64

32-bit Platform Issue

On trying to build the platform, the build fails with the following message:

$ make all
cd urs; make -f makefile FC=gfortran CC=icc USER_C=1;
make[1]: Entering directory `.../bbp_2g/src/urs'
mkdir -p bin;
cd getpar/src; make -f Makefile CC=icc FC=gfortran USER_C=1;
make[2]: Entering directory `.../bbp_2g/src/urs/getpar/src'
icc -O3  -DENVIRONMENT   -c -o defpar.o defpar.c
icc -O3  -DENVIRONMENT   -c -o getpar.o getpar.c
icc -O3  -DENVIRONMENT   -c -o getarg.o getarg.c
getarg.c(112): warning #303: explicit type is missing ("int" assumed)
      register i;
               ^ 
icc -O3  -DENVIRONMENT   -c -o getlocation.o getlocation.c
getlocation.c(9): warning #147: declaration is incompatible with "__ssize_t={int} getline(char **restrict, size_t={unsigned int} *restrict, FILE *restrict)" (declared at line 671 of "/usr/include/stdio.h")
  static int getline(FILE *fd, char *line)

...
...
...

make[2]: Entering directory `.../bbp_2g/src/sdsu/srcV1.4'
icc -w -Wall -c ray3DJHfor.c
gfortran module_bbtoolbox.f90 module_interface.f90 main_bbtoolbox.f90 coda.f90 composition.f90 convolution.f90 fourier.f90 error.f90 geometry.f90 interpolation.f90 io.f90 random.f90 scattering.f90 source.f90 ray3DJHfor.o -o BBtoolbox.exe
ray3DJHfor.o: In function `raytracing_':
ray3DJHfor.c:(.text+0x8cf): undefined reference to `__libm_sse2_log'
ray3DJHfor.c:(.text+0x96b): undefined reference to `__libm_sse2_asin'
ray3DJHfor.c:(.text+0x9cd): undefined reference to `__libm_sse2_asin'
ray3DJHfor.c:(.text+0xa00): undefined reference to `__libm_sse2_tan'
ray3DJHfor.c:(.text+0xa3d): undefined reference to `__libm_sse2_tan'
ray3DJHfor.c:(.text+0xa52): undefined reference to `__libm_sse2_log'
collect2: ld returned 1 exit status
make[2]: *** [BBtoolbox.exe] Error 1
make[2]: Leaving directory `.../bbp_2g/src/sdsu/srcV1.4'
make[1]: *** [all] Error 2
make[1]: Leaving directory `.../bbp_2g/src/sdsu'
make: *** [all] Error 2

Cause: In this case the user tried to build Broadband on a 32-bit platform. Broadband code was designed to be run on 64-bit platform. Any attempt to build Broadband on a 32-bit platform will result in numerous warnings and the build will fail with error messages similar to ones listed above.

Solution: Try building Broadband on a 64-bit platform will the all the dependencies listed in Section 1.1: "System Dependencies" and Section 1.2: "Software Dependencies"

Unit/Acceptance Test Failures and Errors

Unit and Acceptance tests are provided to verify the Broadband platform and it's supporting modules built by the user are functioning as designed. Under certain circumstances, some of these test might fail. While some of these failure might indicate serious problems that will have to be addressed before the platform can used, it is acceptable to ignore some of the failures. This section lists some failures and their solutions.

ERROR: test_syn1d (test_syn1d.Test_syn1D)

The Unit test for Syn1d module fails with the following message:

 ERROR: test_syn1d (test_syn1d.Test_syn1D)
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File ".../bbp_2g/tests/test_syn1d.py", line 62, in test_syn1d
     synObj.run()
   File ".../bbp_2g/comps/syn1D.py", line 236, in run
     shutil.copy2(a_tmpfile, expected_file)
   File "/usr/lib/python2.6/shutil.py", line 99, in copy2
     copyfile(src, dst)
   File "/usr/lib/python2.6/shutil.py", line 52, in copyfile
     fsrc = open(src, 'rb')
 IOError: [Errno 2] No such file or directory: '.../bbp_2g/tmpdata/3310632/s01.3comp'

Cause: Syn1d module requires the Green's Function file to run and the path to this file might be incorrect in 'bbp_2g_gf/ucsb/GreenBank/Northridge/GreenBank.inf' file.

Solution: This issue can be fixed by correcting the paths listed in the GreenBank.inf files in the Green's Function directory. Here are step by step instructions to fix this issue:

1. Locate your Greens Functions directory: 'bbp_2g_gf'
2. Navigate to bbp_2g_gf/ucsb/GreenBank/Northridge folder.
3. Open Green_Bank.inf files with a text editor.
4. Locate the text "The name of file to store Green Bank"
5. Note the location of the Green Bank file under the line found in step 4, similar to "/home/scec-00/scottcal/bband/bbp_2g_gf/ucsb/GreenBank/Northridge/Green_1d.soil".
6. Replace the path found in Step 5 with just the file name, in the case of Northridge it is "Green_1d.soil"
7. Save and close the Green_Bank.inf.
8. Navigate to bbp_2g_gf/ucsb/GreenBank/LomaPrieta folder and repeat steps 3-7. The name of the Green Bank file for LomaPrieta is "Green_1d.LoPr".
9. Navigate to bbp_2g_gf/ucsb/GreenBank/Landers folder and repeat steps 3-7. The name of the Green Bank file for Landers is "Green1D.Landers"

ERROR: test_gensrf (test_genslip.Test_Genslip)

The Unit test for Genslip module fails with the following message:

======================================================================
ERROR: test_gensrf (test_genslip.Test_Genslip)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../bbp_2g/tests/test_genslip.py", line 51, in test_gensrf
    gen.run()
  File ".../bbp_2g/comps/genslip.py", line 153, in run
    ps.run()
  File ".../bbp_2g/comps/plot_srf.py", line 52, in run
    shutil.copy2("%s/%s.ps.png" % (a_tmpdir, self.r_srffile[0:self.r_srffile.find(".srf")]), "%s/%s.png" % (a_outdir, self.r_srffile[0:self.r_srffile.find(".srf")]))
  File "/usr/lib/python2.6/shutil.py", line 99, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.6/shutil.py", line 52, in copyfile
    fsrc = open(src, 'rb')
IOError: [Errno 2] No such file or directory: '.../bbp_2g/tmpdata/3309008/3309008_test_eq.ps.png'

Cause: The Genslip module calls the Plot_SRF module to the plot the SRF file. Plot_SRF uses C-shell code to locate and use GMT package to plot the SRF file. This shell code might fail on some flavors of Linux with a message like "Newline in variable name" in the Plot_SRF log file.

Solution: This issue can be fixed by editing the plot_SRF.csh file. Here are step by step instructions to fix this issue:

1. Locate shell script file: bbp_2g/src/plot/plot_SRF.csh
2. Open the file in your preferred text editor
3. Locate the line 'set FIND_GMT = $?'
4. Replace the line found in step 3 with 'set FIND_GMT = $status'
5. Locate the section of code that starts with the statement "# FIND MAX. SLIP"
6. Replace the following line in section found in step 5:
    set SMAX = `$XYZCODE calc_xy=$CALC_XY type="${TYPE}" nseg=-1 < $SLIPFILE.srf | \
              gawk -v m=$SMAX '{if($3>m)m=$3;}END{printf "%.0f\n",m;}'`

    with (collapse the statement to a single line by removing '\' at the end of first part of the statement):

    set SMAX = `$XYZCODE calc_xy=$CALC_XY type="${TYPE}" nseg=-1 < $SLIPFILE.srf | gawk -v m=$SMAX '{if($3>m)m=$3;}END{printf "%.0f\n",m;}'`
    
7. Save the file and exit

ERROR: test_ucgen (test_ucrmg.Test_UCrmg)

The Unit test for Ucrmg module fails with the following message:

======================================================================
ERROR: test_ucgen (test_ucrmg.Test_UCrmg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../bbp_2g/tests/test_ucrmg.py", line 59, in test_ucgen
    ucObj.run()
  File ".../bbp_2g/comps/ucrmg.py", line 161, in run
    ps.run()
  File ".../bbp_2g/comps/plot_srf.py", line 52, in run
    shutil.copy2("%s/%s.ps.png" % (a_tmpdir, self.r_srffile[0:self.r_srffile.find(".srf")]), "%s/%s.png" % (a_outdir, self.r_srffile[0:self.r_srffile.find(".srf")]))
  File "/usr/lib/python2.6/shutil.py", line 99, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.6/shutil.py", line 52, in copyfile
    fsrc = open(src, 'rb')
IOError: [Errno 2] No such file or directory: '.../bbp_2g/tmpdata/3309831/test_ucsb.ps.png'

Cause: The Ucrmg module calls the Plot_SRF module to the plot the SRF file. Plot_SRF uses C-shell code to locate and use GMT package to plot the SRF file. This shell code might fail on some flavors of Linux with a message like "Newline in variable name" in the Plot_SRF log file.

Solution: This issue can be fixed by editing the plot_SRF.csh file. The steps to fix this issue are listed under the solution for "ERROR: test_gensrf (test_genslip.Test_Genslip)" above.

FAIL: test_site_amp (test_uc_site.Test_UC_Site)

The unit test for UC-Site module might fail with the following message:

======================================================================
FAIL: test_site_amp (test_uc_site.Test_UC_Site)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../bbp_2g/tests/test_uc_site.py", line 92, in test_site_amp
    self.failIf(cmp_bbp.cmp_bbp(a_ref_file_no_header, test_file_no_header, tolerance=0.035)!=0,errmsg)
AssertionError: Output file .../bbp_2g/tmpdata/3310633/s04.3comp does not match reference file .../bbp_2g/ref_data/ucsb/s04.site.3comp

Cause: The synthetic seismogram generated by the UC-Site module on the user system differs from the reference result included in the Broadband release.

Solution: The stochastic nature of the high frequency simulations is considered, and a constant random seed as input for our tests. This is the same constant random seed used to generate the reference results included in the release. This ensures that the time-series generated with unit tests agree with reference results. However, the high frequency codes are more sensitive to changes in compilers and machines due to the algorithm, so small differences that cause the test to fail are generally acceptable. The user is however encouraged to plot the two time-series and visually inspect them to verify they look similar. We are trying to come up with a better comparison for these tests.

Appendix C: Technical Notes

Running Individual Acceptance Tests

The acceptance test script ./tests/AcceptTest.py attempts to run all of the defined tests when executed. However, sometimes it is useful to run an acceptance test in isolation for troubleshooting purposes:

  • Identify the test you want to run. Eg: user-URS-UCSB-SDSU-SDSU
  • Move to the ./comps directory and execute that test's XML workflow with run_bbp_2G.py. Using the above example:
$ ./run_bbp_2G.py -x ../ref_data/accept_inputs/user-URS-UCSB-SDSU-SDSU.xml

Since you are running the test outside of AcceptTest.py, you will not see a PASS/FAIL message once the simulation completes. However, you can review the contents of the ./outdata/<SIMID> directory to verify that the correct seismograms and response spectra tables were produced.

Manaully Setting Random Number Generator Seed Values

Several of the platform codebases employ random number generators to produce SRFs and seismograms. These random number generators are generally seeded with an initial value (typically an integer) which are hard-coded in the platform. Advanced users may desire to change these seed values.

Changing the seeds for seismogram synthesis varies depending on which codebase you want to use:

  • URS: Edit ./comps/hfsims_cfg.py, Line 43 "self.RAND_NUM"
  • UCSB: Edit ./src/ucsb/Syn1DSCEC/syn1D_LAH.f, Line 183 "iseed", and recompile with "make"
  • SDSU:
    • User event: Edit ./bbp_2g_gf/sdsu/scattering_generic.dat (seed values annotated in file)
    • Validation event: Edit ./bbp_2g_gf/sdsu/<event>/scattering.dat