BBP Development

This page lists current Broadband Platform development activities:

Development Work

With Broadband version v11.2.2., all unit tests and acceptance tests pass on broadband.usc.edu. However, we get test failures on intensity.usc.edu, and we are working to determine the reasons. Here are the current Unit test results on intensity.

Unit Tests on Intensity

bash-3.2$ ./UnitTests.py > out.txt &                                            
[1] 16243                                                                       
bash-3.2$ test_gensrf (test_genslip.Test_Genslip) ... ok                        
test_bbp_wid (test_jbsim.Test_Jbsim) ... ok                                     
test_bbp_wid (test_hfsims.Test_hfsims) ... ok                                   
test_bbp_wid (test_wcc_siteamp.Test_Wcc_siteamp) ... ok                         
test_bbp_wid (test_match.Test_Match) ... ok                                     
test_respect (test_respect.Test_Respect) ... ok                                 
test_gof (test_urs_gof.Test_URS_Gof) ... ok                                     
test_bbp_wid (test_uc_makefaultg.Test_UC_Makefaultg) ... ok                     
test_ucgen (test_ucrmg.Test_UCrmg) ... ok                                       
test_syn1d (test_syn1d.Test_syn1D) ... FAIL                                     
test_site_amp (test_uc_site.Test_UC_Site) ... FAIL                              
test_respect (test_respect.Test_Respect) ... ok                                 
test_gof (test_urs_gof.Test_URS_Gof) ... ok                                     
test_bbtoolbox (test_bbtoolbox.Test_BBToolbox) ... ok                           
test_amp_fac (test_amp_fac.Test_Amp_Fac) ... ok                                 
test_respect (test_respect.Test_Respect) ... ok                                 
test_gof (test_urs_gof.Test_URS_Gof) ... ok

======================================================================
FAIL: test_syn1d (test_syn1d.Test_syn1D)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/scec-00/maechlin/bbp/i_host/bbp_2g/tests/test_syn1d.py", line 71,in test_syn1d
    self.failIf(cmp_bbp.cmp_bbp(a_ref_file,a_newfile)!=0,errmsg)
AssertionError: Output file does not match reference file: /home/scec-00/maechlin/bbp/i_host/bbp_2g/tmpdata/5222974/s01.3comp

======================================================================
FAIL: test_site_amp (test_uc_site.Test_UC_Site)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/scec-00/maechlin/bbp/i_host/bbp_2g/tests/test_uc_site.py", line 92, in test_site_amp
    self.failIf(cmp_bbp.cmp_bbp(a_ref_file_no_header, test_file_no_header, tolerance=0.035)!=0,errmsg)
AssertionError: Output file /home/scec-00/maechlin/bbp/i_host/bbp_2g/tmpdata/5223105/s04.3comp does not match reference file /home/scec-00/maechlin/bbp/i_host/bbp_2g/ref_data/ucsb/s04.site.3comp

----------------------------------------------------------------------
Ran 17 tests in 2125.133s

FAILED (failures=2)

Top Development Priorities

1. Create wiki entry describing setting random SEED for three High Freq Modules
2. Create "optional" patch distributions separate from official bbp 11.2.2 release
   1. Define acceptance test patch
   2. Define option parsing patch
3. Define wiki entry on how to run one acceptance test to run one at a time
4. Define Graves study (including list of output files) and create wiki entry
5. Define Kottke study (including list of output files) and create wiki entry
6. Setup Graves study using batch tools
7. Setup Kottke study using batch tools
8. Create a wiki for batch tools for broadband platform

Development Activities

1. Modify broadband distribution to use gnu build tools
2. Create minimized software installation distribution for clusters
   1. Remove dependencies on gmt,imagemagic,...
3. Develop prototype distributed broadband queue manager
4. Distribution tracking
   1. Check for updates capability
   2. Download counter for software
   3. Optional registration announcement
5. Simplify distribution to use one scattering.dat file in SDSU High freq
6. Define data query interface for bbp results

Future Developments

1. 3D wave propagation capabilities
2. Concept for integrating 3D wave prop with BBP
3. Change name of main executable to remove 2g label
4. Define query interface for data products

Port bbp to mac

Steps installing bbp on mac

• install xcode to get gcc - must be registered mac developer
• install gfortran from mac hpc web site
• install gmt from gmt - put it in /opt/local
• include install of netcdf - put in /opt/local
• install macport
• install py27-matplotlib
• install py27-gtk
• reset python configure to use this new install
• edits to current bbp were required including
  • remove all of -D_LARGEFILE (and other similar) params
  • change g77 to gfortran
  • add declaration of DGAMM in URS code to stop return error problem
  • comment out headers like <features.h> and <sys/procs.h> or something like that

Problems:

• I had problems getting matplotlib to be recognized. I could import from the command line python interpreter, but scripts couldn't find it. Eventually it worked
• ifort compilers not install. Currently only testing with urs code

= Logs

zhs158-1:~ maechlin$ sudo port install py27-matplotlib Warning: xcodebuild exists but failed to execute Warning: Xcode appears to be installed but xcodebuild is unusable; some ports will likely fail to build. Warning: You may need to run `sudo xcode-select -switch /Applications/Xcode.app` ---> Computing dependencies for py27-matplotlib ---> Dependencies to be installed: freetype bzip2 zlib libpng py27-configobj python27 db46 gettext expat libiconv gperf ncurses libedit openssl python_select sqlite3 xz py27-dateutil py27-distribute py27-distribute py27-tz py27-tz py27-numpy fftw-3 py27-nose nosetests_select py27-pyobjc-cocoa py27-pyobjc libxml2 libxml2 py27-py2app py27-bdist_mpkg py27-macholib py27-modulegraph py27-altgraph py27-tkinter tk Xft2 fontconfig pkgconfig glib2 libffi perl5 perl5.12 gdbm perl5 perl5 xrender xorg-libX11 xorg-bigreqsproto xorg-inputproto xorg-kbproto xorg-libXau xorg-xproto xorg-libXdmcp xorg-libxcb xorg-libpthread-stubs xorg-xcb-proto xorg-xcmiscproto xorg-xextproto xorg-xf86bigfontproto xorg-xtrans xorg-renderproto tcl xorg-libXScrnSaver xorg-libXext autoconf help2man p5.12-locale-gettext m4 autoconf automake libtool xorg-util-macros xorg-scrnsaverproto ---> Fetching archive for bzip2 ---> Attempting to fetch bzip2-1.0.6_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/bzip2 ---> Attempting to fetch bzip2-1.0.6_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/bzip2 ---> Installing bzip2 @1.0.6_0 ---> Activating bzip2 @1.0.6_0 ---> Cleaning bzip2 ---> Fetching archive for zlib ---> Attempting to fetch zlib-1.2.6_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/zlib ---> Attempting to fetch zlib-1.2.6_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/zlib ---> Installing zlib @1.2.6_0 ---> Activating zlib @1.2.6_0 ---> Cleaning zlib ---> Fetching archive for freetype ---> Attempting to fetch freetype-2.4.9_1.darwin_11.x86_64.tbz2 from http://packages.macports.org/freetype ---> Attempting to fetch freetype-2.4.9_1.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/freetype ---> Installing freetype @2.4.9_1 ---> Activating freetype @2.4.9_1 ---> Cleaning freetype ---> Fetching archive for libpng ---> Attempting to fetch libpng-1.4.11_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/libpng ---> Attempting to fetch libpng-1.4.11_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/libpng ---> Installing libpng @1.4.11_0 ---> Activating libpng @1.4.11_0 ---> Cleaning libpng ---> Fetching archive for db46 ---> Attempting to fetch db46-4.6.21_6.darwin_11.x86_64.tbz2 from http://packages.macports.org/db46 ---> Attempting to fetch db46-4.6.21_6.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/db46 ---> Installing db46 @4.6.21_6 ---> Activating db46 @4.6.21_6 ---> Cleaning db46 ---> Fetching archive for expat ---> Attempting to fetch expat-2.1.0_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/expat ---> Attempting to fetch expat-2.1.0_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/expat ---> Installing expat @2.1.0_0 ---> Activating expat @2.1.0_0 ---> Cleaning expat ---> Fetching archive for gperf ---> Attempting to fetch gperf-3.0.4_2.darwin_11.x86_64.tbz2 from http://packages.macports.org/gperf ---> Attempting to fetch gperf-3.0.4_2.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/gperf ---> Installing gperf @3.0.4_2 ---> Activating gperf @3.0.4_2 ---> Cleaning gperf ---> Fetching archive for libiconv ---> Attempting to fetch libiconv-1.14_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/libiconv ---> Attempting to fetch libiconv-1.14_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/libiconv ---> Installing libiconv @1.14_0 ---> Activating libiconv @1.14_0 ---> Cleaning libiconv ---> Fetching archive for ncurses ---> Attempting to fetch ncurses-5.9_1.darwin_11.x86_64.tbz2 from http://packages.macports.org/ncurses ---> Attempting to fetch ncurses-5.9_1.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/ncurses ---> Installing ncurses @5.9_1 ---> Activating ncurses @5.9_1 ---> Cleaning ncurses ---> Fetching archive for gettext ---> Attempting to fetch gettext-0.18.1.1_2.darwin_11.x86_64.tbz2 from http://packages.macports.org/gettext ---> Attempting to fetch gettext-0.18.1.1_2.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/gettext ---> Installing gettext @0.18.1.1_2 ---> Activating gettext @0.18.1.1_2 ---> Cleaning gettext ---> Fetching archive for libedit ---> Attempting to fetch libedit-20110802-3.0_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/libedit ---> Attempting to fetch libedit-20110802-3.0_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/libedit ---> Installing libedit @20110802-3.0_0 ---> Activating libedit @20110802-3.0_0 ---> Cleaning libedit ---> Fetching archive for openssl ---> Attempting to fetch openssl-1.0.1a_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/openssl ---> Attempting to fetch openssl-1.0.1a_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/openssl ---> Installing openssl @1.0.1a_0 ---> Activating openssl @1.0.1a_0 ---> Cleaning openssl ---> Fetching archive for python_select ---> Attempting to fetch python_select-0.3_1.darwin_11.noarch.tbz2 from http://packages.macports.org/python_select ---> Attempting to fetch python_select-0.3_1.darwin_11.noarch.tbz2.rmd160 from http://packages.macports.org/python_select ---> Installing python_select @0.3_1 ---> Activating python_select @0.3_1 ---> Cleaning python_select ---> Fetching archive for sqlite3 ---> Attempting to fetch sqlite3-3.7.11_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/sqlite3 ---> Attempting to fetch sqlite3-3.7.11_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/sqlite3 ---> Installing sqlite3 @3.7.11_0 ---> Activating sqlite3 @3.7.11_0 ---> Cleaning sqlite3 ---> Fetching archive for xz ---> Attempting to fetch xz-5.0.3_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/xz ---> Attempting to fetch xz-5.0.3_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/xz ---> Installing xz @5.0.3_0 ---> Activating xz @5.0.3_0 ---> Cleaning xz ---> Fetching archive for python27 ---> Attempting to fetch python27-2.7.3_0.darwin_11.x86_64.tbz2 from http://packages.macports.org/python27 ---> Attempting to fetch python27-2.7.3_0.darwin_11.x86_64.tbz2.rmd160 from http://packages.macports.org/python27 ---> Installing python27 @2.7.3_0 ---> Activating python27 @2.7.3_0

To make python 2.7 the default (i.e. the version you get when you run 'python'), please run:

sudo port select --set python python27

---> Installing py27-matplotlib @1.1.0_3+tkinter ---> Activating py27-matplotlib @1.1.0_3+tkinter

The default backend is the interactive Mac OS X backend. Different backends can be specified using the ~/.matplotlib/matplotlibrc file. More details regarding backends can be found in the matplotlib FAQ:

http://matplotlib.sourceforge.net/faq/installing_faq.html#what-is-a-backend

---> Cleaning py27-matplotlib zhs158-1:~ maechlin$