CARC BBP Setup
SCEC has migrated our BBP server from University Park server room to USC's new research Computing group CARC.
Contents
Server Name
- broadband_carc.usc.edu
- [CARC] Getting CARC account
- login: ssh broadband_carc.usc.edu
Disk Quota
- Users have 10GB quota on /home1/username
- Users have 1TB+ quota on /project/scec_608/<username>
We will install broadband software on /project, and configure it to write results to this storage system.
CARC File systems
CARC users have a home directory on the /home/username filesystem. This directory has a quota of about 10GB. To do a Broadband installation, we will need to make use of SCEC storage on the CARC /project/scec_608 filesystem.
To create a broadband installation in on the broadband_carc server, we recommend using the following filesystems:
- /home1/username
This is the users home login account. Edit the .bashrc in this account to setup the required modules and environment variables.
- /project/scec_608/username/scecbbp
This should be the installation directory for bbp. All bbp related software and data (except the .bashrc file) can be found here. Typically, users will log in and move into this directory to run the broadband platform. This is the directory that where the user clones the bbp repo and runs the bbp installation script.
When the bbp platform installation process runs, this is the
- /project/scec_608/username/scecbbp/bbp_gf
- /project/scec_608/username/scecbbp/bbp_val
- /project/scec_608/username/scecbbp/bbp_data
Installing Broadband on CARC
The users account must be configured using the CARC module system. The module systems sets up the users computing environment, including the compilers and libraries. The module system provides a lot of flexibility in building the BBP software. We have tested the broadband platform on broadband_carc using a specific module configuration using recent gnu compilers, that we describe in these installation notes.
Anaconda Installation
To support users running broadband platform on computers where we don't have root, we recommend use of Anaconda python, so the software is managed (adding packages) without root access. Retreived command line installer using curl on broadbnad
Then, it will ask you where to install anaconda. We recommend you install it on /project/scec_608/username directory. Don't accept the default installation directory, rather, change it to poing to the project file system. eg: /project/scec_608/maechlin/anaconda3
.bashrc
Add this entries, modified with your username, to your .bashrc
(base) [maechlin@broadband ~]$ more .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
# CARC Module Load system
module purge
module load gcc/8.3.0
# Setup BBP Environment
export BBP_DIR=/project/scec_608/maechlin/scecbbp/bbp
export BBP_GF_DIR=/project/scec_608/maechlin/scecbbp/bbp_gf
export BBP_VAL_DIR=/project/scec_608/maechlin/scecbbp/bbp_val
export BBP_DATA_DIR=/project/scec_608/maechlin/scecbbp/bbp_data
alias bbp="cd /project/scec_608/maechlin/scecbbp"
export PYTHONPATH=/project/scec_608/maechlin/scecbbp/bbp/bbp/comps:$PYTHONPATH
export PATH=/project/scec_608/maechlin/scecbbp/bbp/bbp/comps:/project/scec_608/maechlin/scecbbp/bbp/bbp/utils/batch:$PATH
ulimit -s unlimited
export PATH
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/project/maechlin_162/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/project/maechlin_162/anaconda3/etc/profile.d/conda.sh" ]; then
. "/project/maechlin_162/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/project/maechlin_162/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
Login and retrieve from github
Be sure to retrieve the dev branch which has been ported to the newest version of the CARC systems.
$git clone --single-branch --branch dev https://github.com/sceccode/bbp.git
