Difference between revisions of "Containers for CyberShake"
Line 7: | Line 7: | ||
== Setting up a serial container == | == Setting up a serial container == | ||
− | + | (Explains the steps involved in building and running serial code in a container) | |
+ | |||
+ | Image Prework: | ||
+ | Convert Sandbox Director to image file: | ||
+ | sudo singularity build new-sif.sif myUbuntu | ||
+ | imageName sandbox directory | ||
+ | |||
+ | User Work: | ||
+ | Install Dependencies | ||
+ | sudo apt-get update && sudo apt-get install -y \ | ||
+ | build-essential \ | ||
+ | uuid-dev \ | ||
+ | libgpgme-dev \ | ||
+ | squashfs-tools \ | ||
+ | libseccomp-dev \ | ||
+ | wget \ | ||
+ | pkg-config \ | ||
+ | git \ | ||
+ | cryptsetup-bin | ||
+ | |||
+ | Download Go | ||
+ | export VERSION=1.13.5 OS=linux ARCH=amd64 && \ | ||
+ | wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \ | ||
+ | sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ | ||
+ | rm go$VERSION.$OS-$ARCH.tar.gz | ||
+ | |||
+ | Set Up Go | ||
+ | echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ | ||
+ | echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ | ||
+ | source ~/.bashrc | ||
+ | |||
+ | Install Singularity | ||
+ | export VERSION=3.5.2 && # adjust this as necessary \ | ||
+ | wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \ | ||
+ | tar -xzf singularity-${VERSION}.tar.gz && \ | ||
+ | cd singularity | ||
+ | |||
+ | Check if Singularity Works | ||
+ | git clone https://github.com/sylabs/singularity.git && \ | ||
+ | cd singularity && \ | ||
+ | git checkout v3.5.2 | ||
+ | |||
+ | |||
+ | Get Image | ||
+ | singularity pull <source> | ||
+ | |||
+ | Execute Command in from Outside Container | ||
+ | Singularity Commands | ||
+ | singularity exec myPythonContainer.sif cat /etc/lsb-release | ||
+ | imageName command | ||
+ | |||
+ | singularity exec myPythonContainer.sif python3 helloWorld.py | ||
+ | imageName command | ||
+ | |||
+ | Find Size: | ||
+ | singularity cache list | ||
== Containers on Frontera == | == Containers on Frontera == |
Revision as of 20:32, 6 August 2020
This page is to document the steps involved in enabling the CyberShake codebase to run in a container environment.
Contents
Selection of Containers
The available HPC Containers at the time of selection were Singularity, Charlie Cloud, and Shifter. Between the 3 of these container technologies, Singularity was widely adapted and had more open source tools. Because of this wide adaptation the module already existed in the Frontera system. Singularity has built-in support for different MPI libraries from OpenMPI, MPICH, IntelMPI to name a few. Shifter is highly reliant on MPICH ABI, which would require site-specific MPI libraries to be copied to the container at run time.
Setting up a serial container
(Explains the steps involved in building and running serial code in a container)
Image Prework: Convert Sandbox Director to image file: sudo singularity build new-sif.sif myUbuntu imageName sandbox directory
User Work: Install Dependencies sudo apt-get update && sudo apt-get install -y \
build-essential \ uuid-dev \ libgpgme-dev \ squashfs-tools \ libseccomp-dev \ wget \ pkg-config \ git \ cryptsetup-bin
Download Go export VERSION=1.13.5 OS=linux ARCH=amd64 && \
wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \ sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ rm go$VERSION.$OS-$ARCH.tar.gz
Set Up Go echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ source ~/.bashrc
Install Singularity export VERSION=3.5.2 && # adjust this as necessary \
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \ tar -xzf singularity-${VERSION}.tar.gz && \ cd singularity
Check if Singularity Works git clone https://github.com/sylabs/singularity.git && \
cd singularity && \ git checkout v3.5.2
Get Image
singularity pull <source>
Execute Command in from Outside Container Singularity Commands singularity exec myPythonContainer.sif cat /etc/lsb-release imageName command
singularity exec myPythonContainer.sif python3 helloWorld.py imageName command
Find Size: singularity cache list
Containers on Frontera
Serial Containers
Explain how you got serial containers running on Frontera.
MPI Containers
Explain how you got MPI containers running on Frontera.