Difference between revisions of "Containers for CyberShake"
Line 168: | Line 168: | ||
TACC - Frontera | TACC - Frontera | ||
− | #TACC Containers [https://github.com/TACC/tacc-containers/tree/master/containers] ( | + | #TACC Containers [https://github.com/TACC/tacc-containers/tree/master/containers] (More geared for people who are familiar with Docker Containers) |
Revision as of 04:24, 7 August 2020
This page is to document the steps involved in enabling the CyberShake codebase to run in a container environment.
Contents
Selection of Containers
The available HPC Containers at the time of selection were Singularity, Charlie Cloud, and Shifter. Between the 3 of these container technologies, Singularity was widely adapted and had more open source tools. Because of this wide adaptation the module already existed in the Frontera system. Singularity has built-in support for different MPI libraries from OpenMPI, MPICH, and IntelMPI to name a few. Shifter, although light weight, is highly reliant on MPICH ABI. This would require site-specific MPI libraries to be copied to the container at runtime.
Installing Singularity
Install Dependencies
sudo apt-get update && sudo apt-get install -y \ build-essential \ uuid-dev \ libgpgme-dev \ squashfs-tools \ libseccomp-dev \ wget \ pkg-config \ git \ cryptsetup-bin
Download Go
export VERSION=1.13.5 OS=linux ARCH=amd64 && \ wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \ sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ rm go$VERSION.$OS-$ARCH.tar.gz
Set Up Go
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ source ~/.bashrc
Install Singularity
export VERSION=3.5.2 && # adjust this as necessary \ wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \ tar -xzf singularity-${VERSION}.tar.gz && \ cd singularity
Check if Singularity Works
git clone https://github.com/sylabs/singularity.git && \ cd singularity && \ git checkout v3.5.2
Setting up a serial container (on your computer)
Get Image singularity pull <source>*
$ singularity build myPythonContainer.sif library://default/ubuntu:latest
- <sources> include Singularity Container Library (library), Singularity Hub (shub) and Docker Hub (docker).
Execute Command in from Outside Container singularity exec imageName command
$ singularity exec myPythonContainer.sif cat /etc/lsb-release
singularity exec image_name command
$ singularity exec myPythonContainer.sif python3 helloWorld.py
Find Size of Container:
$ singularity cache list
- Note: Singularity cannot run on the Login Node
Basic Singularity Commands
Pull - pulls a container image from a remote source.
$ sudo singularity pull <remote source>
<remote source>:
1. Singularity Container Services [1]
$ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION
- Note: the path only needs to match the pull card. please see the remote website for example.
2. Singularity Hub [2]
$ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION
- Note: the path only needs to match the pull card. please see the remote website for example.
3. Docker Hub [3]
$ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION
- Note 1: docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build
- Note 2: the path only needs to match the pull card. please see the remote website for example.
Exec - executes an EXTERNAL COMMAND
$ singularity exec IMAGE_NAME.sif EXTERNAL_COMMAND
Shell - shells into an existing container
$ singularity shell IMAGE_NAME.sif
- Note: Your home directory is mounted by default
Run - runs an image. Run is based on the Run Script parameters that were placed into the container when the image was built based the recipe
$ singularity run IMAGE_NAME.sif
Build (BIG TO DO: Very important... a lot of details and opinions)
$ singularity build IMAGE_NAME.sif <source>
<source> include -Another Image either docker or singularity -Singularity definition file (use to be known as a recipe file), usually denoted with name.def
Note:
You can shell into a docker UI - explore different containers without pulling or building
$ singularity shell docker://ubuntu
Creating Definition Files: (To Do) Set up complex workflows with Recipe File: Alternatively- Sandbox Directory Prototype Final Container: sudo singularity build --sandbox ubuntu_s docker://ubuntu
Building or Using Pre Made Containers
Frontera
Summit
Generic
Containers on Frontera
Serial Containers
1. Prepare
- Install Module:
$ module load tacc-singularity
- Make HelloWorld.py
print("Hello World!")
2. Get a Singularity Image on Frontera (*Note: If you want to wrong a particular program, you must have the dependencies installed in the container)
- Copy the container from your computer to frontera:
- scp
- Pull from the Computation Node
idev -N 1; singularity pull <source>
- Note: This command works in a sbatch file.
3. Interface with Computation Node
- idev session
idev ibrun singularity exec IMAGE_NAME.sif python3 helloworld.py
- sbatch (recommended)
TO DO:
MPI Containers
1. Make MPI Program - (Ex: named sum_sqrt.c)
2. Compile Program
$ mpicc -o sum_sqrt sum_sqrt.c
3. Build or Pull a Singularity Image with the same MPI library installed inside the container [4]
mvapich preinstalled in this container
$ idev -N 1 $ singularity pull <path>
4. Execute your command
$ ibrun singularity exec ./sum_sqrt 100000
Resources
- Singularity Guide [5]
- Singularity Repository [6]
- Singularity Container Library [7]
- Singularity Hub [8]
- Docker Hub [9]
TACC - Frontera
- TACC Containers [10] (More geared for people who are familiar with Docker Containers)