Difference between revisions of "Containers for CyberShake"

From SCECpedia
Jump to navigationJump to search
Line 65: Line 65:
 
<remote source>:
 
<remote source>:
  
1. Singularity Container Services  
+
1. Singularity Container Services [https://cloud.sylabs.io/library]
 
     <pre>$ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION</pre>
 
     <pre>$ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION</pre>
 
*Note: the path only needs to match the pull card. please see the remote website for example.
 
*Note: the path only needs to match the pull card. please see the remote website for example.
2. Singularity Hub
+
2. Singularity Hub [https://singularity-hub.org/]
 
     <pre>$ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION</pre>
 
     <pre>$ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION</pre>
 
*Note: the path only needs to match the pull card. please see the remote website for example.
 
*Note: the path only needs to match the pull card. please see the remote website for example.
3. Docker Hub
+
3. Docker Hub [https://hub.docker.com/]
 
     <pre>$ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION</pre>
 
     <pre>$ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION</pre>
 
*Note 1: docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build  
 
*Note 1: docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build  
 
*Note 2: the path only needs to match the pull card. please see the remote website for example.
 
*Note 2: the path only needs to match the pull card. please see the remote website for example.
 
-Remote Source Websites:-
 
#Singularity Container Services: [https://cloud.sylabs.io/library]
 
#Singularity Hub: [https://singularity-hub.org/]
 
#Docker Hub: [https://hub.docker.com/]
 
You can get different images and host your own images here.
 
  
 
'''Exec''' - executes an EXTERNAL COMMAND
 
'''Exec''' - executes an EXTERNAL COMMAND

Revision as of 03:27, 7 August 2020

This page is to document the steps involved in enabling the CyberShake codebase to run in a container environment.

Selection of Containers

The available HPC Containers at the time of selection were Singularity, Charlie Cloud, and Shifter. Between the 3 of these container technologies, Singularity was widely adapted and had more open source tools. Because of this wide adaptation the module already existed in the Frontera system. Singularity has built-in support for different MPI libraries from OpenMPI, MPICH, IntelMPI to name a few. Shifter, although light weight, is highly reliant on MPICH ABI. This would require site-specific MPI libraries to be copied to the container at run time.

Installing Singularity

Install Dependencies

sudo apt-get update && sudo apt-get install -y \
build-essential \
uuid-dev \
libgpgme-dev \
squashfs-tools \
libseccomp-dev \
wget \
pkg-config \
git \
cryptsetup-bin


Download Go

export VERSION=1.13.5 OS=linux ARCH=amd64 && \
wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz


Set Up Go

echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc


Install Singularity

export VERSION=3.5.2 && # adjust this as necessary \
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz && \
cd singularity


Check if Singularity Works

git clone https://github.com/sylabs/singularity.git && \
cd singularity && \
git checkout v3.5.2

Setting up a serial container

Get Image singularity pull <source>*

singularity build myPythonContainer.sif library://default/ubuntu:latest
  • <sources> include Singularity Container Library (library), Singularity Hub (shub) and Docker Hub (docker).

Execute Command in from Outside Container singularity exec imageName command

singularity exec myPythonContainer.sif cat /etc/lsb-release

singularity exec image_name command

singularity exec myPythonContainer.sif python3 helloWorld.py

Find Size of Container:

singularity cache list

Basic Singularity Commands

Pull - pulls a container image from a remote source.

$ sudo singularity pull <remote source>

<remote source>:

1. Singularity Container Services [1]

$ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION
  • Note: the path only needs to match the pull card. please see the remote website for example.

2. Singularity Hub [2]

$ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION
  • Note: the path only needs to match the pull card. please see the remote website for example.

3. Docker Hub [3]

$ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION
  • Note 1: docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build
  • Note 2: the path only needs to match the pull card. please see the remote website for example.

Exec - executes an EXTERNAL COMMAND

$ singularity exec IMAGE_NAME.sif EXTERNAL_COMMAND


Shell - shells into an existing container

singularity shell IMAGE_NAME.sif
  • Note: Your home directory is mounted by default


Run - runs an image. Run is based on the Run Script parameters that were placed into the container when the image was built based the recipe

$ singularity run IMAGE_NAME.sif


Build (BIG TO DO: Very important... a lot of details and opinions)

$ singularity build IMAGE_NAME.sif <source>

<source> include -Another Image either docker or singularity -Singularity definition file (use to be known as a recipe file), usually denoted with name.def


Note: You can shell into a docker UI - explore different containers without pulling or building

$ singularity shell docker://ubuntu

Creating Definition Files: (To Do) Set up complex workflows with Recipe File: Alternatively- Sandbox Directory Prototype Final Container: sudo singularity build --sandbox ubuntu_s docker://ubuntu

Containers on Frontera

Serial Containers

0. Prepare a Program if you Like Hello World for python.

1. Get a Singularity Image on Frontera (*Note: If you want to wrong a particular program, you must have the dependencies installed in the container)

  1. Copy the container from your computer to frontera:
    1. scp
    2. Pull from the Computation Node
idev -N 1; singularity pull <source>
  • Note: This command works in a sbatch file.

2. Interface with Computation Node

  1. idev session
idev
ibrun singularity exec IMAGE_NAME.sif python3 helloworld.py
  1. sbatch (recommended)

TO DO: 

MPI Containers

Explain how you got MPI containers running on Frontera.