Difference between revisions of "User:Llocsin"

From SCECpedia
Jump to navigationJump to search
 
(5 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
====Steps:====
 
====Steps:====
 
#Download svn2git tool: https://github.com/nirvdrum/svn2git
 
#Download svn2git tool: https://github.com/nirvdrum/svn2git
−
##Add Commands
+
<pre>git clone <remote></pre>
 
#Pull desired SVN repository
 
#Pull desired SVN repository
−
##//Add Commands
+
##<pre>svn checkout <remote></pre>
 
#Create Text Document of Authors to link to Git Accounts
 
#Create Text Document of Authors to link to Git Accounts
 
##create authors.txt
 
##create authors.txt
Line 14: Line 14:
 
##$ svn2git http://source.usc.edu/svn/mesh_partitioner/ --trunk trunk --tags tags --nobranches --authors ~/authors.txt (Note: --trunk trunk (the trunk argument is the name of the trunk repository-tags tags (tags is the name of your repository of tags))
 
##$ svn2git http://source.usc.edu/svn/mesh_partitioner/ --trunk trunk --tags tags --nobranches --authors ~/authors.txt (Note: --trunk trunk (the trunk argument is the name of the trunk repository-tags tags (tags is the name of your repository of tags))
 
#Push to the desired git remote. (In this example we use github)
 
#Push to the desired git remote. (In this example we use github)
−
##Add Remote##git remote add <remote-name> <remote-url>##git remote origin git@github.com:SCECcode/CyberShake.git
+
##Add Remote
−
Commit
+
###git remote add <remote-name> <remote-url>##git remote origin git@github.com:SCECcode/CyberShake.git
−
git add .
+
##Commit
−
git commit -m “Initial Commit of Converted SVN to Git Repository Code”
+
###git add .
−
Use tags command
+
###git commit -m “Initial Commit of Converted SVN to Git Repository Code”
−
git push  --tags
+
##Use tags command
 +
###git push  --tags
  
 +
Rebase History
 +
If the hosted Subversion repository’s history possesses commits not yet in the local Git repository, the dcommit operation will be rejected until the commits are acquired with this command:
  
−
Resources:  
+
<pre>git svn rebase</pre>
−
svn2git tool (from the guide): https://github.com/nirvdrum/svn2git
+
 
−
Official Git Documentation:  
+
 
−
Guide: https://viastudio.com/migrate-svn-git/: https://github.github.com/training-kit/downloads/subversion-migration/
+
===Resources:===
−
Github Importer Tool (alternate): https://docs.github.com/en/github/importing-your-projects-to-github/about-github-importer
+
#svn2git tool (from the guide): https://github.com/nirvdrum/svn2git
 +
#Official Git Documentation:  
 +
#Guide: https://viastudio.com/migrate-svn-git/: https://github.github.com/training-kit/downloads/subversion-migration/
 +
#Github Importer Tool (alternate): https://docs.github.com/en/github/importing-your-projects-to-github/about-github-importer
 +
 
 +
 
 +
Install Singularity
 +
To Do:
 +
 
 +
Basic Singularity Commands
 +
Pull - pulls a container image from a remote source
 +
$ sudo singularity pull <remote source>
 +
<remote source>
 +
Singularity Container Services
 +
    $ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION
 +
Singularity Hub
 +
$ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION  (Note: the path only needs to match the pull card. please see the remote website for example.)
 +
Docker Hub
 +
$ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION
 +
(Note docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build)
 +
 
 +
Exec - executes an EXTERNAL COMMAND
 +
$ singularity exec IMAGE_NAME.sif EXTERNAL_COMMAND
 +
 
 +
Shell - shells into an existing container
 +
singularity shell IMAGE_NAME.sif
 +
*Note: Your home directory is mounted by default
 +
 
 +
Run - runs an image. Run is based on the Run Script parameters that were placed into the container when the image was built based the recipe
 +
$ singularity run IMAGE_NAME.sif
 +
 
 +
Build (BIG TO DO: Very important)
 +
$ singularity build IMAGE_NAME.sif <source>
 +
Sources include
 +
Another Image either docker or singularity
 +
Singularity definition file (use to be known as a recipe file), usually denoted with name.def
 +
 
 +
Note:
 +
You can shell into a docker UI - explore different containers without pulling or building
 +
$ singularity shell docker://ubuntu
 +
 
 +
Creating Definition Files:
 +
Workflow:
 +
Set up complex workflows with Recipe File:
 +
Alternatively-
 +
Sandbox Directory Prototype Final Container: sudo singularity build --sandbox ubuntu_s docker://ubuntu
 +
 
 +
WRITE RECIPE:
 +
File Name: Singularity.def
 +
 
 +
Bootstrap: docker
 +
From: ubuntu:18.04
 +
%post
 +
    apt-get -y update
 +
    apt-get -y install python3
 +
%files
 +
    helloWorld.py /
 +
 
 +
%runscript
 +
    python3 /hello_world.py
 +
 
 +
$ sudo singularity build CONTAINER_NAME.sif Singularity.def
 +
$ sudo singularity run CONTAINER_NAME.sif
 +
Result:
 +
Hello World
 +
 
 +
Resources:
 +
Singularity Guide:
 +
Singularity Help:
 +
 
 +
Pre-Made Containers:
 +
 
 +
TACC Resources:
 +
 
 +
MVAPICH Dependances
 +
MPICH Dependencies:

Latest revision as of 01:22, 14 August 2020

SVN to Git Plan

Convert SVN repository to a Git Repository.

Steps:

  1. Download svn2git tool: https://github.com/nirvdrum/svn2git
git clone <remote>
  1. Pull desired SVN repository 
    1. svn checkout <remote>
  2. Create Text Document of Authors to link to Git Accounts
    1. create authors.txt
    2. add in all authors with the following format:
      1. jcoglan = James Coglan <jcoglan@never-you-mind.com>
      2. stnick = Santa Claus <nicholas@lapland.com> (Recommended to link everyone’s account but if you can’t, it’s not required to link everyone’s account, but you must provide an email, if only a made up one.)
  3. Convert the repositories with the git commands:
    1. $ svn2git http://source.usc.edu/svn/mesh_partitioner/ --trunk trunk --tags tags --nobranches --authors ~/authors.txt (Note: --trunk trunk (the trunk argument is the name of the trunk repository-tags tags (tags is the name of your repository of tags))
  4. Push to the desired git remote. (In this example we use github)
    1. Add Remote
      1. git remote add <remote-name> <remote-url>##git remote origin git@github.com:SCECcode/CyberShake.git
    2. Commit
      1. git add .
      2. git commit -m “Initial Commit of Converted SVN to Git Repository Code”
    3. Use tags command
      1. git push --tags

Rebase History If the hosted Subversion repository’s history possesses commits not yet in the local Git repository, the dcommit operation will be rejected until the commits are acquired with this command: 

git svn rebase


Resources:

  1. svn2git tool (from the guide): https://github.com/nirvdrum/svn2git
  2. Official Git Documentation:
  3. Guide: https://viastudio.com/migrate-svn-git/: https://github.github.com/training-kit/downloads/subversion-migration/
  4. Github Importer Tool (alternate): https://docs.github.com/en/github/importing-your-projects-to-github/about-github-importer


Install Singularity To Do:

Basic Singularity Commands Pull - pulls a container image from a remote source $ sudo singularity pull <remote source> <remote source> Singularity Container Services 

   $ sudo singularity pull --name CONTAINER_NAME.sif library://USER/PULL_PATH:VERSION

Singularity Hub $ sudo singularity pull --name CONTAINER_NAME.sif shub://USER/PULL_PATH:VERSION (Note: the path only needs to match the pull card. please see the remote website for example.) Docker Hub $ sudo singularity build CONTAINER_NAME.sif docker://USER/PULL_PATH:VERSION (Note docker images have layers and it needs to be merged into 1 singularity image. For that to happen you MUST use: build)

Exec - executes an EXTERNAL COMMAND $ singularity exec IMAGE_NAME.sif EXTERNAL_COMMAND

Shell - shells into an existing container singularity shell IMAGE_NAME.sif

  • Note: Your home directory is mounted by default

Run - runs an image. Run is based on the Run Script parameters that were placed into the container when the image was built based the recipe $ singularity run IMAGE_NAME.sif

Build (BIG TO DO: Very important) $ singularity build IMAGE_NAME.sif <source> Sources include Another Image either docker or singularity Singularity definition file (use to be known as a recipe file), usually denoted with name.def

Note: You can shell into a docker UI - explore different containers without pulling or building $ singularity shell docker://ubuntu

Creating Definition Files: Workflow: Set up complex workflows with Recipe File: Alternatively- Sandbox Directory Prototype Final Container: sudo singularity build --sandbox ubuntu_s docker://ubuntu

WRITE RECIPE: File Name: Singularity.def

Bootstrap: docker From: ubuntu:18.04 %post 

   apt-get -y update
   apt-get -y install python3

%files 

   helloWorld.py /

%runscript 

   python3 /hello_world.py

$ sudo singularity build CONTAINER_NAME.sif Singularity.def $ sudo singularity run CONTAINER_NAME.sif Result: Hello World

Resources: Singularity Guide: Singularity Help:

Pre-Made Containers:

TACC Resources:

MVAPICH Dependances MPICH Dependencies:

Retrieved from "https://strike.scec.org/scecwiki/index.php?title=User:Llocsin&oldid=24924"