Difference between revisions of "CSEP Gitlab"

From SCECpedia
Jump to navigationJump to search
Line 1: Line 1:
 
The CSEP2 software development and test environment uses the [https://about.gitlab.com/ Gitlab] integration software.
 
The CSEP2 software development and test environment uses the [https://about.gitlab.com/ Gitlab] integration software.
 
== Proposed System Diagram ==
 
<gallery>
 
Image:Csep gitlab roadmap.pdf|left|150px| "Gitlab Server and Runner"
 
</gallery>
 
*[[Media:Csep gitlab roadmap.pptx | Gitlab Server Configuration as ppt file]]
 
*[[Media:Csep gitlab roadmap.pdf | Gitlab Server Configuration as pdf file]]
 
  
 
== CSEP Computers ==
 
== CSEP Computers ==
Line 21: Line 14:
 
*[https://docs.gitlab.com/omnibus/docker/ GitLab Installation using Docker]
 
*[https://docs.gitlab.com/omnibus/docker/ GitLab Installation using Docker]
 
*[https://docs.gitlab.com/runner/install/docker.html GitLab Runner Installation with Docker]
 
*[https://docs.gitlab.com/runner/install/docker.html GitLab Runner Installation with Docker]
 +
 +
== CSEP2 System Overview Diagram ==
 +
<gallery>
 +
Image:Csep gitlab roadmap.pdf|left|150px| "Gitlab Server and Runner"
 +
</gallery>
 +
*[[Media:Csep gitlab roadmap.pptx | Gitlab Server Configuration as ppt file]]
 +
*[[Media:Csep gitlab roadmap.pdf | Gitlab Server Configuration as pdf file]]
 +
 +
== Gitlab AWS Commercial Configuration ==
 +
*[https://docs.gitlab.com/ee/install/aws/ AWS Gitlab Example Deployment]
  
 
== Data Usage Estimates ==
 
== Data Usage Estimates ==

Revision as of 02:03, 7 May 2020

The CSEP2 software development and test environment uses the Gitlab integration software.

CSEP Computers

Gitlab Installation

CSEP2 System Overview Diagram

Gitlab AWS Commercial Configuration

Data Usage Estimates

Estimates for data usages are based on combining the two use cases for the GitLab repository (1) Code storage and (2) Storage of Large data files and experiment results. These estimates are based on data from the current CSEP project, such as current project members, current storage in current and old repositories, and rough estimates of expected data usage for past and future experiments.

Project Info:
CSEP Project Members: 20
CSEP1 Repo Size (in Gb): 1.2
Storage per experiment (in Tb): 1

(1) Code Storage Total data stored: 25 Gb (High-end estimate including models, codes and benchmark data sets)
Data transferred: [50-500] Gb

(2) Data/Catalog Storage Total Storage: 250 Gb
Data transferred: [100 Gb - 1 Tb]

(3) Experiment Results Total Storage: 4 Tb
Data transferred: [100 Gb - 1 Tb]

Overall Estimates: Total stored: ~4.5 Tb
Transferred: [250 Gb - 2.5 Tb]

Gitlab Basic System Requirements

  • Recent Linux Distro (Ubuntu,Centos...)
  • Generate SSH keys
  • Configure SMTP Server
  • 8GB RAM is the recommended minimum memory size for all installations and supports up to 100 users. 16GB RAM supports up to 500 users
  • Databases: PostgreSQL
  • Redis/Sidekiq: stores all user sessions and background task queue processes the background jobs with a multithreaded process
  • Prometheus and it’s exporters
  • Avoid installing GitLab Runner on the same machine where Gitlab is installed.
  • GitLab needs JavaScript enabled in browsers to support features such as Issue Boards.

Related Entries