Difference between revisions of "Moment-carc configuration"

From SCECpedia
Jump to navigationJump to search
Line 117: Line 117:
  
 
For SSH access, we would like support for logging in from endeavour.
 
For SSH access, we would like support for logging in from endeavour.
 +
 +
== Initial database setup ==
 +
 +
We will copy the following data from the database at moment.usc.edu to moment-carc:
 +
 +
{| border="1" cellpadding="3"
 +
! Table !! Data
 +
|-
 +
! AR_Hazard_Curve_Points
 +
|-
 +
! AR_Hazard_Curves
 +
|-
 +
! AR_Hazard_Datasets
 +
|-
 +
! Atten_Rel_Metadata
 +
|-
 +
! Atten_Rels
 +
|-
 +
! CyberShake_Runs
 +
|-
 +
! CyberShake_Site_Regions
 +
|-
 +
! CyberShake_Site_Ruptures
 +
|-
 +
! CyberShake_Site_Types
 +
|-
 +
! CyberShake_Sites
 +
|-
 +
! ERF_IDs
 +
|-
 +
! ERF_Metadata
 +
|-
 +
! ERF_Probability_Models
 +
|-
 +
! Hazard_Curve_Points
 +
|-
 +
! Hazard_Curves
 +
|-
 +
! Hazard_Curves_old
 +
|-
 +
! Hazard_Datasets
 +
|-
 +
! IM_Types
 +
|-
 +
! Mesh_Vsitop_Metadata
 +
|-
 +
! PeakAmplitudes
 +
|-
 +
! Points
 +
|-
 +
! Rup_Var_Seeds
 +
|-
 +
! Rupture_Variation_Probability_Modifier
 +
|-
 +
! Rupture_Variation_Scenario_IDs
 +
|-
 +
! Rupture_Variation_Scenario_Metadata
 +
|-
 +
! Rupture_Variations
 +
|-
 +
! Ruptures
 +
|-
 +
! SGT_Variation_IDs
 +
|-
 +
! SGT_Variation_Metadata
 +
|-
 +
! Studies
 +
|-
 +
! Time_Spans
 +
|-
 +
! Velocity_Model_Metadata
 +
|-
 +
! Velocity_Models
 +
|-
 +
! Vref_eff_Metadata
 +
|}

Revision as of 21:17, 2 May 2024

This page documents the configuration requirements for moment-carc, the CyberShake production database hosted by CARC.

Functionality

moment-carc should support the following:

  • Relational database to host CyberShake input data, run status, run metadata, and store commonly used intensity measures and hazard curves.
  • Different stages of the workflow are able to access this database, including at workflow creation, during some jobs, and data product creation.

Hardware

  • 2 TB of local storage for the database (SSD, if possible).
  • Current system has 64 GB RAM

Software (installed by CARC)

  • Latest production version of MariaDB (11.1.2 as of 10/18/23)

Software (installed by SCEC)

Database configuration

We would like to use the following settings in the mariadb-server.cnf file:

innodb_file_per_table = 1
innodb_buffer_pool_size = 48G (should be 16 GB less than the available system memory)
innodb_buffer_pool_instances = 8
query_cache_type = 0
tmp_table_size = 32M
max_heap_table_size = 32M
innodb_log_file_size = 2G

In terms of users, The functionality we want is read-only access for the user 'cybershk_ro', and read/write access for the user 'cybershk'. In our current setup, entries for these users in the USER_PRIVILEGES table are:

GRANTEE TABLE_CATALOG PRIVILEGE_TYPE IS_GRANTABLE
'cybershk'@'localhost' def INDEX NO
'cybershk'@'localhost' def CREATE TABLESPACE NO
'cybershk'@'localhost' def ALTER NO
'cybershk'@'localhost' def DELETE HISTORY NO
'cybershk'@'localhost' def SHOW DATABASES NO
'cybershk'@'localhost' def SUPER NO
'cybershk'@'localhost' def CREATE TEMPORARY TABLES NO
'cybershk'@'localhost' def LOCK TABLES NO
'cybershk'@'localhost' def SELECT NO
'cybershk'@'localhost' def EXECUTE NO
'cybershk'@'localhost' def INSERT NO
'cybershk'@'localhost' def REPLICATION SLAVE NO
'cybershk'@'localhost' def UPDATE NO
'cybershk'@'localhost' def REPLICATION CLIENT NO
'cybershk'@'localhost' def DELETE NO
'cybershk'@'localhost' def CREATE VIEW NO
'cybershk'@'localhost' def CREATE NO
'cybershk'@'localhost' def SHOW VIEW NO
'cybershk'@'localhost' def DROP NO
'cybershk'@'localhost' def CREATE ROUTINE NO
'cybershk'@'localhost' def RELOAD NO
'cybershk'@'localhost' def ALTER ROUTINE NO
'cybershk'@'localhost' def SHUTDOWN NO
'cybershk'@'localhost' def CREATE USER NO
'cybershk'@'localhost' def PROCESS NO
'cybershk'@'localhost' def EVENT NO
'cybershk'@'localhost' def FILE NO
'cybershk'@'localhost' def TRIGGER NO
'cybershk'@'localhost' def REFERENCES NO
'cybershk'@'%.usc.edu' def USAGE NO
'cybershk'@'%' def SHOW DATABASES NO
'cybershk'@'%' def DELETE HISTORY NO
'cybershk_ro'@'localhost' def USAGE NO
'cybershk_ro'@'%' def USAGE NO

Network configuration

To support access to the MariaDB server, we would like to enable access to the MySQL port (3306 by default) for external users.

We would like the following systems to be whitelisted:

  • USC VPN
  • opensha.usc.edu
  • Summit compute nodes

For SSH access, we would like support for logging in from endeavour.

Initial database setup

We will copy the following data from the database at moment.usc.edu to moment-carc:

Table Data
AR_Hazard_Curve_Points
AR_Hazard_Curves
AR_Hazard_Datasets
Atten_Rel_Metadata
Atten_Rels
CyberShake_Runs
CyberShake_Site_Regions
CyberShake_Site_Ruptures
CyberShake_Site_Types
CyberShake_Sites
ERF_IDs
ERF_Metadata
ERF_Probability_Models
Hazard_Curve_Points
Hazard_Curves
Hazard_Curves_old
Hazard_Datasets
IM_Types
Mesh_Vsitop_Metadata
PeakAmplitudes
Points
Rup_Var_Seeds
Rupture_Variation_Probability_Modifier
Rupture_Variation_Scenario_IDs
Rupture_Variation_Scenario_Metadata
Rupture_Variations
Ruptures
SGT_Variation_IDs
SGT_Variation_Metadata
Studies
Time_Spans
Velocity_Model_Metadata
Velocity_Models
Vref_eff_Metadata