Difference between revisions of "CyberShake Computational Estimates"

From SCECpedia
Jump to navigationJump to search
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.
 
We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.
  
The UCERF 3 estimates assume that the number of ruptures increases from 15000 to 200,000, but the number of rupture variations per rupture remains the same.
+
The UCERF 3 estimates assume that the number of ruptures increases from 15,000 to 350,000, but the number of rupture variations per rupture on average remains the same.
  
== Future SCEC data needs ==
+
The 0.5 Hz numbers are taken from Study 14.2.
  
These are estimates of CyberShake storage required on SCEC computers for upcoming runs.
+
The node-hours are estimates based on the XE6 and XK7 nodes on Blue Waters.
  
July-August 2013:  6.1 TB to archive (0.5 Hz, CVM-SI, 2 SGT versions, 286 sites, UCERF 2)
+
== 1.0 Hz, 3 component ==
Fall 2013:  12 TB (1 Hz, 286 sites, 2 combinations, UCERF 2)
 
Spring 2014:  45 TB (0.5 Hz, 286 sites, 1 combination, UCERF 3)
 
  
== 0.5 Hz, UCERF 3 (per site) ==
+
SGTs: At 0.5 Hz, it requires 38 GPU node-hrs per component.
 +
  (38 GPU node-hrs per component) x (3 components) x (8 times the gridpoints) x (2 times the timesteps) x (20% more efficient due to more work per GPU) = 1460 node-hrs per site.
  
Number of rupture variations:  5.5 million
+
23% of node hours to calculate SGTs.
  
=== Deterministic ===
+
PP: At 0.5 Hz, it requires 41 CPU node-hrs per component.
 +
  (41 CPU node-hrs per components) x (3 components) x (25 times the rupture points) x (2 times the timesteps) x (20% more efficient due to rupture generator improvements) = 4920 node-hrs per site.
  
Number of jobs:  5.6 million
+
77% of node hours to calculate PP.
  
Storage:  40 GB SGTs, 125 GB seismograms
+
Each site requires about 550,000 rupture variations (410,000 x 4/3 for rupture variations v3.3)
  
SUs: 12k SGTs + 26k post-processing = 38k
+
'''6380''' node-hours per 3-component site (181k core-hours)
  
=== Broadband ===
+
'''1.82M''' node-hours for standard 3-component So Cal 286-site map (51.7M core-hours)
  
Number of jobs:  16.6 million
+
'''5.73M''' node-hours for increased density 3-component So Cal 898-site map (162M core-hours)
  
Storage:  40 GB SGTs, 500 GB seismograms
+
'''8.93M''' node-hours for statewide adaptive 3-component California 1400-site map (253M core-hours)
  
SUs: 12k SGTs + 52k post-processing = 64k
+
== 2.0 Hz ==
  
 +
SGTs: At 1.0 Hz, it requires 485 GPU node-hrs per component.
 +
  (485 GPU node-hrs per component) x (3 components) x (8 times the gridpoints) x (2 times the timesteps) = 23.3k node-hrs per site.
  
== 1 Hz, UCERF 3 (per site) ==
+
PP: At 1.0 Hz, it requires 1640 CPU node-hrs per component.
 +
  (1640 CPU node-hrs per components) x (3 components) x (2 times the timesteps) = 9.8k node-hrs per site.
  
Number of rupture variations:  5.5 million
+
'''33.1k''' node-hours per 3-component site (686k core-hours)
  
=== Deterministic ===
+
'''9.47M''' node-hours for standard 3-component So Cal 286-site map (196M core-hours)
  
Number of jobs:  5.6 million
+
'''117M''' node-hours for increased density 3-component So Cal 3545-site map (2.4B core-hours)
  
Storage:  320 GB SGTs, 250 GB seismograms
+
'''46.3M''' node-hours for statewide adaptive 3-component California 1400-site map (960M core-hours)
 
 
SUs: 80k SGTs + 150k post-processing = 230k
 
 
 
=== Broadband ===
 
 
 
Number of jobs:  16.6 million
 
 
 
Storage:  320 GB SGTs, 2 TB seismograms
 
 
 
SUs:  80k SGTs + 200k post-processing = 280k
 
 
 
== Southern California simulations ==
 
 
 
200 sites
 
=== 0.5 Hz, deterministic ===
 
 
 
Number of jobs:  1.1 billion
 
 
 
Storage:  7.8 TB SGTs, 24.4 TB seismograms
 
 
 
SUs: 7.6 million
 
 
 
=== 0.5 Hz, broadband ===
 
 
 
Number of jobs:  3.3 billion
 
 
 
Storage:  7.8 TB SGTs, 97 TB seismograms
 
 
 
SUs: 12.8 million
 
 
 
=== 1 Hz, deterministic ===
 
 
 
Number of jobs:  1.1 billion
 
 
 
Storage:  62.5 TB SGTs, 48.8 TB seismograms
 
 
 
SUs: 46 million
 
 
 
=== 1 Hz, broadband ===
 
 
 
Number of jobs:  3.3 billion
 
 
 
Storage:  62.5 TB SGTs, 400 TB seismograms
 
 
 
SUs:  56 million
 
 
 
== Statewide simulations ==
 
 
 
1400 sites
 
=== 0.5 Hz, deterministic ===
 
 
 
Number of jobs:  7.8 billion
 
 
 
Storage:  54.7 TB SGTs, 171 TB seismograms
 
 
 
SUs: 53 million
 
 
 
=== 0.5 Hz, broadband ===
 
 
 
Number of jobs:  23.2 billion
 
 
 
Storage:  54.7 TB SGTs, 683 TB seismograms
 
 
 
SUs: 89.6 million
 
 
 
=== 1 Hz, deterministic ===
 
 
 
Number of jobs:  7.8 billion
 
 
 
Storage:  437.5 TB SGTs, 341.8 TB seismograms
 
 
 
SUs: 322 million
 
 
 
=== 1 Hz, broadband ===
 
 
 
Number of jobs:  23.2 billion
 
 
 
Storage:  437.5 TB SGTs, 2800 TB seismograms
 
 
 
SUs:  392 million
 

Latest revision as of 15:52, 20 June 2014

We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.

The UCERF 3 estimates assume that the number of ruptures increases from 15,000 to 350,000, but the number of rupture variations per rupture on average remains the same.

The 0.5 Hz numbers are taken from Study 14.2.

The node-hours are estimates based on the XE6 and XK7 nodes on Blue Waters.

1.0 Hz, 3 component

SGTs: At 0.5 Hz, it requires 38 GPU node-hrs per component.

 (38 GPU node-hrs per component) x (3 components) x (8 times the gridpoints) x (2 times the timesteps) x (20% more efficient due to more work per GPU) = 1460 node-hrs per site.

23% of node hours to calculate SGTs.

PP: At 0.5 Hz, it requires 41 CPU node-hrs per component.

 (41 CPU node-hrs per components) x (3 components) x (25 times the rupture points) x (2 times the timesteps) x (20% more efficient due to rupture generator improvements) = 4920 node-hrs per site.

77% of node hours to calculate PP.

Each site requires about 550,000 rupture variations (410,000 x 4/3 for rupture variations v3.3)

6380 node-hours per 3-component site (181k core-hours)

1.82M node-hours for standard 3-component So Cal 286-site map (51.7M core-hours)

5.73M node-hours for increased density 3-component So Cal 898-site map (162M core-hours)

8.93M node-hours for statewide adaptive 3-component California 1400-site map (253M core-hours)

2.0 Hz

SGTs: At 1.0 Hz, it requires 485 GPU node-hrs per component.

 (485 GPU node-hrs per component) x (3 components) x (8 times the gridpoints) x (2 times the timesteps) = 23.3k node-hrs per site.

PP: At 1.0 Hz, it requires 1640 CPU node-hrs per component.

 (1640 CPU node-hrs per components) x (3 components) x (2 times the timesteps) = 9.8k node-hrs per site.

33.1k node-hours per 3-component site (686k core-hours)

9.47M node-hours for standard 3-component So Cal 286-site map (196M core-hours)

117M node-hours for increased density 3-component So Cal 3545-site map (2.4B core-hours)

46.3M node-hours for statewide adaptive 3-component California 1400-site map (960M core-hours)