CyberShake Computational Estimates: Difference between revisions

From SCECpedia
Jump to navigationJump to search
← Older editNewer edit →
Scottcal (talk | contribs)
8,098 edits
Scottcal (talk | contribs)
8,098 edits
No edit summary
Line 1: Line 1:
We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.
We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.


== SGT Simulation Parameters ==
All these estimates are for UCERF 3, estimating that the number of ruptures increases from 15000 to 200,000, but the number of rupture variations per rupture remains the same.
<pre>
1Hz CyberShake Estimates 2010.11.1
Max Freq 1Hz
Mesh Size Mesh Pts 12,000,000,000 320
Time Steps 40,000
Total Sites 4240
Total CPU Hours 320M
Estimated BW 10+Pflop/s
Estimated 1Hz Jaguar (half machine) 120 Days
Esimated 1Hz BW 40 Days
Estimated Effeciency Jagaur 8-10%
Jaguar 2.6GHz x 4FPU 10.4Gflop/s per core
Blue Waters 4GHz x 8 FPUs 32Gflop/s per core


</pre>
== 0.5 Hz (per site) ==


== Southern California, 0.5 Hz (current functionality) ==
Number of rupture variations:  5.5 million


Sites: 223 sites (802 on 5 km grid)
=== Deterministic ===


Jobs: 190 million
Number of jobs: 5.6 million


CPU-hours: 5.5 million (Ranger)
Storage: 40 GB SGTs, 125 GB seismograms


Data products (seismograms, spectral acceleration): 2.1 TB
SUs: 12k SGTs + 26k post-processing = 38k


Runtime on half-Ranger: 174 hrs (7.3 days)
=== Broadband ===


Runtime on half-Jaguar: 40 hrs (1.7 days)
Number of jobs: 16.6 million


Runtime on half-BW(=half-Mira): 10 hrs
Storage: 40 GB SGTs, 500 GB seismograms


Database entries: 366 million
SUs: 12k SGTs + 52k post-processing = 64k


== Southern California, 1.0 Hz ==


AWP-ODC
== 1 Hz (per site) ==


SRFs increase by 25x to ~25 TB
Number of rupture variations:  5.5 million


Sites: 223 sites
=== Deterministic ===


Jobs: 190 million
Number of jobs: 5.6 million


CPU-hours: 19.3 million (Ranger)
Storage: 320 GB SGTs, 250 GB seismograms


Data products (seismograms, spectral acceleration): 4.0 TB
SUs: 80k SGTs + 150k post-processing = 230k


Runtime on half-Ranger: 613 hrs (25.5 days)
=== Broadband ===


Runtime on half-Jaguar: 142 hrs (5.9 days)
Number of jobs: 16.6 million


Runtime on half-BW(=half-Mira): 35 hrs (1.5 days)
Storage: 320 GB SGTs, 2 TB seismograms


Database entries: 366 million
SUs: 80k SGTs + 200k post-processing = 280k


== California, 0.5 Hz ==
== Southern California simulations ==


=== Current software ===
200 sites
=== 0.5 Hz, deterministic ===


Sites: 4240
Number of jobs: 1.1 billion


Jobs: 3.6 billion
Storage: 7.8 TB SGTs, 24.4 TB seismograms


CPU-hours: 104.6 million (Ranger)
SUs: 7.6 million


Data products (seismograms, spectral acceleration): 39.9 TB
=== 0.5 Hz, broadband ===


Runtime on half-Ranger: 3322 hrs (138.4 days)
Number of jobs: 3.3 billion


Runtime on half-Jaguar: 771 hrs (32.2 days)
Storage: 7.8 TB SGTs, 97 TB seismograms


Runtime on half-BW(=half-Mira): 192 hrs (8 days)
SUs: 12.8 million


Database entries: 6.95 billion
=== 1 Hz, deterministic ===


=== With AWP-ODC ===
Number of jobs:  1.1 billion


Sites: 4240
Storage: 62.5 TB SGTs, 48.8 TB seismograms


Jobs: 3.6 billion
SUs: 46 million


CPU-hours: 83.5 million (Ranger)
=== 1 Hz, broadband ===


Data products (seismograms, spectral acceleration): 39.9 TB
Number of jobs: 3.3 billion


Runtime on half-Ranger: 2652 hrs (110.5 days)
Storage: 62.5 TB SGTs, 400 TB seismograms


Runtime on half-Jaguar: 616 hrs (25.7 days)
SUs: 56 million


Runtime on half-BW(=half-Mira): 153 hrs (6.4 days)
== Statewide simulations ==


Database entries: 6.95 billion
1400 sites
=== 0.5 Hz, deterministic ===


== California, 1.0 Hz ==
Number of jobs:  7.8 billion


AWP-ODC
Storage:  54.7 TB SGTs, 171 TB seismograms


SRFs increase by 25x to ~45 TB
SUs: 53 million


[http://hypocenter.usc.edu/research/cybershake/CA_10km_sites.png Site Map: 4240]
=== 0.5 Hz, broadband ===


Jobs: 3.6 billion
Number of jobs: 23.2 billion


CPU-hours: 376.2 million (337.8 million - SGT generation only, 339.1 million - SGT workflow)
Storage: 54.7 TB SGTs, 683 TB seismograms


Data products (1 Hz seismograms, spectral acceleration): 76.5 TB, (10 Hz seismograms): 230 TB
SUs: 89.6 million


Runtime on half-Ranger: 11947 hrs (497.8 days)
=== 1 Hz, deterministic ===


Runtime on half-Jaguar: 3096 hrs (129 days)
Number of jobs: 7.8 billion


Runtime on half-BW(=half-Mira): 770 hrs (32.1 days) (4.3% of yearly CPU-hrs)
Storage: 437.5 TB SGTs, 341.8 TB seismograms


Database entries: 6.95 billion
SUs: 322 million
 
=== 1 Hz, broadband ===
 
Number of jobs:  23.2 billion
 
Storage:  437.5 TB SGTs, 2800 TB seismograms
 
SUs:  392 million

Revision as of 23:10, 3 April 2012

We will describe or current best estimates for the CyberShake computational and data requirements as we progress in our simulation planning and testing. These estimates will help us identify which aspects of the CyberShake computational system needs to be optimized to work within our time and resource constraints.

All these estimates are for UCERF 3, estimating that the number of ruptures increases from 15000 to 200,000, but the number of rupture variations per rupture remains the same.

0.5 Hz (per site)

Number of rupture variations: 5.5 million

Deterministic

Number of jobs: 5.6 million

Storage: 40 GB SGTs, 125 GB seismograms

SUs: 12k SGTs + 26k post-processing = 38k

Broadband

Number of jobs: 16.6 million

Storage: 40 GB SGTs, 500 GB seismograms

SUs: 12k SGTs + 52k post-processing = 64k


1 Hz (per site)

Number of rupture variations: 5.5 million

Deterministic

Number of jobs: 5.6 million

Storage: 320 GB SGTs, 250 GB seismograms

SUs: 80k SGTs + 150k post-processing = 230k

Broadband

Number of jobs: 16.6 million

Storage: 320 GB SGTs, 2 TB seismograms

SUs: 80k SGTs + 200k post-processing = 280k

Southern California simulations

200 sites

0.5 Hz, deterministic

Number of jobs: 1.1 billion

Storage: 7.8 TB SGTs, 24.4 TB seismograms

SUs: 7.6 million

0.5 Hz, broadband

Number of jobs: 3.3 billion

Storage: 7.8 TB SGTs, 97 TB seismograms

SUs: 12.8 million

1 Hz, deterministic

Number of jobs: 1.1 billion

Storage: 62.5 TB SGTs, 48.8 TB seismograms

SUs: 46 million

1 Hz, broadband

Number of jobs: 3.3 billion

Storage: 62.5 TB SGTs, 400 TB seismograms

SUs: 56 million

Statewide simulations

1400 sites

0.5 Hz, deterministic

Number of jobs: 7.8 billion

Storage: 54.7 TB SGTs, 171 TB seismograms

SUs: 53 million

0.5 Hz, broadband

Number of jobs: 23.2 billion

Storage: 54.7 TB SGTs, 683 TB seismograms

SUs: 89.6 million

1 Hz, deterministic

Number of jobs: 7.8 billion

Storage: 437.5 TB SGTs, 341.8 TB seismograms

SUs: 322 million

1 Hz, broadband

Number of jobs: 23.2 billion

Storage: 437.5 TB SGTs, 2800 TB seismograms

SUs: 392 million

Retrieved from "https://strike.scec.org/index.php?title=CyberShake_Computational_Estimates&oldid=4801"