Difference between revisions of "Blue Waters"

From SCECpedia
Jump to navigationJump to search
(Redirected page to Blue Waters Project)
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
By early December, we expect a single-cabinet test system ("Phase 0") to be delivered
+
#redirect [[Blue Waters Project]]
to NCSA, and plan to allow participating science team members access to it during the Fall
 
PRAC Workshop (Dec. 13).  This system will have 52 XE6 compute nodes (832 AMD Interlagos
 
"Bulldozer" cores) and 32 XK6 compute nodes (256 Interlagos cores plus 32 Fermi accelerators),
 
all in the same interconnect fabric.
 
 
 
Early next year, a substantial (>1.3 PFLOP peak) system ("Phase 1") will be delivered to NCSA for early science
 
runs and possibly some porting and tuning.  This system will be available for a few months, after
 
which it will be integrated with the rest of the cabinets comprising the full Blue Waters system.
 
By early January, we will send out detailed information on the process for getting access to
 
the early science system.
 
 
 
The table below summarizes the key parameters of the various Phases of the
 
Blue Waters system.  Phase 2 is expected to go into production in Summer 2012,
 
and in Fall 2012, the Phase 3 system will have Kepler accelerators installed in the
 
XK nodes.  For more details, see the newly revised Blue Waters PRAC wiki page:
 
 
 
https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information
 
 
 
<pre>
 
 
 
 
 
System Components
 
 
 
 
UIUC/NCSA
 
 
 
Blue Waters
 
 
 
Phase I System
 
 
 
 
UIUC/NCSA
 
 
 
Blue Waters
 
 
 
Phase II System
 
 
 
 
UIUC/NCSA
 
 
 
Blue Waters
 
 
 
Phase III System
 
 
 
System
 
 
 
 
XE6
 
 
 
 
XE6/XK6
 
 
 
 
XE6/XK7
 
 
 
Cabinets – XE6
 
 
 
 
47
 
 
 
 
237
 
 
 
 
237
 
 
 
Cabinets – XK6
 
 
 
 
0
 
 
 
 
32
 
 
 
 
32
 
 
 
Cabinets – XIO
 
 
 
 
1
 
 
 
 
7
 
 
 
 
7
 
 
 
x86 Peak Performance (PF)
 
 
 
 
1.41
 
 
 
 
7.62
 
 
 
 
7.62
 
 
 
GPU Peak Performance (PF)
 
 
 
 
0
 
 
 
 
0
 
 
 
 
> 4
 
 
 
Peak Performance (PF)
 
 
 
 
1.41
 
 
 
 
7.62
 
 
 
 
>11.5
 
 
 
Compute Nodes – XE6
 
 
 
 
4,512
 
 
 
 
22,752
 
 
 
 
22,752
 
 
 
Compute Nodes – XK6
 
 
 
 
0
 
 
 
 
3,072
 
 
 
 
3,072
 
 
 
x86 Compute Cores
 
 
 
 
72,192
 
 
 
 
388,608
 
 
 
 
388,608
 
 
 
Service and I/O Nodes (total)
 
 
 
 
96
 
 
 
 
672
 
 
 
 
672
 
 
 
-    LNET Routers
 
 
 
 
50
 
 
 
 
582
 
 
 
 
582
 
 
 
-    Network
 
 
 
 
4
 
 
 
 
8
 
 
 
 
8
 
 
 
-    Boot
 
 
 
 
1
 
 
 
 
2
 
 
 
 
2
 
 
 
-    SDB
 
 
 
 
1
 
 
 
 
2
 
 
 
 
2
 
 
 
-    Internal Only XIO nodes
 
 
 
 
2
 
 
 
 
4
 
 
 
 
4
 
 
 
-    Unassigned
 
 
 
 
38
 
 
 
 
74
 
 
 
 
74
 
 
 
XE6 Compute Node Perf (GF)
 
 
 
 
313.6
 
 
 
 
313.6
 
 
 
 
313.6
 
 
 
XK6 Compute Node (GF)
 
 
 
 
NA
 
 
 
 
156.8 (CPU)
 
 
 
 
156.8(CPU) + >1,000(GPU)
 
 
 
Compute Node Memory (XE6/XK6)
 
 
 
 
64/32
 
 
 
 
64/32
 
 
 
 
64/32
 
 
 
XE6 Compute Node Memory Bandwidth (GB/s)
 
 
 
 
102.4
 
 
 
 
102.4
 
 
 
 
102.4
 
 
 
XK6 Compute Node Memory Bandwidth (GB/s)
 
 
 
 
51.2
 
 
 
 
51.2
 
 
 
 
51.2+>140
 
 
 
Total System Memory (TB)
 
 
 
 
282
 
 
 
 
1,518
 
 
 
 
1,518
 
 
 
Interconnect Architecture
 
 
 
 
3D Torus
 
 
 
12x8x24
 
 
 
 
3D Torus
 
 
 
23x24x24
 
 
 
 
3D Torus
 
 
 
23x24x24
 
 
 
Peak Node Injection Bandwidth (GB/s)
 
 
 
 
9.6
 
 
 
 
9.6
 
 
 
 
9.6
 
 
 
Average Bisection
 
 
 
Bandwidth (TB/s)
 
 
 
 
8.65
 
 
 
 
17.55
 
 
 
 
17.55
 
 
 
Minimum Bisection
 
 
 
Bandwidth (TB/s)
 
 
 
 
7.50
 
 
 
 
10.35
 
 
 
 
10.35
 
 
 
Peak Interconnect Injection Bandwidth (GB/s)
 
 
 
 
44.24
 
 
 
 
254.36
 
 
 
 
254.36
 
 
 
Storage Cabinets
 
 
 
 
>2
 
 
 
 
>30
 
 
 
 
>30
 
 
 
Total Raw Storage (PB)
 
 
 
 
>2.5
 
 
 
 
>30
 
 
 
 
>30
 
 
 
Total Usable Storage (PB)
 
 
 
 
>2
 
 
 
 
>25
 
 
 
 
>25
 
 
 
Storage SSUs
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Disks
 
 
 
 
 
 
 
 
>17000
 
 
 
 
>17000
 
 
 
External Login Nodes
 
 
 
 
2
 
 
 
 
4
 
 
 
 
4
 
 
 
External Data Movers (esDM)
 
 
 
 
40
 
 
 
 
100
 
 
 
 
100
 
 
 
Peak Power (MW)
 
 
 
 
2.7
 
 
 
 
15.7
 
 
 
 
15.7
 
 
 
Power (Linpack) (MW)
 
 
 
 
2.4
 
 
 
 
< 13.6
 
 
 
 
13.6
 
 
 
Power (Applications) (MW)
 
 
 
 
2.2
 
 
 
 
< 12.6
 
 
 
 
12.75
 
 
 
</pre>
 

Latest revision as of 18:10, 31 October 2013