Difference between revisions of "Blue Waters"

From SCECpedia
Jump to navigationJump to search
(Created page with '<pre> System Components UIUC/NCSA Blue Waters Phase I System UIUC/NCSA Blue Waters Phase II System UIUC/NCSA Blue Waters Phase III System System XE6 XE6/X…')
 
Line 1: Line 1:
 +
By early December, we expect a single-cabinet test system ("Phase 0") to be delivered
 +
to NCSA, and plan to allow participating science team members access to it during the Fall
 +
PRAC Workshop (Dec. 13).  This system will have 52 XE6 compute nodes (832 AMD Interlagos
 +
"Bulldozer" cores) and 32 XK6 compute nodes (256 Interlagos cores plus 32 Fermi accelerators),
 +
all in the same interconnect fabric.
 +
 +
Early next year, a substantial (>1.3 PFLOP peak) system ("Phase 1") will be delivered to NCSA for early science
 +
runs and possibly some porting and tuning.  This system will be available for a few months, after
 +
which it will be integrated with the rest of the cabinets comprising the full Blue Waters system.
 +
By early January, we will send out detailed information on the process for getting access to
 +
the early science system.
 +
 +
The table below summarizes the key parameters of the various Phases of the
 +
Blue Waters system.  Phase 2 is expected to go into production in Summer 2012,
 +
and in Fall 2012, the Phase 3 system will have Kepler accelerators installed in the
 +
XK nodes.  For more details, see the newly revised Blue Waters PRAC wiki page:
 +
 +
https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information
 +
 
<pre>
 
<pre>
  

Revision as of 04:49, 13 December 2011

By early December, we expect a single-cabinet test system ("Phase 0") to be delivered to NCSA, and plan to allow participating science team members access to it during the Fall PRAC Workshop (Dec. 13). This system will have 52 XE6 compute nodes (832 AMD Interlagos "Bulldozer" cores) and 32 XK6 compute nodes (256 Interlagos cores plus 32 Fermi accelerators), all in the same interconnect fabric.

Early next year, a substantial (>1.3 PFLOP peak) system ("Phase 1") will be delivered to NCSA for early science runs and possibly some porting and tuning. This system will be available for a few months, after which it will be integrated with the rest of the cabinets comprising the full Blue Waters system. By early January, we will send out detailed information on the process for getting access to the early science system.

The table below summarizes the key parameters of the various Phases of the Blue Waters system. Phase 2 is expected to go into production in Summer 2012, and in Fall 2012, the Phase 3 system will have Kepler accelerators installed in the XK nodes. For more details, see the newly revised Blue Waters PRAC wiki page:

https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information



System Components
	

UIUC/NCSA

Blue Waters

Phase I System
	

UIUC/NCSA

Blue Waters

Phase II System
	

UIUC/NCSA

Blue Waters

Phase III System

System
	

XE6
	

XE6/XK6
	

XE6/XK7

Cabinets – XE6
	

47
	

237
	

237

Cabinets – XK6
	

0
	

32
	

32

Cabinets – XIO
	

1
	

7
	

7

x86 Peak Performance (PF)
	

1.41
	

7.62
	

7.62

GPU Peak Performance (PF)
	

0
	

0
	

> 4

Peak Performance (PF)
	

1.41
	

7.62
	

>11.5

Compute Nodes – XE6
	

4,512
	

22,752
	

22,752

Compute Nodes – XK6
	

0
	

3,072
	

3,072

x86 Compute Cores
	

72,192
	

388,608
	

388,608

Service and I/O Nodes (total)
	

96
	

672
	

672

-    LNET Routers
	

50
	

582
	

582

-    Network
	

4
	

8
	

8

-    Boot
	

1
	

2
	

2

-    SDB
	

1
	

2
	

2

-    Internal Only XIO nodes
	

2
	

4
	

4

-    Unassigned
	

38
	

74
	

74

XE6 Compute Node Perf (GF)
	

313.6
	

313.6
	

313.6

XK6 Compute Node (GF)
	

NA
	

156.8 (CPU)
	

156.8(CPU) + >1,000(GPU)

Compute Node Memory (XE6/XK6)
	

64/32
	

64/32
	

64/32

XE6 Compute Node Memory Bandwidth (GB/s)
	

102.4
	

102.4
	

102.4

XK6 Compute Node Memory Bandwidth (GB/s)
	

51.2
	

51.2
	

51.2+>140

Total System Memory (TB)
	

282
	

1,518
	

1,518

Interconnect Architecture
	

3D Torus

12x8x24
	

3D Torus

23x24x24
	

3D Torus

23x24x24

Peak Node Injection Bandwidth (GB/s)
	

9.6
	

9.6
	

9.6

Average Bisection

Bandwidth (TB/s)
	

8.65
	

17.55
	

17.55

Minimum Bisection

Bandwidth (TB/s)
	

7.50
	

10.35
	

10.35

Peak Interconnect Injection Bandwidth (GB/s)
	

44.24
	

254.36
	

254.36

Storage Cabinets
	

>2
	

>30
	

>30

Total Raw Storage (PB)
	

>2.5
	

>30
	

>30

Total Usable Storage (PB)
	

>2
	

>25
	

>25

Storage SSUs
	

 
	

 
	

 

Disks
	

 
	

>17000
	

>17000

External Login Nodes
	

2
	

4
	

4

External Data Movers (esDM)
	

40
	

100
	

100

Peak Power (MW)
	

2.7
	

15.7
	

15.7

Power (Linpack) (MW)
	

2.4
	

< 13.6
	

13.6

Power (Applications) (MW)
	

2.2
	

< 12.6
	

12.75