|
|
| Line 1: |
Line 1: |
| By early December, we expect a single-cabinet test system ("Phase 0") to be delivered
| |
| to NCSA, and plan to allow participating science team members access to it during the Fall
| |
| PRAC Workshop (Dec. 13). This system will have 52 XE6 compute nodes (832 AMD Interlagos
| |
| "Bulldozer" cores) and 32 XK6 compute nodes (256 Interlagos cores plus 32 Fermi accelerators),
| |
| all in the same interconnect fabric.
| |
|
| |
|
| Early next year, a substantial (>1.3 PFLOP peak) system ("Phase 1") will be delivered to NCSA for early science
| |
| runs and possibly some porting and tuning. This system will be available for a few months, after
| |
| which it will be integrated with the rest of the cabinets comprising the full Blue Waters system.
| |
| By early January, we will send out detailed information on the process for getting access to
| |
| the early science system.
| |
|
| |
|
| The table below summarizes the key parameters of the various Phases of the | | The table below summarizes the key parameters of the various Phases of the |
| Line 17: |
Line 7: |
|
| |
|
| https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information | | https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information |
|
| |
| <pre>
| |
|
| |
|
| |
| System Components
| |
|
| |
|
| |
| UIUC/NCSA
| |
|
| |
| Blue Waters
| |
|
| |
| Phase I System
| |
|
| |
|
| |
| UIUC/NCSA
| |
|
| |
| Blue Waters
| |
|
| |
| Phase II System
| |
|
| |
|
| |
| UIUC/NCSA
| |
|
| |
| Blue Waters
| |
|
| |
| Phase III System
| |
|
| |
| System
| |
|
| |
|
| |
| XE6
| |
|
| |
|
| |
| XE6/XK6
| |
|
| |
|
| |
| XE6/XK7
| |
|
| |
| Cabinets – XE6
| |
|
| |
|
| |
| 47
| |
|
| |
|
| |
| 237
| |
|
| |
|
| |
| 237
| |
|
| |
| Cabinets – XK6
| |
|
| |
|
| |
| 0
| |
|
| |
|
| |
| 32
| |
|
| |
|
| |
| 32
| |
|
| |
| Cabinets – XIO
| |
|
| |
|
| |
| 1
| |
|
| |
|
| |
| 7
| |
|
| |
|
| |
| 7
| |
|
| |
| x86 Peak Performance (PF)
| |
|
| |
|
| |
| 1.41
| |
|
| |
|
| |
| 7.62
| |
|
| |
|
| |
| 7.62
| |
|
| |
| GPU Peak Performance (PF)
| |
|
| |
|
| |
| 0
| |
|
| |
|
| |
| 0
| |
|
| |
|
| |
| > 4
| |
|
| |
| Peak Performance (PF)
| |
|
| |
|
| |
| 1.41
| |
|
| |
|
| |
| 7.62
| |
|
| |
|
| |
| >11.5
| |
|
| |
| Compute Nodes – XE6
| |
|
| |
|
| |
| 4,512
| |
|
| |
|
| |
| 22,752
| |
|
| |
|
| |
| 22,752
| |
|
| |
| Compute Nodes – XK6
| |
|
| |
|
| |
| 0
| |
|
| |
|
| |
| 3,072
| |
|
| |
|
| |
| 3,072
| |
|
| |
| x86 Compute Cores
| |
|
| |
|
| |
| 72,192
| |
|
| |
|
| |
| 388,608
| |
|
| |
|
| |
| 388,608
| |
|
| |
| Service and I/O Nodes (total)
| |
|
| |
|
| |
| 96
| |
|
| |
|
| |
| 672
| |
|
| |
|
| |
| 672
| |
|
| |
| - LNET Routers
| |
|
| |
|
| |
| 50
| |
|
| |
|
| |
| 582
| |
|
| |
|
| |
| 582
| |
|
| |
| - Network
| |
|
| |
|
| |
| 4
| |
|
| |
|
| |
| 8
| |
|
| |
|
| |
| 8
| |
|
| |
| - Boot
| |
|
| |
|
| |
| 1
| |
|
| |
|
| |
| 2
| |
|
| |
|
| |
| 2
| |
|
| |
| - SDB
| |
|
| |
|
| |
| 1
| |
|
| |
|
| |
| 2
| |
|
| |
|
| |
| 2
| |
|
| |
| - Internal Only XIO nodes
| |
|
| |
|
| |
| 2
| |
|
| |
|
| |
| 4
| |
|
| |
|
| |
| 4
| |
|
| |
| - Unassigned
| |
|
| |
|
| |
| 38
| |
|
| |
|
| |
| 74
| |
|
| |
|
| |
| 74
| |
|
| |
| XE6 Compute Node Perf (GF)
| |
|
| |
|
| |
| 313.6
| |
|
| |
|
| |
| 313.6
| |
|
| |
|
| |
| 313.6
| |
|
| |
| XK6 Compute Node (GF)
| |
|
| |
|
| |
| NA
| |
|
| |
|
| |
| 156.8 (CPU)
| |
|
| |
|
| |
| 156.8(CPU) + >1,000(GPU)
| |
|
| |
| Compute Node Memory (XE6/XK6)
| |
|
| |
|
| |
| 64/32
| |
|
| |
|
| |
| 64/32
| |
|
| |
|
| |
| 64/32
| |
|
| |
| XE6 Compute Node Memory Bandwidth (GB/s)
| |
|
| |
|
| |
| 102.4
| |
|
| |
|
| |
| 102.4
| |
|
| |
|
| |
| 102.4
| |
|
| |
| XK6 Compute Node Memory Bandwidth (GB/s)
| |
|
| |
|
| |
| 51.2
| |
|
| |
|
| |
| 51.2
| |
|
| |
|
| |
| 51.2+>140
| |
|
| |
| Total System Memory (TB)
| |
|
| |
|
| |
| 282
| |
|
| |
|
| |
| 1,518
| |
|
| |
|
| |
| 1,518
| |
|
| |
| Interconnect Architecture
| |
|
| |
|
| |
| 3D Torus
| |
|
| |
| 12x8x24
| |
|
| |
|
| |
| 3D Torus
| |
|
| |
| 23x24x24
| |
|
| |
|
| |
| 3D Torus
| |
|
| |
| 23x24x24
| |
|
| |
| Peak Node Injection Bandwidth (GB/s)
| |
|
| |
|
| |
| 9.6
| |
|
| |
|
| |
| 9.6
| |
|
| |
|
| |
| 9.6
| |
|
| |
| Average Bisection
| |
|
| |
| Bandwidth (TB/s)
| |
|
| |
|
| |
| 8.65
| |
|
| |
|
| |
| 17.55
| |
|
| |
|
| |
| 17.55
| |
|
| |
| Minimum Bisection
| |
|
| |
| Bandwidth (TB/s)
| |
|
| |
|
| |
| 7.50
| |
|
| |
|
| |
| 10.35
| |
|
| |
|
| |
| 10.35
| |
|
| |
| Peak Interconnect Injection Bandwidth (GB/s)
| |
|
| |
|
| |
| 44.24
| |
|
| |
|
| |
| 254.36
| |
|
| |
|
| |
| 254.36
| |
|
| |
| Storage Cabinets
| |
|
| |
|
| |
| >2
| |
|
| |
|
| |
| >30
| |
|
| |
|
| |
| >30
| |
|
| |
| Total Raw Storage (PB)
| |
|
| |
|
| |
| >2.5
| |
|
| |
|
| |
| >30
| |
|
| |
|
| |
| >30
| |
|
| |
| Total Usable Storage (PB)
| |
|
| |
|
| |
| >2
| |
|
| |
|
| |
| >25
| |
|
| |
|
| |
| >25
| |
|
| |
| Storage SSUs
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
| Disks
| |
|
| |
|
| |
|
| |
|
| |
|
| |
| >17000
| |
|
| |
|
| |
| >17000
| |
|
| |
| External Login Nodes
| |
|
| |
|
| |
| 2
| |
|
| |
|
| |
| 4
| |
|
| |
|
| |
| 4
| |
|
| |
| External Data Movers (esDM)
| |
|
| |
|
| |
| 40
| |
|
| |
|
| |
| 100
| |
|
| |
|
| |
| 100
| |
|
| |
| Peak Power (MW)
| |
|
| |
|
| |
| 2.7
| |
|
| |
|
| |
| 15.7
| |
|
| |
|
| |
| 15.7
| |
|
| |
| Power (Linpack) (MW)
| |
|
| |
|
| |
| 2.4
| |
|
| |
|
| |
| < 13.6
| |
|
| |
|
| |
| 13.6
| |
|
| |
| Power (Applications) (MW)
| |
|
| |
|
| |
| 2.2
| |
|
| |
|
| |
| < 12.6
| |
|
| |
|
| |
| 12.75
| |
|
| |
| </pre>
| |
The table below summarizes the key parameters of the various Phases of the
Blue Waters system. Phase 2 is expected to go into production in Summer 2012,
and in Fall 2012, the Phase 3 system will have Kepler accelerators installed in the
XK nodes. For more details, see the newly revised Blue Waters PRAC wiki page:
https://wiki.ncsa.illinois.edu/display/BWpublic/User+Information
