Cray colaborating with Nvidia for GPU-based XE6 blades
At the recent 2010 GPU Technology Conference in San Jose, California, Cray announced that it is developing blades based on the Nvidia Tesla 20-Series GPUs for use in the Cray XE6 product line. The combination of Cray’s new Gemini system interconnect (which is featured in the Cray XE6 system) paired with Nvidia GPUs will give Cray XE6 customers a powerful combination of scalability and production-quality GPU-based HPC in a single system.
‘We spend a significant amount of time meeting with our customers and talking about how we can build the most scalable HPC systems that blend industry-leading technologies designed to meet their supercomputing needs,’ said Barry Bolding, vice president of Cray’s products division. ‘Our customers have expressed a growing interest in having accelerator technology in Cray systems, and we believe a Cray XE6 blade with Nvidia GPUs will provide the performance, scalability, and reliability that a growing segment of our customer base is looking for. The use of GPUs as accelerators in the HPC marketplace is maturing, and we are excited to be working with Nvidia to offer this functionality to our customers.’
Cray XE6 Specifications
| Processor | Eight or 12-core 64-bit AMD Opteron 6100 Series processors; up to 192 per cabinet 64k L1 instruction cache, 64k L1 data cache, 512 KB L2 cache per processor core, 12 MB shared L3 cache |
| Memory | 32 GB or 64 GB registered ECC DDR3 SDRAM per compute node Memory Bandwidth: 85.3 GB/s per compute node |
| Compute Cabinet | Cores: 1,536 or 2,304 processor cores per system cabinet Peak Performance: 12.2 to 20.2 teraflops per system cabinet |
| Interconnect | 1 Gemini routing and communications ASIC per two compute nodes |
| 48 switch ports per Gemini chip (160 GB/s switching capacity per chip) | |
| 3-D torus interconnect | |
| System Administration | Cray System Management workstation Graphical and command line system administration |
| Single-system view for system administration | |
| System software rollback capability | |
| Reliability Features (Hardware) | Cray Hardware Supervisory System (HSS) with independent 100 Mb/s management fabric between all system blades and cabinet-level controllers |
| Full ECC protection of all packet traffic in the Gemini network | |
| Redundant power supplies; redundant voltage regulator modules | |
| Redundant paths to all system RAID | |
| Variable-speed axial turbofan with integrated pressure and temperature sensors | |
| Reliability Features (Software) | HSS system monitors operation of all operating system kernels Lustre file system object storage target failover; Lustre metadata server failover |
| Software failover for critical system services including system database, system logger, and batch subsystems NodeKARE (Node Knowledge and Reconfiguration) |
|
| Operating System | Cray Linux Environment (components include SUSE Linux SLES11, HSS and SMW software) Extreme Scalability Mode (ESM) and Cluster Compatibility Mode (CCM) |
| Compilers, Libraries & Tools | PGI compilers, Cray Compiler Environment, PathScale Support for Fortran 77, 90, 95; C/C++, UPC, Co-Array Fortran MPI 2.0, Cray SHMEM, other standard MPI libraries using CCM Cray Apprentice, Cray PAT and Cray Compiler included with systems |
| Job Management | PBS Professional job management system Moab Adaptive Computing Suite job management system Platform LSF job management system |
| External I/O Interface | InfiniBand, 10 Gigabit Ethernet, Fibre Channel (FC) and Ethernet |
| Disk Storage | Full line of FC-attached disk arrays with support for FC and SATA disk drives |
| Parallel File System | Lustre, Data Virtualization Service allows support for NFS, external Lustre and other file systems |
| Power | 45-54.1 kW (45.9 – 55.2 kVA) per cabinet, depending on configuration Circuit requirements: three-phase wye, 100 AMP at 480/277 and 125 AMP at 400/230 (three-phase, neutral and ground) |
| Cooling | Air-cooled, air flow: 3,000 cfm (1.41 m3/s); intake: bottom; exhaust: top Optional ECOphlex liquid cooling |
| Dimensions (Cabinet) | H 93 in. (2,362 mm) x W 22.50 in. (572 mm) x D 56.75 in. (1,441 mm) |
| Weight (Maximum) | 1,600 lbs. per cabinet (725 kg) air cooled; 2,000 lbs. per cabinet (907 kg) liquid cooled |
| Acoustical Noise Level | 75 dBA at 3.3 ft (1.0 m) |
| Regulatory Compliance | UL 60950-1, CAN/CSA – C 22.2 No. 60950-1, CE-mark, RoHS, WEEE |
| Safety | FCC Class A, VCCI Class A, ICES-003, EN 50022:2006 Class A, AS/NZS CISPR 22:2006, EN 55024: 1998 +A1:2001 +A2:2003 |
With this future addition, Cray will be able to offer its customers a full range of accelerator solutions for HPC – from the deskside to the supercomputer. The Cray CX line of deskside and midrange systems, including the Cray CX1 and Cray CX1000 systems, are currently available with Nvidia Tesla GPUs.
Cray and Nvidia continue to work together on the development of future GPU accelerator technologies in HPC as a potential path towards exascale computing. Cray is partnering with Nvidia on a team that was recently awarded a $25m research grant from the Defense Advanced Research Projects Agency (DARPA) as part of its Ubiquitous High Performance Computing (UHPC) programme.
‘The Cray XE6 is designed to solve the system challenges of petascale computing just as Tesla GPUs are providing the computational power efficiency to bring this level of computing to the larger HPC community,’ said Andy Keane, general manager, Tesla business at Nvidia. ‘Cray’s planned Tesla GPU-based system will be a powerful combination of technologies that will provide the highest level of performance and scalability for power efficient supercomputing.
Category: HPC
Subscribe
If you enjoyed this article, subscribe to receive more just like it.
