IBM BladeCenter PS703 Technical Overview And Introduction

Also see for BladeCenter PS703: Service guide

Contents

Chapter 2. Architecture and technical overview 63top-of-rack switch. A combination of the appropriate I/O switch module in these bays and theproper Fibre Channel-capable modules in bays 3 and 5 can eliminate the top-of-rack switchrequirement. See 1.7, “Supported BladeCenter I/O modules” on page 28.InfiniBand Host Channel adapterThe InfiniBand Architecture (IBA) is an industry-standard architecture for server I/O andinterserver communication. It was developed by the InfiniBand Trade Association (IBTA) toprovide the levels of reliability, availability, performance, and scalability necessary for presentand future server systems with levels significantly better than can be achieved usingbus-oriented I/O structures.InfiniBand is an open set of interconnected standards and specifications. The main InfiniBandspecification has been published by the InfiniBand Trade Association and is available at thefollowing web page:http://www.infinibandta.org/InfiniBand is based on a switched fabric architecture of serial point-to-point links. TheseInfiniBand links can be connected to either host channel adapters (HCAs), used primarily inservers, or target channel adapters (TCAs), used primarily in storage subsystems.The InfiniBand physical connection consists of multiple byte lanes. Each individual byte laneis a four-wire, 2.5, 5.0, or 10.0 Gbps bi-directional connection. Combinations of link width andbyte lane speed allow for overall link speeds of 2.5 - 120 Gbps. The architecture defines alayered hardware protocol as well as a software layer to manage initialization and thecommunication between devices. Each link can support multiple transport services forreliability and multiple prioritized virtual communication channels.For more information about InfiniBand, read HPC Clusters Using InfiniBand on IBM PowerSystems Servers, SG24-7767, available from the following web page:http://www.redbooks.ibm.com/abstracts/sg247767.htmlThe 4X InfiniBand QDR Expansion Card is a 2-port CFFh form factor card and is onlysupported in a BladeCenter H chassis. The two ports are connected to the BladeCenter H I/Oswitch bays 7 and 8, and 9 and 10. A supported InfiniBand switch is installed in the switchbays to route the traffic either between blade servers internal to the chassis or externally to anInfiniBand fabric.2.7.4 Embedded SAS ControllerThe embedded 3 Gb SAS controller is connected to one of the Gen1 PCIe x8 buses on theGX++ multifunctional host bridge chip. The PS704 uses a single embedded SAS controllerlocated on the SMP expansion blade.More information about the SAS I/O subsystem can be found in 2.9, “Internal storage” onpage 68.2.7.5 Embedded Ethernet ControllerThe Broadcom 2-port BCM5709S network controller has its own Gen1 x4 connection to theGX++ multifunctional host bridge chip. The WOL, TOE, iSCSI, and RDMA functions of theBCM5709S are not implemented on the PS703 and PS704 blades.The connections are routed through the 5-port Broadcom BCM5387 Ethernet switch ports 3and 4. Then port 0 and port 1 of the BCM5387 connect to the Bladecenter chassis. Port 2 of