20 IBM eX5 Implementation GuideFigure 2-3 Independent processor buses, as in the x3850 M2 and x3950 M2Instead of a parallel bus connecting the processors to a core chip set, which functions as botha memory and I/O controller, the Xeon 6500 and 7500 family processors implemented in IBMeX5 servers include a separate memory controller to each processor. Processor-to-processorcommunications are carried over shared-clock, or coherent QPI links, and I/O is transportedover non-coherent QPI links through I/O hubs. Figure 2-4 shows this information.Figure 2-4 Figure 2-4 QPI, as used in the eX5 portfolioIn previous designs, the entire range of memory was accessible through the core chip set byeach processor, a shared memory architecture. This design creates a non-uniform memoryaccess (NUMA) system, in which part of the memory is directly connected to the processorwhere a given thread is running, and the rest must be accessed over a QPI link throughanother processor. Similarly, I/O can be local to a processor, or remote through anotherprocessor.For QPI use, Intel has modified the MESI cache coherence protocol to include a forwardingstate, so when a processor asks to copy a shared cache line, only one other processorresponds.For more information about QPI, go to the following website:http://www.intel.com/technology/quickpath/Memory I/OProcessor ProcessorProcessor ProcessorCore Chip setI/O HubProcessor ProcessorProcessor ProcessorI/O HubMemoryI/OI/OMemoryMemory Memory
