If you're like a lot of design engineers, you want to design a high-performance communications system that's scalable, supports "five-nines" high availability (HA), and is low in cost—and you need to start now. You've seen the switched-fabric architectures available today but don't know how to achieve the optimal design of your backplane-based system. A few of the issues you face include fabric-slot placement, chassis configuration, and components involved.
Not only do switched fabrics allow far superior speed and bandwidth capabilities, but they inherently support HA designs and system scalability. Moreover, switched fabrics eliminate the need for shared bus architectures. Only one device can communicate at a time using a bus. All other devices must wait until an arbitration scheme determines that it's their turn to use the bus.
To increase the total throughput of a bus, it must be sped up or else widened. Both options usually limit the number of devices that can be effectively connected to it. With a switch fabric, each device is hooked up to every other device in the system through a network of connections. Thus, several devices can communicate simultaneously.
Also, redundancy can be built into the switched-fabric interconnections to support five-nines HA designs. The point-to-point nature of switched fabrics can enhance reliability by isolating faults to single endpoints. With buses, an errant endpoint can bring down the entire bus. Plus, point-to-point connections are inherently friendly to device insertion and removal.
Ring, Star, Dual-Star, and Full-Mesh topologies: There are several topologies for switched fabrics. PICMG 2.16 and StarFabric can use Star or Dual-Star topologies (Fig. 1a and 1b). PLX's GigaBridge uses the Ring (Fig. 1c). Future implementations of these technologies will likely move to Full Mesh for high-end applications with complete point-to-point interconnections (Fig. 1d).
A Star topology is centralized and has only one fabric slot supported on the backplane. A Dual Star has two fabric slots supported on the backplane, providing redundancy. The Ring topology uses controllers that act as a node capable of managing multiple bus segments. They're connected via a dual-counter-rotating ring to other controllers. This arrangement forms a vast network of bus segments.
Higher bandwidth and better quality-of-service (QoS) applications call for Mesh fabrics, where each node slot is interconnected to the others with point-to-point links. Also, in Mesh fabrics, each node is an endpoint that basically manages its own traffic, without a central resource. The data rates and protocols don't depend on other data transfers in other slots. So it's highly scalable, eliminating latency and determinism problems.
If you're designing a system using PICMG 2.16, StarFabric, or GigaBridge, one of the first things to determine is whether or not to implement the CompactPCI (cPCI) bus. (These technologies are 100% compatible with cPCI and fit into the existing 1101.10/.11 mechanical framework.) When using cPCI, the cPCI bus acts as the control plane on P1 and P2, with P3 and P5 acting as the dataplane. P4 is reserved for optional H.110 bus implementation for computer telephony. The cPCI bus width can be 32 or 64 bits. Also, one can forego the cPCI bus and use the area for custom signals.
Remember that the physical positions of the fabric, system, and node slots on a switched-fabric backplane are important. You have to decide which side (left, right, or middle) to place the fabric slot(s). The backplane manufacturer may also suggest this position, taking into account that for each position, the routing complexity is different. Placing the two fabric slots on the right (same) side of the backplane makes the routing much easier. Thus, fewer layers are needed, which reduces the design and material costs. But if the system has only one fan tray, a fan going out above the two fabric slots presents a single point of failure. So when employing this method, select redundant fan trays in the chassis.
Fabric slots, node slots, and system slots: Basically, a fabric card provides switching and/or routing functions to create a fabric between the node boards. For example, with a PICMG 2.16 cPSB backplane, the fabric slots (one or two) can support a standard fabric board (1 through 19 link ports), or an extended fabric board (20 through 24 link ports).
The node slots are the points where one can have PCI bridges, Ethernet cards, DSP cards, etc. In a standard cPSB (PICMG 2.16) application in a 19-in. rack, the maximum number of node slots is 20 for a single-fabric topology, and 19 for a dual-fabric topology. In an extended cPSB for ETSI racks of up to 24 in., the number of node slots can be 20 to 24. The links between the fabric slot and node slots can be 10, 100, or 1000 Mbits/s. For Star Fabric, each link is made up of four 622-Mbit/s low-voltage differential-signaling (LVDS) transmit and receive pairs. This translates to a 2.5-Gbit/s full-duplex bandwidth.
GigaBridge uses a PCI switch-fabric controller on a PCI bus segment and interoperates with other GigaBridge controllers as nodes on the switch fabric. Each node is linked on the dual counter-rotating rings via two dual 16-bit wide LDVS-based links operating at 400 MHz.
Development backplanes provide an excellent medium to continually modify and work out your design. Figure 2 shows the layout of a development backplane for PICMG 2.16, with various options for implementations of the cPSB links, cPCI bus, H.110 bus, etc.
Gigabit speeds are more difficult to come by, but by proper design, you can avoid high backplane-layer counts. An intelligent design ensures excellent signal integrity, shielding of the high-speed lines, and creative routing based on simulation study. When dealing with the high speeds and performance demands in switched fabrics, controlling impedance and minimizing crosstalk are important backplane design issues.
Reprints
