[Technology Report]
PCI Express Design: A Lesson In Techno-Shock
Start the coffee. Embedded designers are in for the long haul when it comes to implementing PCI Express, but the rewards are worth it.
BRIDGES AND SWITCHES PCI Express systems are built from four components: masters, end-nodes, bridges, and switches (Fig. 5). PCI/PCI-X does not include switches. Masters are typically processors or processor support chip sets. End-nodes consist of devices like Ethernet interfaces. Bridges provide links between PCI Express and other technologies (e.g., PCI). And switches link things together, including other switches. (Although in many cases, a single switch is all that's required.) Many processor chip sets also incorporate a switch.
Companies like PLX Technology and IDT provide a range of bridges and switches. PLX Technology developed a PCI Express-to-PCI bridge, the PEX 8111, and a PCI Express-to-PCI/PCI-X bridge, the PEX 8114 (Fig. 6). IDT's family includes the PEB20N1 and PEB20N2, which support PCI-X 2.0 Mode 1 and 2. These bridges will be used primarily to link legacy designs to PCI Express. Routing for a single lane is much easier than multilane designs, as is keeping the bridge chip close to the connector or switch that supports it.
Both IDT's and PLX Technology's switches support nontransparent bridging. The bridge then can be used as a master in multimaster applications such as cluster servers.
IDT's PES12N3 and PES24N3 are 12- and 24-lane dual-port switches. The port specifications translate to three ports with up to x8 lanes per port. PCI Express can autonegotiate all the way down to a x1 port.
PLX Technology's parts include the PEX 8516 and PEX 8532, with 16 and 32 lanes, respectively. The PEX 8516 supports up to 16 lanes and can have up to eight downstream ports where each can autonegotiate down to one lane if necessary. These chips usually wind up in higher-end systems with wide, multilane connections, which tend to be more difficult to lay out.
While designing with PCI Express can be more difficult, there are some benefits other than high performance and hot-plug capability that will keep you in the game. For example, unused lanes needn't be terminated, because the SERDES for these lanes are turned off. This means the pins won't be used. This also aids in reducing power consumption. Autonegotiation can determine the number of lanes used.
Switch chips typically come in a high-density, ball-grid-array (BGA) package. One question concerning this packaging is the ability to cleanly escape the BGA fields with differential signals. The small-geometry package leads to a smaller system footprint. But the typical 1-mm pitch BGA is very difficult to route when it comes to differential pairs. This is particularly true with PCI Express, which requires a 100-Ω differential impedance. Most chip designs try to use the outer BGA pins for the PCI Express signals. Yet this isn't always an easy task when there's a large number of overall lanes, as is the case with switch chips.
GETTING TESTY Two new types of test systems will be needed for PCI Express—one to address the analog aspect of the design, and the other to address the protocol. Agilent and Tektronix already offer tools in these realms.
Tektronix addresses signal testing with tools like its Data Timing Generators and RT-Eye Serial Data Compliance and Analysis Software. Many of these tools will be more familiar to RF designers, but now they must be incorporated into the typical digital board designer's toolkit.
Agilent's E2960A Protocol Analyzer and Protocol Exerciser for PCI Express looks at the interface at a higher level. Unfortunately, it means a design must incorporate the appropriate test points so that cabling can be attached. This becomes less of an issue when working with prototypes, though.
Still, even adding test points can affect the functionality of a PCI Express link. Therefore, it pays to do the testing as well as have in-house or solicited support for the design. Luckily, most designs will deal with PCI Express chips from vendors who already tested the protocol support. PCI-SIG regularly has "plugfests," in which chip and system vendors can demonstrate interoperability.
Tools like Altium's Protel 2004, Cadence's Allegro system, and others for pc-board layout already target high-speed differential designs. However, with PCI Express, the tools will require further tuning. For example, some PCI Express chips will autonegotiate pins as well as speeds. In some cases, the transmit pairs of two chips could be connected together. Then the chips would switch one set to the SERDES receiver within the chip with autonegotiation support.
It's a great feature, but pc-board layout tools can't handle it well when it comes to autorouting. Essentially, pin 1 and 2 on chip A can be connected to pins 5 and 8 on chip B, where the choice is left to the layout program or designer. It also complicates schematic capture. For now, pc-board layout programs will continue to have fixed pin usage.
High-speed test and analysis tools are embedded within pc-board design products, but many digital designers may be unfamiliar with these features. It's time to dust off the manuals and do some reading before diving into PCI Express designs.
Basically, the bottom line for developers is this: Get to know the design requirements and follow them religiously. High-speed serial systems are very unforgiving in some areas and extremely flexible in others. The differences in design, implementation, and testing are unlike anything encountered with a PCI design. Likewise, the performance jumps from ISA to PCI to PCI-X are trivial compared to the jump to PCI Express.
PCI Express board and system design will simplify as time goes on. Tools will improve to meet designer needs, and knowledge about PCI Express' idiosyncrasies as well as its design implementation will steadily expand. For now, most developers are crouched at the starting line. The benefits of PCI Express are significant, and it's the trend for new system designs, so don't get left in the dust. Prepare now and seek assistance.
Reprints
