Do you know where your traffic is? Serial protocol analyzers easily locate bits, bytes, packets, and headers.
To win a trade-magazine design challenge, Ted needed to list 15 high-speed buses in order of popularity. He had just started when Joanne dropped by. She explained that specific application requirements, speed, compatibility with legacy systems, shared or switched access, serial vs. parallel architecture, scalability, breadth of industry support, standardization status, and cost all contribute to bus adoption. Ted made another pot of coffee.
The recently introduced PCI Express bus satisfies most of Joanne�s concerns as a successor to the decade-old peripheral component interconnect (PCI) bus. The PCI bus originally was used to interconnect chips on PCBs and appeared as a PC bus about 1994. It is a synchronous, 32-b, multiplexed parallel bus with accompanying control, arbitration, and interrupt signals.
More recently, PCs have implemented the optional 64-b addressing and data-transfer capability, adding more connector pins to do so. Speed has reached 66 MHz, giving a maximum throughput of 528 MB/s, up from the 132 MB/s maximum rate for a 32-b, 33-MHz system. During the time these changes were occurring, digital chips moved from 5 V to 3.3 V, and 133 MHz was reached via PCI-X technology. Nevertheless, it was obvious that parallel buses were reaching their performance limits.
PCI Express replaces the parallel physical-layer bottleneck with a switched full-duplex serial link. Initially, the separate differential current-mode logic (CML) links in each direction have been specified to operate at a 2.5-GHz rate, corresponding to 250 MB/s when the 8b/10b data encoding is accounted for. Throughput is scaleable by ad-ding more traffic lanes, with �32 being the current maximum giving an 8-GB/s rate in each direction. A 5-GHz speed is planned for the future.
In its basic form, PCI Express addresses high-bandwidth applications such as storage devices and servers, communications, and embedded control. All data is transmitted in-band, compared with PCI that uses sideband signals to handle interrupts and power management requests. In PCI Express, actual payload data is termed transaction layer packet (TLP) and management data as data link layer packet (DLLP). Both data packet types are dispersed across the lanes.
Management data includes the older PCI functions as well as new PCI Express flow control and packet acknowledgement (ACK/NACK). Flow control ensures that if a receiving device cannot accept a new packet, it won�t be sent. This approach is intended to reduce errors and subsequent retries, improving throughput. After a packet has been sent, its correct reception is signaled by ACK or if incorrect by NACK.
Data bytes are formed into packets by the transaction layer and a header added. Three PCI addressing spaces, memory, I/O, and configuration, are preserved in PCI Express and a fourth message space added. It stems from the message signaled interrupt (MSI) alternate method of propagating system interrupts introduced in PCI version 2.2 and has been extended to include all PCI sideband signals.
The link layer adds a sequence number and cyclic redundancy check (CRC) characters. Should an error occur during packet reception, the link layer controls the retry mechanism. Finally, the physical layer assigns bytes to lanes for transmission, and within each lane 8-b to 10-b encoding takes place as well as parallel to serial translation.
Because the clock is embedded, the process of acquiring and interpreting this serial data is closer to the operation of a serial communications analyzer than to a logic analyzer. Data recovery is further complicated by the distributed nature of the packets. Before analysis can begin, the lane corresponding to the start of a packet must be determined and the data bits and bytes reorganized according to the original transmission scheme.
The really clever thing about PCI Express is that to higher level layers the transaction layer appears identical to the PCI transaction layer. This means that legacy software applications can�t tell the difference between PCI and PCI Express, and the user�s software investment is protected.
PCI Express Analyzers
Like analyzers for other protocols, PCI Express analyzers must receive the signal, decode it, filter the data to accept only those packets of interest, trigger on a complex mix of conditions, and provide a comprehensive search utility as well as a complete and detailed display of the acquired information. In addition, complementary exerciser products are available that generate PCI Express test data, with or without errors, to drive your new device or system.
Analyzers and exercisers from Catalyst, Agilent Technologies, Finisar, LeCroy, and Tektronix are presented and discussed.
Catalyst
Twelve versions of the SPX Series Exerciser and Analyzer address PCI Express protocol debug on one, four, eight, or 16 lanes. The exerciser with data capture or just the analyzer can be purchased separately, or both the analyzer and exerciser are available as a bundled system. A range of adapters, extenders, and probes completes the product offering.
The exerciser operates as an endpoint or as a platform and includes two levels of pattern-generation capability together with an arbitrary waveform generator-like looping feature. The user can elect to manipulate the physical and data link layers directly, or the exerciser�s built-in automatic logic will manage the majority of the layer-to-layer data interchange with only limited user control possible.
A wide range of error types can be inserted into the traffic stream. By mimicking bus problems in a stable, repeatable way, underlying problems can be approached from a variety of directions and eventually pinpointed. For example, inserting delay into one or more lanes creates a data skew across the overall link, which can cause data to arrive out of order. Similarly, always responding NACK stresses the data link layer retry mechanism.
Predefined filtering addresses the most common test scenarios and serves as a good starting point to develop more specific customized filters. You can elect to exclude DLLPs or TLP data payloads, for example, or to accept only DLLPs, TLPs, messages, or flow control credits or to filter nothing. Three built-in filter selections can be combined in both directions and additional parameters applied to each filter.
Triggering also is approached via predefined categories that can be further refined. Events such as the recognition of symbols, DLLPs, TLPs, messages, or protocol errors can cause a trigger, or a sequence of such events can be defined that results in a final trigger.
Protocol analyzers are not yet capable of autonomously determining the necessary corrections to all the problems they discover. That�s the operator�s job, aided by the information displayed on the controlling PC�s screen.
Through the use of a structured screen layout, colors, and the capability to drill down to several related, more-detailed screens, a great amount of information is simultaneously presented to the user. Data can be viewed across all protocol layers or only one; related packets grouped; packet fields and symbols decoded; and a variety of data formats, timing capabilities, and display attributes made available to improve troubleshooting efficiency (Figure 1).
Figure 1. Display Featuring Transaction Timing, Direction, and TypeCourtesy of Catalyst
The SPX instrument provides a standard component object model (COM) application program interface (API) that supports automation via scripting, bypassing the graphical user interface (GUI). In addition, a set of SpekCheck� Compliance Tests is included that implements tests from the PCI Special Interest Group (PCI-SIG�) compliance specification check lists.
Agilent
The PC-controlled Agilent Serial Protocol Tester has a mainframe with either two or four slots for protocol-specific serial I/O modules. The Model E2960A Protocol Analyzer is a PCI Express version of this instrument populated with E2960A-E01/04/08 Exerciser and E2960A-A01/04/08 Analyzer modules. There are three versions capable of handling one, four, or eight PCI Express lanes.
The exerciser can generate and respond to transactions, insert errors to test corner cases, and monitor the system response to these errors (Figure 2). In particular, the exerciser emulates the configuration space of a switch and acts as an end-node for platform testing or a root-complex for add-in card testing. Access to the system under test for both the exerciser and analyzer is via a plug-in slot/interposer probe board or a connectorless midbus probe.
Figure 2. Multiwindow Display Showing Decode of Selected PacketsCourtesy of Agilent Technologies
The analyzer is nonintrusive, provides up to 1-GB of captured data, and acquires both TLP and DLLP transactions. It operates via a GUI with trigger, search, and filter capabilities. The exerciser can be controlled through the command application programming interface (CAPI), the tool command language (TCL) interface, or a GUI.
The PC-controlled Serial Analyzer platform supports cross-protocol exercise and analysis of systems using combinations of PCI Express, fibre channel, and advanced switching interconnect (ASI) protocols. The enhanced E2960A instrument handles 1-, 2-, and 4-Gb/s fibre-channel and PCI Express systems. The E2980A ASI Exerciser/Analyzer inherently caters for almost unlimited mixtures of protocols because the ASI protocol supports protocol encapsulation. Virtually any protocol can be embedded within ASI, although ASI itself is built on the PCI Express specification.
There are other PCI Express tools. If you already have an Agilent 16700 or 16900 Logic Analyzer, the Model N4220B Packet Analysis Probe can capture �1, �2, �4, �8, or �16 PCI Express transactions. You don�t have the capability to exercise or emulate parts of a system with a logic analyzer, but you do get cross-bus correlated analysis and sequenced event triggering across multiple buses, including processor systems, memory, serial ATA, Infiniband, and AGP.
For the final development stage when formal proof of compliance is required, the E2969A Protocol Test Card can save a great deal of time and expense. Because the card was developed jointly by Agilent and Intel and is continuously updated to the latest PCI-SIG specifications, passing the card�s preprogrammed tests ensures compliance. In addition, it can be used in conjunction with the E2960A Exerciser for more thorough protocol examination.
Finisar
In 2004, Finisar acquired the assets of Data Transit including the Bus Doctor Protocol Analyzer product line. Fifteen types of bus pods are available for the Bus Doctor and Bus Doctor Rx Analyzers. Computer as well as LAN and storage applications are represented by ATA, SAS, SCSII-320, GbE, USB, PCI-X, PCI Express, IEEE 1394a, and PCMCIA protocols.
The Bus Doctor is a PC-controlled analyzer capable of dealing with a single protocol determined by the type of attached pod. In contrast, a single Bus Doctor Rx also handles only one protocol, but up to eight of these instruments can be interconnected to capture and analyze traffic from the same or a mixture of bus types. In addition, the analyzer also accepts inputs from a logic pod, allowing protocol transaction timing to be compared with related logic events. Multiple analyzers share a common clock and can trigger across buses.
A histogram display presents an overview of the entire captured data trace with commands, data, and errors shown in blue, green, and red, respectively. Drilling down is done at the command level, in a state display with individual byte granularity, or via a timing waveform display with details of physical-layer signal activity.
Real-time statistical analysis is supported by dedicated hardware event counters. A real-time statistics menu presents peak and average values of selected parameters such as transfers per second, data transfers per error, or bit error rate measurements (Figure 3).
Figure 3. Real-Time Statistics Screen With Separate Parameter DisplaysCourtesy of Finisar
There are 36, 108, and 252 memory channels available in three different Bus Doctor Rx Analyzer models. In addition, 16-M, 64-M, or 256-M event acquisition memory length can be purchased corresponding to the manufacturer�s product configuration designations of slim, fit, and buff.
LeCroy
LeCroy acquired Computer Access Technology (CATC) in 2004. The 16-lane PETrainer EML PCI Express Exerciser and the complementary 16-lane PETracer EML Protocol Analyzer are built on the CATC� Universal Protocol Analyzer System 100K platform. Both instruments also include the CATC Trace� Expert User System Software with embedded details of the PCI Express protocol.
The PETrainer EML Exerciser features script-level traffic generation, platform or end-point emulation, and programmable data link and transaction layers. Having programmable layers means that the user can define an arbitrary sequence of transactions as well as modify flow control, ACK/NACK, and retry behavior. Any packet fields not specified by the user are generated automatically.
With 8 GB of memory, the PETracer EML Analyzer supports long recording sessions although hardware filtering allows you to exclude unnecessary data in real time. In addition, multilayer trigger conditions can be developed by cascading individual trigger definitions. The programmable trigger position displays variable amounts of data ahead and after the trigger to aid cause/effect diagnosis.
Tektronix
The Tektronix TMS817 and TMS818 PCI Express Bus Support products include the hardware preprocessing necessary to deserialize, descramble, and decode PCI Express signals before they can be input to a logic analyzer. Although it is a good tool to use for bus analysis, no general-purpose logic analyzer has the necessary capture clock rate nor do its inputs electrically and physically match the serial bus signals.
A preprocessor accepts the PCI Express 2.5-GHz data rate on the required number of lanes at the proper voltage levels and translates the serial bus to a number of parallel signals to match the logic analyzer�s input structure. The preprocessor also handles real-time filtering to record only the data of interest.
The TMS817 can interface to one to 16 lanes. The TMS818 is intended for applications having four or fewer lanes. With either preprocessor, one or more midbus probes or slot-style interposers are required to make the physical connections.
Multibus acquisition is supported across a wide range of processor, memory, and general-purpose bus types through preprocessors available from Tektronix and partners New Wave and Nexus Technology. Very large systems of more than 8,000 channels and 60 independent buses can be addressed via the TLA700 Logic Analyzer with up to 10 extension chassis.
Summary
The good news is that there are several products with which to address PCI Express bus exercising and analysis. The bad news is that there aren�t more although the limited number isn�t too surprising.
PCI Express is relatively new and physically very different from the previous PCI bus. Some of the companies supplying PCI bus instruments haven�t yet moved on to PCI Express. Also, for engineers using PCI Express in complex system designs, it is important to understand not only what�s happening on this bus but also on many others at the same time.
This requirement implies the need for a wide variety of bus-specific adaptors or, to use the Tektronix nomenclature, preprocessors. Agilent and Tektronix have the product depth to handle applications with a single PCI Express bus or with several different buses. Few other test and measurement companies do.
On the other hand, should you infrequently need multibus capabilities, perhaps the flexibility of interconnecting Bus Doctor Rx Analyzers may appeal. And, if you are working with only the PCI Express protocol, the CATC and Catalyst products offer attractive capabilities.
Before buying an analyzer, arrange to borrow it for a few days to make sure that it really is the best tool for the job. Also, when comparing products prior to making a final choice, be sure to ask about future upgrade paths. You would like to avoid going through yet another instrument evaluation exercise when 5-GHz PCI Express becomes available.
FOR MORE INFORMATION
on the Catalyst SPX Series
Exerciser and Analyzer
www.rsleads.com/506ee-185
on the Agilent E2960A
Protocol Analyzer
www.rsleads.com/506ee-186
on the Agilent N4220B
Packet Analysis Probe
www.rsleads.com/506ee-187
on the Finisar Bus Doctor Rx Protocol Analyzers
www.rsleads.com/506ee-188
on the LeCroy PETracer
EML Analyzer
www.rsleads.com/506ee-189
on the Tektronix TMS817
and TMS818 PCI Express
Bus Support products
www.rsleads.com/506ee-190
June 2005