Understanding Gained One Layer at a Time

Protocols are sets of rules. In data and communications networks, they define how information must be organized for it to be correctly interpreted. There are thousands of protocols in use today, and they all have a life cycle.

Protocols evolve and change through the standardization process. In addition, protocols fragment, resulting in several dedicated protocols replacing a more general one that originally served a larger application space. Because of the continuous change occurring, flexibility is a key attribute of any protocol test solution.

PCI Express (PCIe) is a typical protocol that consists of several distinct layers: physical with logical sub-block, data link, and transaction. Each of the layers actually is a separate protocol within the overall PCIe protocol, and to analyze PCIe bus transactions, you need to consider each layer. The open systems interconnection (OSI) Seven-Layer Model defines a structured approach to creating protocols.

The PCIe physical layer is more complex than in many other protocols because the number of lanes being used and the speed can change dynamically. These kinds of changes are controlled by the logical sub-block within the physical layer.

Not all of the seven layers must explicitly be part of a protocol, only three being listed for PCIe. In this case, because there is no intention to specify how the information transported via PCIe is used, the application, presentation, and session OSI layers don't apply. Neither does the network layer because PCIe uses real, physical lanes not virtual paths within a broader network.

A general-purpose tool such as an oscilloscope traditionally has offered little help beyond the physical layer. Several scopes now include limited protocol analysis capability for a few popular serial data protocols such as I²C, SPI, UART, and RS-232. This capability decodes the meaning of the physical-layer signal and advances your understanding of the system behavior at least one or two layers up the protocol stack.

You can easily compare the real data values against those that were intended. Some scopes offer serial bus decoding and triggering as standard, and in others it's optional. If more complex protocol decoding is available, such as for CAN, FlexRay, and LIN, it usually is an extra cost option.

A protocol's physical layer specifies more than just signal characteristics. It also covers the hardware connectors and the medium used to carry the signals. Especially for very high-speed communications standards, care must be taken to preserve the signal integrity, which requires that special-purpose protocol analyzer front-end electronics, test connectors, and interposers often are used.

What Makes a Protocol Analyzer?

To analyze high-speed data buses and for complex applications with many different protocols, you really need a proper protocol analyzer. There are two very broad categories that describe all analyzers: software-only and hardware-based. Software-only protocol analyzers are very popular because of their low cost and ready availability. On the other hand, they tend to be limited to only those interfaces built into a PC, such as Ethernet and USB. You cannot buy a software-only analyzer for PCIe, for example.

LAN Analyzers
A ClearSight Networks white paper lists several problems that can occur when using an Ethernet software-only analyzer. It explains that when port mirroring is used to access network traffic, typically either one channel of a full-duplex link or only a half-duplex link is used. This means that the maximum data rate can be supported in one direction of a full-duplex link.

Or, equivalently, mirroring can only support full duplex operation at half the maximum rate. A full-duplex test access port (TAP) used with a full-duplex protocol analyzer avoids the dropped frames that can be caused by port mirroring restrictions.

Other limitations include possible missed packets if the network switch's CPU utilization is too high to consistently perform port mirroring. Frames with media access control (MAC) errors usually are discarded by the network switch so they cannot be passed on for analysis. And even if a standard network interface card (NIC) performs well, it will not provide the accurate hardware-based timestamping needed to time-align activity at different points in a network.¹

James Messer, director of technical marketing at NetScout Systems, commented, ” Capturing network traffic depends on the device where the software [protocol analyzer] resides. It's not uncommon to have two different laptop models with the same protocol analysis software display dramatic inconsistencies in capture performance. Hardware-based analyzers,” he clarified, “deliver an efficient and standardized traffic capture rate. In addition, they also provide features such as modular network interface options and stream-to-disk capture functionality.”

Most of the time, you might not have any problems using a software-only Ethernet analyzer. However, if you did encounter a marginal condition, could you determine that the signal was at fault or that the standard NIC had mishandled the data? Special-purpose test hardware guarantees 100% capture at full line rate, handles large amounts of data, and provides triggering and filtering.

The problems that commonly require LAN troubleshooting are very different from those typically encountered during product development (Figure 1). A statement in the Fluke Networks OptiView Network Analyzer technical datasheet makes this clear: “Management and users are constantly demanding new technologies, new services, and better performance, which inevitably require changing infrastructure, deploying new applications, and dealing with security.”

Figure 1. Network Utilization DisplayCourtesy of NetScout Systems

In the OptiView Analyzer and LAN analyzers made by ClearSight, NetScout, and GL Communications, for example, the emphasis is on the upper layers of the OSI model. Communications network operators need to understand how their network capacity is being used so they monitor, record, and analyze activity. Typically, Ethernet analyzers can distinguish among audio, video, image, and data traffic. This can be important because latency affects each type of traffic differently, causing some applications such as voice over IP (VoIP) to perform poorly but hardly noticed in others.

Although Ethernet is well understood, it doesn't follow that higher-level protocol analysis is simple. David Skingley, Fluke's product marketing manager for the OptiView product, explained, “At data centers and network cores, the design of the infrastructure with multiple paths between devices and redundant links makes it impossible for a software analyzer to capture the data because a packet can take any path. Therefore, the hardware appliance needs to have multiple, synchronized interfaces that connect to the mesh infrastructure to ensure that the packet stream is decoded in the correct order.”

An example of the innovative capabilities being incorporated in LAN analyzers is Fluke's Free String Match feature, which identifies and matches any set of words or phrases in network traffic regardless of its position in a packet. A Fluke customer described how he simply enters the error message that his terminal has received from the server, triggers on it, and captures traffic ahead of and after the event. With a detailed description of events leading up to and following the error event, the cause soon is identified.

The protocol analyzers made by GL Communications handle wireless and wired T1, E1, and Ethernet networks with as many as 50 separate protocols for each of data, voice, and video traffic. One hardware platform suffices with specific software for the various protocols. This approach makes it possible to troubleshoot problems involving both the LAN IP network and the public switched telephone network.

NetScout's nGenius® Network Monitor is optimized to work with the protocols encountered in virtual private networks (VPN). A VPN is deployed on public networks but, because of the tunneling protocols used, appears much like part of an enterprise's private network. VPNs combine tunneling protocols such as generic routing encapsulation along with encryption technologies and are delivered as a layer 2 or layer 3 service.

High-Speed Analyzers
Similar to the hardware approach used by some of the communications network analyzers and monitors, it is common for high-speed data protocol analyzers to handle groups of similar, related protocols. A good example is the Finisar Xgig® Test Platform that supports specific solutions for 10-Gb/s Fibre Channel and Ethernet, 6-Gb/s SAS and SATA, and iSCSI applications. The platform provides monitoring, analysis, and testing capabilities such as BERT, jammer, and data generation for these storage device protocols.

The identical hardware is used with protocol-specific software and features comprehensive triggering, data tapping with minimal signal integrity impact, and a trace buffer up to 16 GB. Using the same hardware in a multiprotocol environment results in greater instrument utilization as well as reduced development time because interactions among the various protocols are easily correlated.

LeCroy has taken a somewhat more general approach in its 10K mainframe analyzer platform that accepts a range of personality modules or blades. Currently, PCIe, SAS, SATA, and Fibre Channel protocols are supported.

Mike Micheletti, a product manager at the company, commented, “Only dedicated protocol analyzers that use special analog front-end (AFE) circuitry can provide lossless capture with sufficient granularity to resolve link layer issues. These AFEs include programmable SERDES that are carefully calibrated to tolerate features like spread-spectrum clocking, out-of-band signals, or power-management behaviors.

“Because of the very high-speed networks used today, tens of millions of messages can be exchanged in minutes. This makes it necessary to trigger on errors in real time, but the process is complicated,” he explained. “Serial triggers often rely on pattern matching, which only can be performed after symbol alignment, descrambling, and packet delimiter decoding have been accomplished. This requires the fastest hardware and may not be supported at all in general-purpose instruments because of the wide range of encoding schemes used in serial protocols.”

Agilent Technologies manufactures a number of protocol analyzers including one for PCIe that handles both the 5-Gb/s generation 2 and 2.5-Gb/s generation 1 implementations through a software upgrade. Yenyi Fu, a product manager at the company, explained that this is one way a protocol analyzer can help customers by following changes in technology.

“Another example is the capability to move from a point tool to a system-wide view depending on where the problems lie,” she continued. “The PCIe tool typically is used as a point solution. However, there are cases where PCIe activity needs to be monitored in correlation to a memory bus or a network protocol such as Fibre Channel. The point solution can easily be connected to a logic analysis tool or another protocol analyzer to provide a fully correlated system view.”

Additional value is provided through compliance test capabilities for certain protocols. For PCIe, Agilent's Protocol Test Card (PTC) stimulus tool generates more than 180 PCI SIG-recommended tests to actively validate DUT behavior. When the PTC is integrated with the PCIe analyzer, failed responses to PTC tests can be captured for further debugging. The combination of stimulus compliance testing and protocol analyzer debugging helps identify potential interoperability issues faster than traditional methods.

Although there are thousands of protocols, a designer will work with only a few during any one project. So, at least for the project duration, he will be working with PCIe, Ethernet, and USB, for example. This means that the interactions among these buses will be of interest, and a means must be provided to track data as it moves among them. Test solutions that integrate simultaneous analysis of several buses maintain time-alignment, facilitating root-cause analysis. In cases where parallel data is involved, combining a serial protocol analyzer with a logic analyzer can be a good solution.

Logic Analyzer Pre-Processor
There are different types of tools available. One is based on a mainframe logic analyzer with special pre-processor adaptors for each bus being probed. Sometimes, these adaptors are separate, such as the range of FuturePlus Systems pre-processors that are used with a Tektronix TLA7000 Series Logic Analyzer. These are small instruments that handle Fibre Channel, PCIe, HyperTransport™, Serial RapidIO®, DDR Memory, PCI-X®, and USB buses (Figure 2).

In addition, a Tektronix serial acquisition module directly plugs into the company's TLA7000 Series Logic Analyzers. With this module, parallel and serial data together with time-aligned oscilloscope waveforms can be displayed and analyzed.

Agilent also offers a wide range of analysis probes, extenders, and interposers for the company's 16700 and 16900 Series Logic Analyzers. Some of these are made by Advanced Logical Design and others by FuturePlus. The buses and protocols addressed include CAN, SPI, RS-232, SATA, SAS, PCIe, FibreChannel, Serial Rapid IO, and HyperTransport.

Logic analyzer manufacturers claim with some justification that a logic analyzer is a more general-purpose tool than a protocol analyzer. If you have only one or two special serial buses to deal with as well as parallel data within a system, a logic analyzer could be the tool of choice. Especially if you already have a logic analyzer and are familiar with its use, adding pre-processors as required could be an economical way to proceed without encountering the learning curve of a new instrument.

On the other hand, a dedicated protocol analyzer can be a good investment and may better suit your company's needs. According to LeCroy's Mr. Micheletti, “Modern protocol analyzers have ease of use advantages when compared to logic analyzers. Dedicated analyzers support comprehensive software decodes to provide better abstraction of link behavior. These systems include mature software tools that allow hiding or filtering of redundant symbols and automatically can flag protocol violations and generate reports and statistics. In terms of setup and data presentation, protocol analyzers require less technical expertise to operate.”

Summary

At least three or four types of test instruments can analyze data and communications network protocols. Network monitors and communications network protocol analyzers focus on the upper layers of the OSI protocol stack and provide a wealth of statistics about network performance. To do so, they must have comprehensive hardware at the physical layer with emphasis on large amounts of storage and full-rate capture capabilities.

To develop and troubleshoot devices that interoperate on a communications, storage, or data network, the lower few protocol stack layers typically are more important, even down to bit timing and signal integrity. Dedicated protocol analyzers manufactured by test and measurement companies and appropriately configured logic analyzers are the tools of choice for this type of work.

So, the first step in determining which protocol analyzer to buy is to determine the types of problems you need to solve. If troubleshooting means determining why certain application programs run more slowly in the afternoon than early morning, then you need a network monitor or communications network analyzer.

If you're developing a device that must conform to several protocols or troubleshooting a system that uses several, a dedicated protocol analyzer or logic analyzer is a good choice. These instruments are much more powerful when lower-level issues are involved. You may need to simultaneously timestamp events on several types of buses to determine the cause of a malfunction. These instruments are particularly appropriate for cross-bus analysis.

Bill Furch, senior applications engineer at FuturePlus, summarized the situation: “Our products analyze the incoming protocol in real time while connected to the target system. With this method, the user sees any protocol failures that may be caused by signal integrity issues. The logic analyzer connected to the protocol analyzer then can be used to trigger a high-speed oscilloscope to investigate the signal integrity problem. A software-only protocol analyzer normally is more useful to a software designer while the logic analyzer/protocol analyzer combination better suits a hardware designer or system integrator familiar with cross-domain analysis.”

Reference

1. Mitchell, S. H., “Why You Should Consider a Hardware Based Protocol Analyzer,” ClearSight Networks, White Paper, October 2006.