Board and system interconnects have migrated
from parallel buses to high-speed serial technologies.
Mainstays like Peripheral Component Interconnect (PCI),
Integrated Device Electronics (IDE), and Small Computer
Serial Interface (SCSI) remain. Yet new systems are more
likely to use PCI Express (PCIe), Universal Serial Bus
(USB), Serial ATA (SATA), or Serial Attached SCSI (SAS).
Many standards are available, but they tend to be complementary (). A typical system can use two
or more standards (). For example, HyperTransport may be used to connect CPUs together while
PCI Express is used with peripheral interfaces. Storage is
typically linked via SATA or SAS.
The interconnect taxonomy can get rather complicated
because of the different approaches designers use. Most
standards incorporate a range of speeds, while others are
scalable in width. Packet size, protocols, and latency also
come into play. Some, like PCI Express and SAS, are based
on the older PCI and SCSI standards.
While this provided a clear
upgrade path, the main driving factor
for the new interconnects was more
performance. One way to get that is
to move to high-speed serial transfers. The other is to scale the width of
the interconnects.
SCALABLE-WIDTH
INTERCONNECTS
HyperTransport, PCI Express,
Serial RapidIO (SRIO), and InfiniBand can use different widths
that normally range from 1 to 32
lanes, based on the technology.
The source and destination handle conversion to and from a parallel data stream, such as 32-bit
words into a stream of packets.
They all can use the same type
of serializer/deserializer
(SERDES), but the handshaking,
timing, and protocols differ
between standards.
All of the high-speed serial
interfaces transmit data using
differential pairs, doubling the
number of wires per signal.
However, it also significantly increases noise immunity.
This is critical at the RF gigahertz range where these
interfaces run. It's a quantum leap from the slower, parallel bus speeds of yore, but standard design rules and
enhanced EDA tools are addressing design issues.
The major increase in speed allows a single serial lane to
exceed the speed of a much wider parallel bus. For
instance, a x1 lane PCI Express connection offers more
throughput than a parallel PCI bus. New standards with
higher-speed signaling provide corresponding throughput
increases without a change in the number of connections.
HyperTransport (HT) is the odd man out when it comes to
clocking. It uses an explicit clock (or clocks), which simplifies implementation and increases overall throughput but
at the expense of more lines. Likewise, HyperTransport is
normally implemented in eight- or 16-lane widths.
PCI Express, Serial RapidIO, and InfiniBand use independent serial lanes with embedded 8b/10b encoding (8 data bits/10 signal bits). This 20% throughput
penalty is made up by running the serial links at very
high speeds. Making the lanes independent means that
connections can be more robust versus a parallel
scheme where all signals are in sync with each other.
The difference becomes more significant as speed and
distance increase.
The SERDES can easily handle synchronization with
respect to their own lane of traffic, and data streams from multiple lanes are synchronized and merged within the
receiver. This makes the SERDES, transmitters, and
receivers critical and complex design tasks. Yet users of
the technology simply deal with the data streams at the
extreme endpoints.
One commonality among these interfaces is their full-duplex operation. At a low level, the systems are implemented as independent transmitter/receiver pairs. Higher-level
protocols built on this hardware perform handshaking.
SERIAL NETWORK
Ethernet
fits in a category by itself for a variety
of reasons, but the SERDES used by
Gigabit Ethernet and other standards are the same found in the prior
standards. The big difference is that
Ethernet comes in fixed flavors and
cannot be incorporated incrementally like PCI Express or Serial RapidIO.
Still, Ethernet will be found inside
the box connecting boards together
like InfiniBand and Serial RapidIO.
These two standards have lower
overhead and latency compared to
Ethernet, but Ethernet has familiarity
pushing it. Standards for backplanes
like AdvancedTCA included all the
high-speed serial standards, though
Gigabit Ethernet was the first to be
implemented.
Ethernet often is integrated directly inside microcontrollers, with PCI
Express being a common interface to
standalone chips and controllers.
Transmission Control Protocol/Internet Protocol (TCP/IP) is often the
high-level protocol riding atop Ethernet. This enables Ethernet to coexist
nicely with InfiniBand and RapidIO,
both of which support TCP/IP as well.
The Ethernet family has a wide
range of implementations, possibly
more than any other interconnect
standard. Media and speed tend to
separate these standards (see
"Speed Shifting").