Gigabit Serial Link Aggregation Combats Cable Overload
The use of cabling within electronic systems is widespread. For many applications, cable interconnect has provided a standardized way to connect two points within a system or between systems. Cable interconnects can be used with copper media to transmit data electronically, or optical fiber media can be used to transmit data optically. Advances in semiconductor signal integrity technologies such as transmit and receive equalization have increased the application base that can be served with copper and optical cable interconnects, both in terms of data rates and transmission distances.
Related Articles
- Category 8 Cable. Really?
- How To Determine When Shielding Network Cabling Is Required
- What's The Difference Between Cable And DSL Broadband Access?
Defining The Problem
Today, cable interconnects support data rates beyond 10 Gbits/s for copper media and much higher for optical media. However, most cable interconnects operate at much lower data rates. Common cable interconnects operate from hundreds of megabits per second to the 1-Gbit/s range. Communication and computing systems incorporate many of these cable links, giving rise to cable congestion within systems. As systems scale to support higher bandwidth levels, cable bundles within these systems become larger and much more difficult to implement, manage, and service.
This file type includes high resolution graphics and schematics when applicable.
One possible solution is to consolidate multiple cable links into one data pipe with a higher data rate that can be easily implemented, serviced, and managed. Until recently, consolidating multiple serial links meant having to develop a custom FPGA-based solution involving a costly development effort. Recent advances in serialzer/deserializer (SERDES) technologies now make the process of serial link aggregation much more approachable in terms of engineering and the overall cost of implementation. Using the same implementation approaches used in standards such as 10G Ethernet, PCI Express, and Serial Rapid I/O, silicon vendors now can bring robust serial link aggregation to standard product silicon implementations that system developers can easily integrate into their designs without having to re-architect their entire system.
The need for cable consolidation is no more apparent than in today’s data centers that drive the Internet and large cloud computing applications such as Web e-mail, video, and smartphone data traffic. Data centers commonly have an array of networking equipment including servers, switches, and data storage systems. Data center networking equipment is connected together in a logical fashion using many cables. The implementation cost and cable management within a data center often limit the scalability of a given data center site.
One approach to alleviate the issues associated with the large number of cable links within a data center is to strategically use serial link aggregation techniques to reduce the number of cables at critical points. One critical interface type that drives many cable links within a data center is the 1G Ethernet port, the main data interface on data center networking equipment. The 1G Ethernet interfaces are interconnected via copper category-5/6 cables or by optical fiber media. These cable links are used to connect one networking equipment rack to another or connect sections within a rack.
The Aggregation Solution
Let’s examine how serial link aggregation can be used to reduce the number of 1G Ethernet cable links that need to be implemented within a data center. Figure 1 shows a typical rack server with cable interconnects from one section of the rack to another. Implementation of the actual 1G Ethernet lanes can vary from system to system, but the end result is multiple lanes for 1G Ethernet data traffic.
One simple way to aggregate these links is to use available off-the-shelf SERDES interface devices, such as the Texas Instruments TLK10081 serial link aggregator, which enables aggregation of up to eight lanes of 1.25-Gbit/s (1G Ethernet) data traffic onto a single 10-Gbit/s link for transmission over short distance copper cable (up to 10 meters) or over optical links.
This aggregator can act as a full-duplex aggregator for eight lanes of 1G Ethernet. In our example, it accepts eight channels of 1G Ethernet in the transmit direction, as well as eight channels of 1G Ethernet in the receive direction. Gigabit Ethernet requires full duplex links, which complicates the aggregation function design for quick implementation within FPGAs. Figure 2 compares the signal flow of 1G Ethernet lanes within a system with and without the implementation of aggregation.
The functions performed by devices such as the TLK10081 involve the application of modern serial link technologies, such as wide bandwidth serdes, clock data recovery, transmit and receive adaptive equalization, smart lane switching, and intelligent auto link training. The actual aggregation function involves combining multiple serialized time synchronous sub-rate, lower-data-rate data streams into a full-rate, higher-data-rate serial data stream by interleaving the individual sub-rate input data streams in a specific way.
Implementation complexity arises when the multiple individual serialized data streams are interleaved so they can be de-interleaved on the receiving side of the link. To accomplish this process, appropriate serial stream encoding and decoding, as well as lane demarcation and alignment, have to be implemented within the aggregator device. This is both on the data origination side as well as on the termination side of the link. Figure 3 shows the generalized conceptual view of aggregation.
Intelligent aggregators, such as the TLK10081, can handle both raw, un-encoded data and data that has been coded using common serial encoding schemes such as 8-bit/10-bit coding. If the data is encoded, input data streams can be accepted in any order and in any number up to eight streams. For practical use cases, this means that an application can start up with four 1G Ethernet data streams being aggregated, and four more can be dynamically added later without having to break the original aggregated link. Additionally, serial link aggregators can be connected in different ways, enabling designers to construct different link topologies such as daisy-chained serial links (Fig. 4). The flexibility provided by aggregation devices opens up a vast array of serial link design possibilities that are truly easy to implement.
Conclusion
Using off-the-shelf serial link aggregation devices can significantly cut a designer’s development time and reduce the overall implementation risk of a design. Additionally, reducing the number of serial links can dramatically reduce power consumption and associated thermal management costs. These are two very important metrics associated with the operation of data centers as well as many other application spaces.
Application of serial link aggregation is likely to spread beyond traditional communication equipment into areas such as video surveillance, machine vision, medical imaging, and consumer electronics that support high-definition 4K signals and many other spaces that need to process large amounts of data. Over the next few years, serial link aggregation is likely to become one of the key design methods that high-performance system designers use to cost-effectively meet the increasing need to support higher data bandwidths within their applications.
References
For more information on SERDES, visit www.ti.com/serdes-ca.
Download these datasheets: www.ti.com/tlk10081-ca and www.ti.com/tlk100220-ca.
Atul Patel is new business development manager for Gigabit SERDES products within the Communication Interface Products Group at Texas Instruments. He has a bachelor of science degree in computer engineering as well as an MBA from the University of Central Florida. He can be reached at [email protected].