[Leapfrog: First Look]
Aggregating Ethernet Ports Slashes System Cost
Today's Ethernet network connections typically operate at under 50% of the available bandwidth utilization due to many factors, but especially the bursty nature of the data being sent.
Today's Ethernet network connections typically operate at under 50% of the available bandwidth utilization due to many factors, but especially the bursty nature of the data being sent. However, equipment aggregating Ethernet links, like switches and routers, are usually designed for 100% link utilization. This results in the underutilization of bandwidth, increased equipment cost, and a reduced number of customers served per system.
To maximize revenues per system, an oversubscription-based solution can be used. In this approach, unused bandwidth can be eliminated at the physical connection so the data stream can fully utilize shared hardware, such as the network processors, the backplane, and the switch fabric.
Ample Communications has developed such a solution. Its Harrier family of port aggregation chips provides intelligent oversubscription support. These chips can aggregate up to 24 Gbits of bandwidth from customers to a highly utilized 10-Gbit/s system datapath.
By using weighted-random-early-detection (WRED) and modified-deficit round-robin (MDRR) based algorithms, along with in-band PAUSE frame generation, the scheme can minimize frame drops and ensure that lower-priority and most offending customers are penalized in times of traffic congestion. The resulting system-level solution reduces hardware cost per customer by up to 40% and lets system vendors double the number of customers served per system, lowering the cost per customer.
Ample's oversubscription scheme uses intelligent buffering. It enables the aggregation of up to 24 1-Gbit/s ports into a single 10-Gbit/s channel, reducing the number of 10-Gbit/s interfaces by almost 2.5 times. The approach leverages the fact that data tends to be fairly "bursty," with large idle periods separating streams of bits.
By using that aspect and implementing large on-chip buffers and intelligent buffering algorithms, Ample crafted its highly integrated A2510 Harrier chip. It greatly reduces the number of components required to create an Ethernet solution with oversubscription and billing support. The chip also includes protection switching to better implement nonstop systems. The A2510 aggregates 24 Ethernet ports on the line side, with each port able to handle 10/100/1000-Mbit/s Ethernet traffic. On the host side, the chip implements a 10-Gbit/s SPI-4.2 interface (see the figure). The system-peripheral-interface (SPI) port also will operate at quarter speed.
Each Ethernet port supports industry-standard RMII/RGMII version 1.3 interfaces that connect to readily available Ethernet physical-layer devices. For the 10/100-Mbit/s mode, the interfaces can be programmed to operate in the RMII mode to take advantage of the low-cost physical-layer chips. Each line-side port can be independently set to run at 10 or 100 Mbits/s, or 1 Gbit/s.
On the system side, the chip's standard optical-internetworking-forum (OIF) system packet interface level 4, phase 2 (SPI-4.2) port can be configured to run at full speed (311 to 400 MHz) or at quarter speed (77.7 to 100 MHz) for a total data throughput of up to 12.8 Gbits/s and 2.5 Gbits/s, respectively. The company will also offer two other versions of the Harrier chip. One is a 12-port line aggregation chip (the A2511). The other has 24 Ethernet ports, which are limited to 10- or 100-Mbit/s throughputs (the A2512).
Each chip includes an extensive set of statistics counters on both the transmit and receive sections for billing applications. These counters collect statistics on different types of frames sent, received, and dropped, as well as CRC errors, jabbers, byte counts, and aborts. Designers also packed enough on-chip memory (over half a megabyte) to support transmit, receive queues, and FIFO buffers so each port can handle the Ethernet jumbo frames of up to 9.6 kbytes without any external memory. On-chip media-access controllers (MACs) can also generate and process in-band PAUSE frames for flow control.
An on-chip Ethernet management link (EML) pass-through permits flexible management and redundancy protection of the Ethernet ports. When deploying this feature, Ethernet frames are optionally forwarded over the SPI-4.2 interface with the first 8 bytes designated as preamble and the SFD bytes unchanged. Systems can then perform operation, administration, and management functions based on information embedded in these bytes. Also, the A2510 Harrier chip's internal logic helps implement 1+1 or 1:N protection switching between independent line ports.
The chip's ability to oversubscribe the Ethernet ports starts with a pool of queues associated with the 24 Ethernet ports. The queues are allocated dynamically in page increments and can grow to support multiple jumbo frames. Once a port is serviced, the corresponding queues are de-allocated and go back into the available pool. To avoid congestion, a flexible layer 2 WRED-based scheme is used to limit incoming data rate. In this approach, frames are dropped with some probability if a set threshold is exceeded. Anticipating congestion and dropping frames early prevents congestion due to bursty traffic.
Reprints
