Everyone uses it. You can’t escape it. It affects you every day. And, its impact on your life is expanding. I couldn’t help but get that feeling as I listened to the many sessions at the Ethernet Technology Summit at the Santa Clara Marriott, Feb. 23-24. Ethernet truly is the ubiquitous networking technology, and this conference provided a good look at the current industry status as well as what is coming in the near future.
40/100 Gigabit Ethernet
Much of the conference was devoted to the most recent Ethernet standard, 802.3ba, which defines 40 Gigabit Ethernet (40GE) and 100 Gigabit Ethernet (100GE) versions for local-area networking (LAN) aggregation and wide-area networking (WAN) end points. That IEEE standard was ratified in June 2010, and equipment is now becoming available.
This was the first time that the IEEE has defined a speed version that was not a decade multiple of the previous standard. One Gigabit Ethernet (1GE) jumped to 10GE and now 100G. The 40G version was defined as an intermediate standard to accommodate the current state of the technology as well as to bless a version that would have higher-volume equipment potential than the lofty, low-volume 100G hardware.
Some of the major trends driving the adoption of 40/100G are virtualization of servers in the data center and cloud computing. The increased movement of video over the Internet and other networks is also having a major impact. More and faster mobile data networks are also pushing carriers to faster network connections.
The 802.3ba standard defined a wide range of versions from a 40G 1-m backplane to 7-m 40/100G copper cables and fiber versions of 100 m on OM3 multimode fiber (MMF), 125 m on OM4 MMF, and 10 km and 40 km on single-mode fiber (SMF).
Some of the common interfaces for the 40/100GE systems are the CFP (C form-factor pluggable, where C means 100) and QSFP (quad small form-factor pluggable). These are multi-source agreements (MSAs) or standards made by competing optical module vendors. The lowest-cost CFP is a unit using ten 10GE lanes of fiber. Another CFP module is four fibers carrying 25 Gbits/s. A QSFP module using four lanes or wavelengths of fiber is also available.
With the Internet doubling about every 18 months, the pressure is on to carry more data sooner. 40/100G fiber systems are the way to go, but prices still need to come down before widespread adoption. A typical 100GE port today sells in the $250,000 to $500,000 range, which is too high for all but the most demanding customers.
What About 10GE?
Ten Gigabit Ethernet was also mentioned repeatedly during the conference. Even though it is considered “old news” in some quarters, it is actually a work in progress. While some data centers have already added 10GE equipment, most are still transitioning: 1GE plants with 10GE being incorporated as needed. One estimate put 10GE penetration in the data center at only 15%. The general consensus is that 10GE will really take off when we begin to see 10GE LAN on motherboard (LOM) chips on blades and in other servers.
The really big conversion to 10GE will occur when more 10GBaseT products arrive in the form of network interface cards (NICs) and LOM. This 10GE copper version is still an order of magnitude cheaper than optical fiber versions. I saw several 10GBaseT NICs from Broadcom, QLogic, and others on exhibit.
Fibre Channel Over Ethernet
This conference usually includes a look at Fibre Channel (FC), the popular storage-area network (SAN) technology. FC is the most popular SAN technology in the enterprise and other large organizations. The other SAN technology of note is iSCSI (“I skuzzy”), which uses the legacy SCSI storage protocol sent in TCP/IP packets over Ethernet. It is a more affordable SAN technology that has found its way into smaller companies and businesses thanks to the convenience and lower cost of standard Ethernet products.
While some have predicted the demise of FC in favor of the faster Ethernet versions, FC seems to be alive and well. It currently offers data speeds to 8 Gbits/s, and the next-generation 16-Gbit/s FC is forthcoming. Big datacenters still appear to favor separate networks for data and storage; FC fulfills that preference.
One trend that is growing is FC over Ethernet (FCoE), in which the Fibre Channel protocol is encapsulated in Ethernet. It provides a way to maintain the FC protocol but also to take advantage of the faster 10G and 40/100G Ethernet speeds. FCoE allows some convergence of the LAN and SAN. Special interfaces called converged network adapters (CNAs), featuring Ethernet NIC and FC host bus adapters (HBA), facilitate the alliance. The general consensus is that FC will continue to grow and thrive, at least for a while longer.
The Future
There are many new Ethernet developments to be absorbed before the next big wave of change crests. Datacenters are at different places in their technological upgrade cycles, so the rollout of 10G, 40G, and 100G will continue over the years to come. The scarcity of 40G and 100G boxes and equipment will contribute to delays in adoption. Cost is also a major factor in deploying the faster equipment, but volume will eventually take care of that.
On the other hand, there are some power users of Ethernet. Datacenters for Google, Facebook, Yahoo, Amazon, and others are rapid adopters of the faster equipment. These companies want even faster systems as soon as possible. The standards groups are already forming, and the next generation is being defined. Think 1 Terabit Ethernet, with the requisite 400G Ethernet in the interim. The challenges are overwhelming when you consider 40 25G paths or some other configuration. And here’s something to look forward to: no copper version.
