Ethernet continues to be a defining influence in nearly every aspect of networking and communication systems even though it is more than 40 years old. Originally conceived as a simple, inexpensive local-area network (LAN) technology, it’s now used throughout most of the wide-area networks (WANs) responsible for delivering the world’s telecom, wireless, and Internet traffic. Its low cost and high performance also have made it a popular medium in many emerging applications, such as storage-area networks and video transport. Next, it will play a big role in the emerging Internet of Things (IoT).
Ethernet’s Odyssey
Originally created in 1973 by the legendary PARC research labs at Xerox Corp., Ethernet used a shared coaxial cable to pass its packet-based messages between computers at 1 Mbit/s (Fig. 1).1 Over the next 30 years, it underwent several evolutionary cycles that allowed it to use several other types of copper-based and fiber media. The new media, coupled with advances in transceiver technology, produced dramatic gains in its channel capacity and similar reductions in deployment costs.
Towards the close of the 20th century, Ethernet’s standards gurus—the Metro Ethernet Forum (MEF) and the IEEE 802.3 committee—began the next series of changes that would enable it to operate seamlessly across WANs. Today, it’s the medium of choice for nearly everyone from carriers and enterprises to residential customers.
Accelerating the adoption of IP-based (Internet protocol) Ethernet infrastructures are two of the most significant trends occurring within the communications industry: the migration from fixed wire lines to wireless networks, and the steadily growing demand for “anything as a service” (XaaS), also known as cloud-based services.
Mobile and cloud networking services are largely responsible for the exponential growth in demand for global IP traffic capacity, anticipated by Cisco to reach 1.4 zettabytes (1 billion terabytes) in 2017. Ethernet provides the most cost-effective and scalable way to support the demand for mobile backhaul and cloud-based service delivery.
But Ethernet’s migration from LAN to WAN and the datacenter is only the latest part of the technology’s journey toward near-universal adoption. In the near future, its versatility and low cost will make it the upgrade path of least resistance for most of the non-standard legacy protocols that are still prevalent within many industrial, utility, transportation, and government networks.
IoT is a recently coined term used to describe the web of machine-to-machine (M2M) networks that enable the free exchange of messages between so-called “smart objects” and the applications that depend on them within commercial, industrial, and civil organizations. Although still in its infancy, the IoT’s ability to provide real-time visibility and control of real-world objects such as water and electricity meters, point-of-sale terminals, and traffic lights has the potential to bring unprecedented efficiency and transparency to nearly any aspect of our lives.
According to Morgan Stanley projections, 75 billion devices will be connected to the IoT by 2020. Unsurprisingly, Ethernet is evolving again to serve as the IoT’s foundation.
To meet the demand of the mobile, cloud, and IoT networks, Ethernet technology must acquire new capabilities that its creators never anticipated. While overkill for the LAN applications that Ethernet was originally created to support, the new services it’s expected to deliver and the challenging environments it will encounter will require its next generation to provide native support for three critical capabilities: deterministic service delivery, accurate timing and synchronization, and security.
Reliability And Deterministic Performance
Most IT and telecom analysts agree that the mass migration of traditional enterprise IT networks to the cloud is responsible for the recent demand curve steepening for high-bandwidth, reliable, and secure access to XaaS. Released in October 2013, the third annual Cisco Global Cloud Index forecasts growth by a factor of 4.5 in global cloud traffic (a 35% compound annual growth rate) between 2012 and 2017.2 Additionally, a September 2013 study conducted by Forrester Consulting found that more than 75% of enterprises currently use or plan to use some type of cloud-based service.3
What’s less well understood is that the WAN’s steady migration to Ethernet-based infrastructures is largely facilitating the recent explosion in the availability and adoption of cloud-based services. Indeed, the capital, deployment, and operational costs of Ethernet-based networks are a fraction of the legacy TDM-based (time domain multiplexing) systems they displace.
Service providers (SPs) are also seeing meaningful revenue growth from delivery of Ethernet-based access, transport, and WAN services. For example, total Ethernet services revenue in the United States alone will increase from $6.2 billion in 2013 to $10.1 billion in 2016 according to the U.S. Carrier Ethernet Services 2013-2017 Forecast from the International Data Corporation (IDC).4
Although packet-based WANs had been steadily displacing frame-based SONET/SDH systems for over a decade, the major catalyst for widespread adoption of Ethernet services came from MEF-led activities, defining and standardizing the additional attributes that Ethernet requires to meet user expectations. The addition of these new attributes, such as standardized services, operation, administration, and management (OAM) capabilities, quality of service (QoS) classes, and reliability requirements, evolved Standard Ethernet to Carrier Ethernet.
Founded in 2001, the MEF is a global industry alliance of telecommunications SPs, cable multiple systems operators (MSOs), network equipment/software manufacturers, semiconductor vendors, and testing organizations. It first defined Carrier Ethernet, now known as CE 1.0, creating the first generation of standardized carrier-class services. CE 1.0 established a firm base from which Carrier Ethernet could grow and enabled standardized Ethernet service delivery over a single provider’s network.
In January 2012, after Ethernet had become a ubiquitous WAN presence, the MEF defined a new generation of Carrier Ethernet: CE 2.0. It delivers differentiated applications globally over interconnected, managed networks. CE 2.0 differentiates itself from CE 1.0 most notably in its specifications for multiple classes of service (multi-CoS) and its interconnect and manageability standards (see the table).
These standards are especially important in today’s Ethernet marketplace, as every enterprise relies on a specific, unique set of networking services. Enterprises do not want surprises in their networks’ performance and cannot afford any differences in the way their service-level agreements (SLAs) behave in different physical office locations. CE 2.0 ensures that enterprises enjoy consistent, stable service reliability throughout their networks.
A good example of such new capabilities involves an SP’s ability to monitor the connectivity and performance of any given service provided to its customers. The use of up maintenance endpoint (MEP) OAM technology gives SPs remote monitoring of service performance right up to the customer location. This dramatically improves the ability to detect and localize faults, reducing truck rolls and operational expenses, and enables carrier-class service availability. It also supports the delivery of highly profitable low loss and delay services, as needed in enterprise, finance, and mobile backhaul markets.
Additionally, SPs must increasingly work with multiple network operators to expand the scale and reach of their Carrier Ethernet service offerings. Multi-operator services are especially common in mobile backhaul and global enterprise services. With up MEP OAM technology, SPs can monitor the connection health and performance end to end, while simultaneously monitoring through each operator’s network.
A recent announcement from Carrier Ethernet SP Masergy Communications further highlights the growing importance of Carrier Ethernet for the cloud. In this case, Amazon chose Masergy to provide direct private access to Amazon Web Services (AWS) throughout North America, solving the challenges of non-deterministic performance, reduced security, and lack of QoS capabilities associated with access via the public Internet.
As more enterprises move toward using cloud-based services, SPs can deploy cost-effective and high-performance CE 2.0-compliant equipment for cloud service delivery. In fact, a new generation of Ethernet access devices already incorporates hardware-based CE 2.0, leading to higher-performance, lower-cost power and form-factor solutions for SPs looking to provide these on-demand, high-performance cloud networking services.5
Timing And Synchronization Are Key
Mobile networks require precise timing—for example, to support call handoff between towers—as well as a stable reference clock to generate radio frames for transmission between cell sites and customers. For 2G/3G networks, timing is typically delivered by either a SONET/SDH-based T1/E1 line or from GPS satellites.
But to support exponentially growing demand for bandwidth without increasing capital expenditures (CAPEX) proportionally, operators have rapidly transitioned to packet-based Ethernet backhaul networks. The idea of delivering timing over SONET/SDH is not only expensive, but also inadequate for TD-LTE (time domain) and LTE-Advanced (LTE-A) wireless networks, since SONET/SDH can only deliver frequency synchronization, but not the requisite phase synchronization. That’s why we see a critical need for the IEEE 1588v2 Precision Time Protocol (1588 or PTP), whose packet-based synchronization mechanism can deliver both phase and frequency with the precision that advanced wireless networks require.
Initially, the low cost and excellent performance of 1588-enabled Ethernet made it popular with wireless carriers for upgrading their existing T1/T3-based backhaul networks. More recently, Ethernet’s 1588 capabilities have become invaluable for the deployment of micro-cell and femto-cell basestations used by carriers to provide added capacity in urban corridors and indoor environments (Fig. 2).
Nearly all mobile basestations rely on GPS/GNSS satellite transmissions to acquire an all-important signal known as 1 pps, which is used to make the time-of-day (ToD) calculation for next-second rollover and to synthesize the fundamental source radio frequencies. In many of the environments where these so-called small-cell nodes may be located, they won’t have reliable access to the GPS signals traditionally used to provide the highly accurate frequency and ToD synchronization information required by TD-LTE and LTE-A networks. Without the ability to maintain a unified timing domain across its network, calls will drop, messages will fail to send, and overall user quality of experience (QoE) will diminish.
1588 solves this problem by transmitting timing information within special packets and using a second mechanism to manage packet delay variation (PDV), i.e., the small but inevitable variations in the time 1588 packets take to traverse the network introducing errors in the recovered time. Each mobile technology defines a minimum level of frequency and phase error that it can tolerate, which sets a maximum time error that the backhaul network can introduce. This in turn limits the amount of error that each network element, or network equipment in general, can introduce to the 1588 packets.
As a result, operators need to design their networks carefully, accounting for maximum error introduced by each node and different mediums, such as fiber and microwave links, in the path of the 1588 packets. To assist in the process, the International Telecommunication Union-Telecommunication (ITU-T) is working to standardize different equipment classes with different levels of maximum time errors in the range of 20 to 50 ns. Available silicon solutions can enable network equipment designs to meet these budgets. At least one of these solutions, Vitesse’s VeriTime chipset, includes compensation mechanisms that can even maintain a 1588 timing domain across notoriously variable microwave links.
1588 will also play an important role in many other next-generation network applications, including wired IoT networks. Originally created to enable highly accurate timing within industrial automation systems, 1588 can provide real-world/real-time applications with precise ToD information, time-stamped inputs, scheduled outputs, and synchronized outputs.6
By eliminating much of the so-called “response time jitter” that limits the performance of traditional control networks, 1588 can enable even extremely large, widely dispersed groups of smart objects to work together on time-critical real-world tasks ranging from managing smart utility grids to operating autonomous vehicles on public roadways.
In the Smart Grid, for example, utilities can use 1588 to stabilize their grids against sudden spikes in demand. How? By providing real-time load data and the ability to quickly control the load shedding systems already in place in residential, commercial, and industrial buildings (Fig. 3). 1588-enabled smart grids can also help utilities manage distributed energy sources, such as rooftop photovoltaic systems. Similarly, in autonomous vehicle systems, 1588 can ensure that roadways get the real-time traffic and congestion data they need and that the resulting commands reach the vehicles on their roadways quickly enough for smooth traffic flow.
Notably, there are commercially available silicon solutions with 1588 timing support that not only can meet these stringent accuracy requirements, but also meet the demanding needs, such as small form factor and extended temperature range support, of IoT environments.
Security Is Vital
Ethernet traces its origins back to an era when computers were rare and expensive and lived mostly within the confines of corporate, government, and academic complexes. Often, the only protection many of them had was a simple login password. Yet they were considered relatively secure without much more than a simple password because they tended to be physically isolated from the “real world,” very few people had the technical expertise necessary to access them, and there was very little on the networks of interest to outsiders.
In reality, it’s likely these assumptions weren’t correct even in those early years, and it’s dead certain that whatever small bits of security they might have provided vanished as soon as networks began to touch the real world.
Today, securing information, computing infrastructure, and the networks connecting them has come to the forefront of the popular discussion with almost daily reports about malware, identity theft, intellectual property theft, and denial-of-service attacks on commercial and government cyber infrastructure. To meet these challenges, multiple models of network and data security have been implemented, encompassing:
• Authentication, authorization, and accounting (AAA) at the forefront
• Confidentiality, typically addressed by encryption
• Data integrity, ensuring the data was not modified
• Availability, usually targeted by denial-of-service attacks
• Physical security, building access and protection from destruction
We are going to focus on encryption, as it provides one of the most effective ways to secure a network link, which is highly applicable to an enterprise using a cloud-based service or a datacenter interconnect. In addition, many government regulations involving health care and finance, including HIPAA, PCI, Sarbanes-Oxley, Basel-II, and the Food and Drug Administration, require all data to be encrypted when it leaves the physical security of a building.
There are also growing concerns around mobile backhaul network security, driven by sharing of backhaul infrastructure by multiple operators, geographically distant deployment of cell towers, and lack of physical security in outdoor small cells. Currently, the most common network-based security protocol is IPsec, which operates at Layer 3 of the IP hierarchy (L3). While IPsec works well in routed networks, it is costly to implement because it usually requires dedicated standalone or embedded encryption engines.
The IEEE 802.1AE “MACsec” security standard and its companion key management protocol 802.1af, also known as KeySec, offers an alternative that works in the Layer 2 (L2) of the Internet Protocol hierarchy. Although MACsec/802.1X has seen limited adoption (primarily in data centers), its L2 operation makes it simpler and less costly to implement, easier to use, more efficient, and easily scalable. This makes it a very cost-effective solution for the growing number of service providers offering MEF CE 2.0 services and MPLS-based (multiprotocol label switching) L2 virtual private networks (VPNs) and other Carrier Ethernet L2 VPN services.
As we think about encryption for mobile backhaul links, we need to consider the strict timing requirements discussed earlier. While there is no industry-wide definition for “secure 1588,” solutions are available today, notably Intellisec-enabled devices from Vitesse, for both Gigabit and 10 Gigabit Ethernet links that can deliver line rate encryption without compromising timing accuracy. These solutions can deliver a universal encryption solution for confidential communications over Ethernet networks and any MEF CE 2.0 Ethernet or MPLS service provider connections.
The IoT, much like enterprise and carrier networks, will need strong encryption to secure its many access points. Ethernet for the IoT, or “Industrial Internet” (a broad umbrella encompassing industrial process control, energy distribution, automotive, and even public transport) has expanded rapidly as more and more machines become intelligent and need network connectivity.
Within IoT applications, networking was historically characterized by proprietary communications and networking protocols. However, higher bandwidth and increased networking demands within IoT applications make Ethernet a necessary technology. While still comprising a minority of industrial connections, Ethernet is expanding in this space and is largely driven by Smart Grid energy distribution, transportation, and automotive applications. Any of these connected devices with an IP address is theoretically hackable, and the more connected “things” there are, the more information will be accessible to unwanted parties.
Trustwave Holdings researchers David Bryan and Daniel Crowley, who have hacked into many connected devices, say that security is often overlooked during product development.7 Those 75 billion IoT connections will comprise everything from traditional industrial connections, like factory networks, to individual home appliances. All of these additional connections, as with small cells in the backhaul network, will increase access points for hackers. Scalable, cost-effective security technologies will only become more important as the IoT grows and the world becomes more connected.
Though end-to-end MACsec encryption will grow significantly in the coming years as Ethernet connections proliferate, it must do so without affecting another important aspect of IoT networking: timing and synchronization. As discussed previously, OEMs will need to use the latest security technologies, which can simultaneously allow for accurate network synchronization and line rate encryption.
Conclusion
Ethernet connections, already diverse, will only continue to expand as applications move to the cloud and previously unconnected devices join smart phones and tablets in the network. But many questions still need answering. The networking technology now in its fifth decade of existence is continuously evolving to serve enterprises, carriers, and the IoT. No matter the direction of this evolution, the three requirements of services, security, and synchronization will be important part of Ethernet’s future evolution to meet end user demands.
References
1. Ethernet was developed at Xerox PARC between 1973 and 1974, commercially introduced in 1980, and standardized in 1983 as IEEE 802.3. Also see “IEEE 802.3 ‘Standard for Ethernet’ Marks 30 Years of Innovation and Global Market Growth,” IEEE, June 24, 2013, and The History of Ethernet, NetEvents.tv, 2006.
2. “Cisco projects data center cloud traffic to triple by 2017,” Steven J. Vaughan-Nichols, www.zdnet.com/cisco-projects-data-center-cloud-traffic-to-triple-by-2017-7000021985/
3. “New independent consulting study shows value of cloud services remain elusive for many enterprises,” www.xo.com/about-xo/news-and-events/press-releases/new-independent-consulting-study-shows-value-of-cloud-services-remains-elusive-for-many-enterprises/
4. “Datacenter, VPN, and Cloud Service Connectivity to Drive U.S. Ethernet Revenues to $10.1 Billion by 2017, IDC Says” www.idc.com/getdoc.jsp?containerId=prUS24365613
5. “Carrier Ethernet 2.0 comes to silicon” www.carrierethernetnews.com/articles/636232/carrier-ethernet-2-0-comes-to-silicon/
6. “An Application of IEEE 1588 to Industrial Automation,” Ken Harris, Rockwell Automation, http://literature.rockwellautomation.com/idc/groups/literature/documents/wp/1756-wp005_-en-e.pdf
7. “More connected homes, more problems,” Rachel Metz, August 13, 2013, www.technologyreview.com/news/517931/more-connected-homes-more-problems/
Uday Mudoi, vice president of product marketing at Vitesse Semiconductor, has nearly two decades of experience in the communications and semiconductor industries. He holds a bachelor of science degree in electrical engineering from the Indian Institute of Technology, Kharagpur, a master’s degree in computer science from North Carolina State University, Raleigh, and an MBA from Columbia University, New York, N.Y. He can be reached at [email protected].