With the remarkable surge in demand for connectivity, the internet has become the communications highway for billions of users. According to “Internet Live Stats,” 4.2 billion people traveled the internet in 2019. That’s about 55% of the world’s population, and the amount of traffic generated by this connected population is truly staggering.
The vast web of computers and mobile devices that transmit, receive, and deliver this data wouldn’t even be possible without advances in networking technologies. Ethernet is an iconic example—one that continues to evolve with astonishing rapidity.
Since the 1980s, when a seven-port Ethernet switch was introduced, port count has scaled dramatically. Today's Ethernet switches and routers now reach 1000 ports, and by the end of 2020, will accommodate 4000 ports. Also, in contrast to the throughput of 10 Mb/s of the seven-port Ethernet switches, they can now handle throughputs of 1/10/25/40/50/100/200/400 Gb/s.
While the Ethernet Alliance can foresee the future growth in number of ports, the increase of bandwidth to 1600 Gb/s is less obvious due to limits in the transmission medium. Dense wavelength division multiplexing and protocol analysis module signaling have been used to overcome these limits to date. We may anticipate a move to adopt more parallelism to expand the bandwidth. Latency in networking switches has constantly decreased, and today the lowest latencies have dropped below 1 µs.
A larger number of ports, expanding throughput, decreasing latency, and overall improvement in security and ease of use are making today’s network switches and routers among the largest IC designs ever developed, reaching well beyond a billion gates and now on par with the largest processor and graphics chips. Verifying such complex IC designs, before silicon availability, is a daunting task. Fortunately, help has already arrived in the form of hardware emulation and, specific to Ethernet, with the VirtuaLAB Ethernet solution.
This article explains some of the technology behind why virtual emulation is the only way to keep up with the growing speeds and capacities of Ethernet designs.
In-Circuit Emulation Limits
For example, a design of a modern system-on-chip (SoC) with a 256-port Ethernet interface and a variable bandwidth of 1/25/40/50/100/200/400 Gb/s surpasses the capabilities of traditional digital simulators. While hardware description language simulation can be used at the block level, verification of an entire design of a billion gates with simulated traffic is unrealistic and must be ruled out. This is a primary case for using hardware emulation in what’s called in-circuit-emulation (ICE) mode.
Unique to the ICE verification mode is the ability to test a design with real traffic. An ICE configuration requires one Ethernet tester per port. Since a direct connection isn’t possible due to the different speed domains between the tester and the emulated design under test (DUT), a speed rate adapter is inserted between the two. This reconciles the fast speed of the tester to the relatively low speed of the emulated DUT.
But when we tackle a modern switch design that can support a port configuration of 256 ports and variable bandwidths of 1/10/25/40/50/100 Gb/s, we start to see some cracks in the ice. Because the design has 256 ports, the ICE setup requires 256 Ethernet test ports, 256 Ethernet speed adapter ports, as well as CAT5 and IO cables to the emulator system (1K cable channels) (Fig. 1). An ICE approach becomes difficult, not only from a cabling perspective, but also from a traffic-shape accuracy perspective when exercising a DUT through speed adapter ports.
1. A 256-port Ethernet switch is verified using ICE.
Apart from the spaghetti-cable arrangement, the potential unreliability of the hardware, overall cost, and loss of traffic-shape accuracy, a frustrating limitation is that the entire setup supports only a single user located in the proximity of the emulation lab. Time-sharing using modern parallel and batch mode techniques is extremely difficult.
Untangling Ethernet Design Verification
On the other hand, a virtual-emulation environment frees users from the limitations and restrictions of ICE. Let’s compare the ICE setup to an equivalent one using a virtual approach. In the virtual scenario, Ethernet testers are modeled in software running under Linux and/or virtual machines on a workstation connected to an emulator. This model is an accurate representation of the actual physical tester, based on proven-in-implementation intellectual property.
Virtual testers typically include Ethernet Packet Generator and Monitor (EPGM) functions that do exactly that—generate, transmit, and monitor Ethernet packets exchanged with the DUT. It should have the ability to configure any GMII, XGMII, 25GMII, XLGMII, 50GMII, CGMII, 200GMII, 400GMII, or Physical Coding Sub-layer (PCS) interfaces for 1G, 10G, 25G, 40G, 50G, 100G, 200G, 400G, respectively.
The interface between the virtual tester and the DUT can consist of one too many instances of the data programming interface (DPI) to scale up the emulation test system to support terabit throughputs for the largest networking systems. Packet stimulus for transmission and packet reception for analysis flows bidirectionally through an EPGM tool to/from a virtual Ethernet transactor and Null-PHY connected to the DUT. One xRTL transactor is required for each port of any Media Independent Interface (xMII) or PCS-supported type (Fig. 2).
2. One instance of EPGM-DPI communicating to a Virtual Ethernet xRTL is all that’s required.
For example, VirtuaLAB Ethernet can provide up to 64 ports per workstation for any speed type. Multiple VirtuaLAB applications are able to be bundled together across multiple workstations to support large port count configurations. High-speed optical link cards are used to co-model host workstations to the emulator.
This tightly integrated transport mechanism is tuned for maximum wall-clock performance rates and is completely transparent to the testbench. VirtuaLAB Ethernet’s parallel runtime architecture has been implemented such that data-plane emulation throughput scales linearly with co-model port count (Fig. 3). For example, emulation throughput performance with VirtuaLAB Ethernet for a 256-port SoC is in excess of 1T packets per day.
3. This is a high-level view of the multi-co-model topology.
Apart from enabling high data-plane transport, several other benefits come from using this approach. First, reconfiguring the virtual tester to perform various functions is fast via remote access. Second, the workstation is a stable and reliable piece of equipment acquired at a fraction of the cost of a complex Ethernet tester of equivalent functionality. Third, these workstations fall neatly into standard data-center-grid engine requirements.
Even more important is the ability to handle multiple concurrent users and concurrency across distributed software models that are essential in supporting large software development teams. Emulation hardware and software architecture views must mind this scalability aspect from the ground up.
A virtual Ethernet device should support a directed test mode of operation to define and control the streaming of specific packet types, traffic shape, scheduling and error injection into the emulated DUT, as well as trace and analysis of packets returned from the DUT. VirtuaLAB Ethernet is an example of how Tcl, Python, and GUI interfaces can ease generation, analysis, and scoreboarding through rules-based user definitions.
VirtuaLAB Ethernet can be configured for multiple co-model hosts supported by one instance of software running on up to 16 co-model hosts, with up to 1K Virtual Ethernet xRTL Transactors connected to Null-PHYs and to the DUT ports on the emulator. These 1K VirtuaLAB instances can be managed through a centralized “controller” software application invoked from one of the workstations.
Complex Testing and Analysis
A virtual Ethernet solution is well-suited for complex test scenario generation, monitoring, and automation. For example, an interactive and batch-mode Tcl/Python programming command line interface (CLI) can be incorporated to control all media access controllers, and to generate myriad protocol and traffic flow scenarios. VirtuaLAB Ethernet supports traffic construction CLIs over non-homogenous packet types. Percentages of traffic per protocol type can be mixed, with differing packet sizes or random size per flow or stream. It’s also possible to control packet transmission and arbitration.
Driven by the high rate of networking technology convergence, virtual Ethernet is part of a class of virtual networking solutions that includes other related protocols, such as Flexible Ethernet, 5G Enhanced Common Packet Radio Interface, and Optical Transport Networking. Virtual Ethernet solutions also perform offline traffic analysis, per port/flow statistics, and many other complex analysis functions. OSI Level 1 – Level 7 stack testing should also be provisioned.
Moreover, VirtuaLAB Ethernet supports stress testing and error injection for complex switching topology using the mutable (dynamic) port group reconfiguration features for 1/10/25/40/50/100/200/400-Gb/s full-duplex speeds. For example, xMII/PCS widths, link speeds, link up/down, and fault states can all be dynamically configured during emulation runtime, and need no recompile steps to support testing numerous port group configurations.
Many protocol and performance violations are automatically reported, such as cyclic redundancy check, preamble, inter-frame gap (IFG), line rate, and forward error correction. Packets can be reviewed in interactive or batch sessions to examine packet statistics per flow/port, ingress/egress trace, meta-data such as signatures, time stamps, latency and content all on the wire.
One of the hallmarks of virtual Ethernet solutions in emulation is their ability to do complex performance analysis of large and deeply state-full SoC and multichip systems. Virtual Ethernet in emulation has been used in validating packet classification, filtering, bandwidth rates, policing, traffic shaping, class-of-service, congestion, memory management, packet sequence drops, IFG analysis, dynamic load balancing, group multi-path and network fabrics using signature analysis features available in virtual Ethernet systems.
Virtual Ethernet solutions reside entirely within software, so anything can be imagined and implemented quickly for capture and analysis with 100% visibility. Examples include custom and vendor-specific protocols, support for higher function analysis add on and even non-standard packet interface types. Virtual Ethernet solutions easily accommodate architectural exploration, integration with third-party tools, and bypass SoC logic that has little value in being re-verified for systems seeking greater test throughput for subsystems.
For some complex test and analysis, millions of packets may be needed to hit steady state or transient analysis points of interest in the design. Take, for example, measuring bit rates for traffic flows in a terabyte Ethernet switch with a 32-MB packet buffer. Virtual Ethernet signature generation and packet time stamps can be used to measure cut-through and store and forward latency conditions, RFC2544, RFC2889, etc.
Testing World-Class Streaming Devices
Virtual Ethernet provides a software-controlled environment for generating, transmitting, and analyzing Ethernet packets to test Ethernet SoCs with an emulation platform. Because the emulator runs on enterprise servers, multiple users located anywhere in the world can test over 20 million SoC packets per day concurrently. VirtuaLAB Ethernet brings complex Ethernet SoC designs to market on schedule with significantly increased productivity.
Ron Squiers is a Solutions Architect for Mentor, a Siemens Business.
