Consumers demand more and faster data. They no longer tolerate even the smallest delay to data access. In some applications, such as autonomous driving and medical IoT devices, delayed communication can lead to life-and-death situations. 5G’s high-bandwidth and real-time capabilities open the door to these and many other new exciting applications. But the core communication network and data centers must be ready to support the insatiable demand for data and ultra-fast data-transfer rates.
Many data centers are currently at maximum capacity with 100-Gigabit Ethernet (GbE) speeds. Thus, data-center networks need to move from 100- to 400-GbE speeds to support the requirements of emerging technologies. And faster networking speeds require faster memory and faster serial bus communications.
In addition to moving to 400-GbE speeds, data-center operators must move to the next generation of high-speed computing interfaces, such as Peripheral Component Interconnect Express (PCI Express or PCIe) and double-data-rate (DDR) memory. Data-center operators also need to consider new specialized interconnect technologies that offer alternatives to PCIe.
PCIe expansion bus speeds will switch from PCIe 4.0 to PCIe 5.0 to support 400-GbE speeds. The same is true for memory, as DDR moves from DDR 4.0 to DDR 5.0. Increasing the speed of serial data communications requires high-speed precision testing at every level. Testing at these faster speeds demands full compliance testing to the latest standards.
Each generational change of high-speed computing standards provides new features and faster data-transfer rates, creating new test challenges for digital designers. The figure shows a roadmap of high-speed computing interface standards. The need to measure complex specifications complicates the design and validation process and requires a long learning curve for test engineers. Since standards evolve quickly from one generation to the next, test engineers can save significant time and get their designs to market faster using test solutions that ensure full compliance with industry standards.
Shown is a roadmap of high-speed computing interface standards.
DDR5 Doubles DDR4’s Data Rate
Each new generation of the DDR synchronous dynamic random-access-memory (SDRAM) standard delivers substantial improvements over the previous generation, including increased speeds, reduced footprint, and improved power efficiency. DDR4 is designed for the computing server industry and supports data-transfer speeds up to 3.2 Gtransfers/s (GT/s).
The Joint Electronic Devices Engineering Council (JEDEC), the organization that defines the DDR specifications, is currently working on the next generation of DDR memory—DDR5—to fulfill the need for faster data rates. DDR5 will operate at up to 6.0 GT/s or higher and will effectively double the data rate of DDR4.
PCI Express Evolves to PCIe 5.0
Server speeds increased from 16 Gb/s to over 30 Gb/s in about a year, and future technologies will use PAM4 to push data rates above 50 Gb/s to support 400 GbE in the data center. PCIe 4.0, with a data rate of 16 GT/s, is no longer sufficient to support 400 GbE speeds. As a result, the PCI Special Interest Group (PCISIG), the standard body that defines the PCI Express specifications, has fast-tracked the development of the PCIe 5.0 standard, which is due for completion by the end of 2019. With a data rate of 32 GT/s, PCIe 5.0 provides twice the throughput of PCIe 4.0.
Emerging new standards offer alternatives to PCI. They include Open Coherent Accelerator Processor Interface (OpenCAPI), Gen-Z, and Cache Coherent Interconnect for Accelerators (CCIX). These bus standards specialize in areas where PCIe isn’t customized.
OpenCAPI Accelerates Computing
OpenCAPI is an open, coherent, high-performance bus standard that accelerates computing through tighter integration of different types of technologies, such as advanced memory, accelerators, networking, and storage, within servers. The OpenCAPI standard, defined by the OpenCAPI Consortium, provides a 25-Gb/s data rate and aims to improve server performance by moving computing power closer to the data. OpenCAPI enables a very-low-latency interface between the CPU and an attached device, removing bottlenecks caused by I/O inefficiencies.
Gen-Z Targets Memory-to-CPU Connections
Gen-Z, defined by the Gen-Z Consortium, is an open interconnect standard optimized for storage technology to increase the speed of memory-to-CPU connections. Gen-Z version 1.0, based on IEEE-802.3 physical-layer specifications, provides 25-GT/s and 28-GT/s interconnect speeds and is scalable to 112 GT/s and higher. Gen-Z components use low-latency read and write operations for direct data access with minimal application or processor involvement.
CCIX Increases Data Throughput
The principle behind CCIX, defined by the CCIX Consortium, is to use the PCIe physical layer (PHY) but change the function of the bus for increased efficiency and faster speeds. The CCIX standard currently supports data rates up to 25 Gb/s and is expected to extend that to 40 Gb/s soon.
In addition to faster interconnect speeds, CCIX enables cache coherency. Cache coherency quickly propagates any changes to data in one area of memory to all other instances of the data stored in different memory locations throughout the entire system. For example, there can be a copy of data in main memory as well as one in the local cache of every processor that has previously requested the data. With cache coherency, CPUs can communicate faster with the rest of the system.
400 GbE Introduces New Test Challenges
The design of high-speed serial data links becomes significantly more complex as data rates increase—channel topologies become more diverse, and the number of parameters tuned for active components multiply. Signal integrity at 400-GbE speeds is a critical challenge for high-speed computing interface designers.
Design and simulation tools enable the optimization of the transmitter, receiver, and channel designs for best performance and reliability at the desired data rate. Such tools enable designers to plan upfront to resolve signal-integrity issues, ensure power efficiency, and stay within tight error margins before the first prototypes.
Test engineers must develop comprehensive test plans to ensure compliance with industry standards and interoperability with devices from other vendors. Testing next-generation standards is particularly challenging when test requirements differ drastically from one generation to the next.
For example, the DDR5 standard specifies that test engineers need to test both the transmitter and the receiver. Previous DDR technologies required only transmitter testing. Testing new interconnect technologies, such as OpenCAPI, Gen-Z, or CCIX, is equally challenging since test engineers have never tested them before and need to ramp up on test requirements and procedures.
Compliance test software developed in accordance to industry standards can reduce test time down from days or weeks to hours. Industry-compliant test solutions ensure that next-generation devices comply with industry standards and are interoperable with devices from other vendors. Using fully compliant test solutions, engineers can focus on designing next-generation devices, instead of learning the details of each new standard.
Nicole Faubert is the Industry Solutions Marketing Director for the Communication Solutions Group at Keysight Technologies.
