What is Gen-Z? An Overview of Gen-Z Memory-Semantic Fabric Architecture

>> Electronic Design Resources
.. >> Library: Article Series
.. .. >> Series Topic: System Design
.. .. .. >> Gen-Z: Opening Up the Data Center

Fabric-based architectures provide cloud providers with a platform to build extremely large computational solutions. Gen-Z is on the forefront of this wave. I spoke with Tim Symons, Associate Technical Fellow at Microchip and the Gen-Z Consortium Marketing Workgroup Co-Chair, about the architecture and the consortium.

What is Gen-Z technology?

Gen-Z is an open-systems fabric-based architecture designed to interconnect processors, memory devices, and accelerators. The fabric enables resource provisioning and sharing to support a diverse range of applications and provide flexibility for system reconfiguration as application demands for different resources change.

Gen-Z fabric utilizes memory-semantic communications to move data between memories on different components with minimal overhead. Memory-semantic communications are extremely efficient and simple, which is critical to delivering low-latency, high-bandwidth data transactions.

The technology was developed to enhance existing server solutions and enable new architectures, while increasing performance (high-bandwidth, low-latency), software efficiency, power optimization, security, and industry agility.

What are the key technical advantages of Gen-Z?

Gen-Z introduces many new key technical advantages:

Memory media independence enables any mix of dynamic RAM (DRAM), non-volatile memory (NVM), and future persistent-memory technologies
High-bandwidth, low-latency, efficient protocol that simplifies hardware and software designs, reducing solution cost and complexity
Multipath communication enables very high signaling rates and solution resiliency
Scalability without sacrificing performance and solution flexibility
Security and hardware-enforced isolation
Supports advanced workloads and technologies
Mechanical compatibility enables Gen-Z to be integrated into existing platforms
Software compatibility
High-speed signaling rates of 56 GT/s (per link), with potential for 112 GT/s and beyond
High-efficiency protocol

What specifications have been released by the Gen-Z Consortium? Are they available to the public?

The Gen-Z Consortium has released a total of 11 specifications, including the Core Specification 1.1, Physical Layer Specification 1.1, Scalable Connector Specification 1.2, and Fabric Management Specification 0.7 Draft. The Consortium has also contributed six mechanical form factor and connector specifications to SNIA SFF. Gen-Z final specifications are available for public download on the Consortium’s website here.

What is the Gen-Z Consortium?

The Gen-Z Consortium is an industry organization developing an open-systems, fabric-based architecture designed to deliver high-speed, low-latency, secure access to data and devices.

The Consortium continues its mission to provide a suite of specifications that establish an open ecosystem where leaders in the technology industry can directly engage with one another in developing solutions based on these specifications.

What is Gen-Z’s role in the industry?

Gen-Z is an open set of specifications that define a protocol and fabric to interconnect memory-centric resources. It enables servers to be developed that can be reconfigured as application loads demand. Open specifications indicate that Gen-Z member organizations may implement Gen-Z technologies without requiring additional licenses or risking infringing on other member patents. Gen-Z leverages existing technologies wherever possible to reduce costs and development time of the ecosystem.

What are some of the unique features of a Gen-Z fabric?

There a number of them:

Gen-Z abstracts memory the memory controller, allowing the IT industry to deploy a wide range of memory media types. This enables multiple generations of memory devices and new storage media to be transparently supported in any system. It also enables customers to independently replace and upgrade components based on their needs—for example, FPGA accelerators, GPUs, memory modules, NVM modules, etc.
The fabric can be reconfigured to accommodate varying resource requirements of existing and emerging memory-centric applications.
Gen-Z fabric management is secure and robust. It utilizes industry-standard protocols (i.e., MCTP) to enable rapid deployment and ensure compatibility with existing management tools.
Gen-Z supports a robust hardware-enforced zoning and security framework to protect the fabric from cyber threats.
Gen-Z interoperability enables the architecture to simultaneously and efficiently transport standard and customized communications between components, which enables customers and vendors to rapidly innovate and deploy new capabilities and services.

Is the Gen-Z Consortium involved in any industry partnerships?

The Gen-Z Consortium formed a partnership with DMTF to develop extensions to the Redfish API and other DMTF standards. Earlier this year, Gen-Z announced Memorandum of Understanding (MoU) Agreements with the Compute Express Link (more information below) and the OpenFabrics Alliance (OFA). The latter agreement with OFA advances the industry standardization of open-source fabric management. In addition, the Consortium has donated six Gen-Z-developed mechanical form-factor and connector specifications to SNIA SFF to enable a broader range of applications to adopt the innovations developed by Gen-Z.

What advantages does the recent MoU between the Gen-Z Consortium and CXL Consortium provide?

The Memorandum of Understanding (MoU) between CXL and Gen-Z establishes a framework for collaboration between the two industry standards. It also demonstrates the commitment of each organization to promoting interoperability between the technologies, while leveraging and further developing complementary capabilities.

The agreement outlines the formation of common workgroups intended to implement cooperation guidelines and define potential bridging capabilities between the protocols while leveraging the strengths of each technology. The figure illustrates a scenario in which the CPU provides a CXL interface, as well as the standard DDR channels, and Gen-Z provides a bridging device to attach to external (or internal) pools of resources, such as memory, accelerators, or network interface controllers (NICs). (More information about the MOU can be found here.)

The CPU supplies a CXL interface, as well as the standard DDR channels, and Gen-Z provides a bridging device to attach to external (or internal) pools of resources.

What physical layers does Gen-Z support?

Gen-Z supports two of the highest-volume physical layers: the PCI Express physical layer and an optimized version of the IEEE 802.3 electrical-layer specification.

Gen-Z also specifies a physical-layer abstraction that allows the protocol to be transported across the physical layers without impacting the higher-level functional blocks. The protocol utilizes flit packets that encapsulate packets and makes it possible to implement error correction at the link layer, creating improved data integrity, faster deployment of new physical layers, and faster signaling rates.

What Gen-Z technology demonstrations has the Consortium showcased?

The Gen-Z booth at SC19 featured several live demonstrations showcasing Gen-Z progress, including:

Multi-vendor rack featuring HPE and Dell EMC using AMD and Intel x86 processors and IBM servers with Power Processors; disaggregated, pluggable DRAM drives from SMART Modular utilizing shareable memory resources
Intel Ultra Path Interconnect (UPI) to Gen-Z memory-semantic bridge POC with IntelliProp IP
Gen-Z media box architecture showcasing remote memory over 5-meter copper cable
Application of Gen-Z in a machine-learning environment (Samsung) and unique memory applications (SK Hynix)
Optical Interconnect demonstration using PLDA IP on Xilinx FPGAs and Arm cores
Summit M5x Gen-Z Protocol Analyzer / Jammer from Teledyne LeCroy
Gen-Z cable and connector products from Amphenol, Lotes, Luxshare, Samtec, and TE Connectivity

Where can developers learn more about Gen-Z?

They can visit https://genzconsortium.org/ for resources, including white papers and educational materials, and to download the Gen-Z specifications. They can also subscribe to the Gen-Z webinar channel to view live webinars on a variety of topics, as well as previous webinars on demand.

Don’t forget to follow Gen-Z on Twitter, LinkedIn, and YouTube (@GenZConsortium).

Tim Symons is an Associate Technical Fellow at Microchip Technology and Gen-Z Consortium Marketing Workgroup Co-Chair. He is a specialist in memory and storage interface protocols, PHYs, and the interface ecosystem. Tim has represented storage industry standards for nearly 20 years, including the development of SATA, SAS, NVMe, PCIe, and Gen-Z. He is currently editor for the INCITS T10 SAS Protocol (SPL) standard and an expert in hardware platform architectures for secure, high-availability, reliable data-storage systems.