Check out more coverage of the 2022 Flash Memory Summit.
NVIDIA said it plans to plug chiplets into future generations of its processors, linking them together at the package level using a new standard die-to-die interconnect backed by AMD, Arm, Intel, and Qualcomm.
The Santa Clara, California-based company joined the board of the industry group coalescing around the “Universal Chiplet Interconnect Express” (UCIe) interface that could help reshape the world of chip design.
Ideally, the new UCIe standard would allow everyone from semiconductor giants to startups to buy chiplets for CPU cores, peripherals, memory, accelerators, and many other subsystems, and then place them on a substrate in a similar way to assembling parts on a tiny circuit board. With UCIe, it will also be possible to mix and match chiplets no matter the companies that designed and manufactured each one.
What may make the UCIe standard succeed is that many of the biggest names in the industry support it. In addition to processor vendors, participants include leading foundries Samsung and TSMC; cloud providers and hyperscalers Google, Meta, and Microsoft; and packaging giant ASE Technology. They all launched the UCIe Consortium in early 2022 to drive a chiplet ecosystem and future generations of chiplet technologies.
Way of the Future
But the backing of the largest U.S. semiconductor maker by market value promises to give the UCIe standard a major boost, setting it up as the die-to-die interconnect of the future. NVIDIA executives said it also plans to build the UCIe interface into future generations of its GPUs, Arm CPUs, and other products.
“NVIDIA welcomes industry-standard methods to connect multiple chiplets to scale computing,” said VP of hardware engineering Ashish Karandikar. “We’re committed to advancing the UCIe standard, both by joining its board and by supporting it in our upcoming GPUs, DPUs and CPUs, creating new possibilities for custom chips and systems-level integration.”
The UCIe standard will also open the door for it to integrate non-NVIDIA intellectual property (IP) from its partners—for instance, silicon-photonics chiplets from startup Ayar Labs—in its future data-center products.
The UCIe standard is based on top of the Peripheral Component Interconnect Express (PCIe) bus, which is one of the most widespread standards in the computing world and the backbone of everything from servers in data centers to embedded systems. With the UCIe standard, NVIDIA and other companies hope to take it inside the package in an attempt to mimic what it accomplished outside the package—on the board level.
PCIe is also the physical and electrical interface behind the emerging "Compute Express Link" (CXL) chip-to-chip interconnect, which allows for accelerators and memory to attach to CPUs with cache coherency.
In the same way, NVIDIA and other semiconductor giants are trying to transform UCIe into a standard plug-and-play interface for chiplets, giving them a universal way to chat with each other in everything from 2.5D to the most advanced 3D packages. According to the group, the UCIe 1.0 specification is a complete standard die-to-die interconnect, covering the physical layer, protocol stack, software model, and testing suite.
As more companies adopt chiplet-based designs, the UCIe standard should also help reduce costs through economies of scale and ease of integration, making the approach more accessible to everyone.
Impossible to Ignore
While the UCIe standard is in its early stages, the chiplet as a concept is not. The likes of Intel and AMD have started rolling out server processors with chiplets inside to counter the physical constraints of Moore’s Law.
A major problem is that it is currently impossible to print out a semiconductor die larger than the reticle limit of the most advanced lithography equipment in a modern fab, which stands at around 850 mm2. That limits the number of transistors—and thus the amount of computing power—that can fit on the same chip. NVIDIA hit the wall for the first time with its Volta GPU introduced in 2017.
To work around this issue, NVIDIA is investing in its in-house high-speed interconnect dubbed NVLink. The chip-to-chip interface is being used to assemble so-called “superchips” that merge CPUs and GPUs in compact modules that are ideal to handle AI workloads in colossal cloud data centers or supercomputers.
But the move to support the UCIe standard signals that the merits of chiplets are impossible for it to ignore.
The concept behind chiplets—or tiles as Intel calls them—is divide and conquer. These tiny squares of silicon are sliced out of what would previously have to be a single monolithic die. The chiplets are then reassembled in a package that is more than the sum of its parts. In addition to saving space, this gives you the freedom to glue together many different types of tiles, each one based on the process technology best suited for them.
The “system-in-package” approach contrasts with the system-on-chip approach frequently used in mobile phones, where all of the different components that used to be slapped on a circuit board share a single IC.
This pays dividends in power efficiency (placing everything in a package instead of on a PCB reduces the distance that data and signals must travel between components) and performance (spreading out the transistors over separate die means you can cram more computing power in a package than an SoC).
Monolithic dies—specifically in high-end server processors such as NVIDIA’s—have other shortcomings, too. Chiplets are more compact, leaving less room for imperfections that can hurt yields, thus increasing costs.
In the Express Lane
But when you disaggregate a SoC, you need to reassemble all the chiplets. That means making sure all of the different ICs can communicate with each other almost as fast as they would sharing a single die.
Today, companies tend to use proprietary interconnects that lock them into using IP designed in-house on each die. UCIe is an attempt to build a universal interconnect that allows for seamless mixing and matching.
More than 60 companies—including Broadcom, Cadence, Marvell, MediaTek, Micron, and Synopsys—have joined the UCIe Consortium, contributing to a vibrant ecosystem. Many are adding UCIe-based offerings to their roadmaps.
“The industry response to our announcement of UCIe has been overwhelmingly positive,” said Intel’s Debendra Das Sharma, chairman of the UCIe Consortium. “We have an ambitious plan to continue evolving UCIe technology to meet industry requirements and develop a global interoperable chiplet ecosystem.”
Still, it will probably take several more years for the first processors with UCIe inside to hit the market. It could take even longer for UCIe to become as widespread at the package level as PCIe at the board level.
Check out more coverage of the 2022 Flash Memory Summit.