Check out Electronic Design's coverage of APEC 2024.
While AI chips in the data center are becoming progressively more power-hungry, they’re not alone. The power demands of CPUs, GPUs, FPGAs, SoCs, and everything in between also continue to rise.
In that context, it’s becoming more difficult to deliver the appropriate power over the “last inch” of the power-delivery network (PDN) that stretches over the printed circuit board (PCB) to the load itself. It’s the responsibility of the voltage regulator (VR) to supply large amounts of current to the load and then deliver it at the very small voltages used by the system-on-chip (SoC) to run as efficiently and reliably as possible under dynamic loading conditions.
However, lots of obstacles stand in the way of delivering clean, stable power to the load. To prevent noise and ripple from interfering with the power integrity (PI) of the PDN, it’s standard practice to run at a higher voltage, which unfortunately wastes power.
“You end up setting the supply voltage higher to account for all the dips that can happen, but if you have a smaller, tighter voltage drop, you can effectively run the load at a lower voltage,” said Mukund Krishna from Empower Semiconductor, a power chip startup.
Empower is trying to replace traditional power-management solutions with the integrated voltage regulator (IVR). The IVR comprises the entire voltage regulator in a single chip. Since it integrates most of the magnetics and passives responsible for regulating power as it enters the point of load (POL), it can be placed very close or even underneath the processor to wring out additional power savings.
The secret to Empower’s IVR is its speed. The chips can run at orders of magnitude higher frequencies, giving them faster transient response and higher bandwidth so that they can adjust the supply voltage of the load more rapidly.
“That speed ultimately allows the load to be more efficient,” said Krishna, Empower’s senior manager of product marketing. Higher frequencies also reduce the inductors and magnetics that must be placed in front of the load, allowing integration or elimination of these large, tall components.
Bolstered by more than $75 million in funding, Empower has introduced several high-density power-management ICs in the last several years. They’re designed to supply up to tens of watts of power. But the company is confident it will start hitting its stride by upgrading the IVR technology to handle power-hungry GPUs and other massive AI chips that can burn through more than 1000 W.
Why Is Power Integrity Becoming a Problem?
While a power supply distributes power inside data centers, on factory floors, or other electronic and embedded systems, the voltage regulator is the final stage it encounters before entering the load itself.
But pushing the power over the circuit board to the finish line is becoming a much more daunting task. One of the problems stems from the fact that many more semiconductor companies are upgrading to 16-nm and more advanced nodes. These process technologies have inherently stringent limits on core operating voltages to limit the power consumed by the FinFET transistors within.
Krishna pointed out that “1 V is yesterday, 0.8 V is today, and 0.5 and 0.6 V is the future for high-performance ASICs.”
As a result, the voltage regulators have significantly less margin for error. Even a difference as slight as 50 mV in the supply voltage can cause a “non-trivial” reduction in a processor’s efficiency, said Krishna.
There are tradeoffs. Since Ohm’s Law states that power is equal to current times voltage (P = I × V), these increasingly power-hungry chips require hundreds to thousands of amps of current, which are inherently harder to handle.
Such heavy currents can lead to unsustainable power losses because of the resistance on the current’s path. Krishna explains, “When you’re handling tens of amps, maybe it doesn’t matter that much, but when you’re thinking about high-power processors that have hundreds of amps or thousands of amps, there are tens of watts that you’re losing to the impedance of the PDN alone if you don’t pay attention.”
A greater difference exists between the peak and idle current consumed by modern chips. Therefore, the voltage regulator must be able to handle larger step changes in the current, also called load transients (di/dt). If the amount of current rushing into a processor rises suddenly—for instance, to run at higher clock speeds—the output voltage will drop while current rises accordingly. The voltage being delivered to the load must be increased—or “recover”—as fast as possible to keep everything running efficiently.
Empower said the IVR uses its ultra-fast response times to regulate the voltages entering the processor more tightly. While traditional DC-DC converters can accurately supply the core voltages required by the load, they’re forced to run at frequencies from 0.3 to 3 MHz to do so efficiently. The slower switching means it must be equipped with many large inductors and capacitors to “rectify” the output voltage during load transients, while reducing the parasitic inductance and capacitance in the current’s path.
The inductors are heavy, bulky parts used to prevent sudden changes in current from overpowering the load, while the capacitors store and release energy at a constant rate, smoothing out the power racing into the processor. Specifically, they’re used to “decouple” or prevent the output voltage from dipping when large currents rush into the digital load. They also dampen ringing, making sure the current traveling to the point of load remains unaffected by the ripple that may be present on the power plane.
All of these components require real estate on the PCB, limiting how close the voltage regulator can be placed to the processor. As current travels a longer lateral distance between the DC-DC converter and the load, it encounters more resistance on the power rails, increasing impedance in the PDN. In general, higher currents require more phases and every phase adds more magnetics and passives to the system.
“You can’t get too close because there are too many components standing in the way, and sometimes the parts are so tall that you can’t physically get any closer because the heatsink blocks them,” said Krishna.
Empower’s IVRs Set New Standard for Speed
Empower said the IVR integrates most of the input and output capacitors crowded around the processor so that it can deliver load steps to the processor significantly faster. That reduces the amount of voltage drop and how long it takes to recover from large currents rushing into the load.
As it features up to 100X higher frequency operation than other technologies, the IVR gives you the ability to tighten the voltage margining budget of the processor, resulting in power savings of 10% to 20%, according to the company.
The IVR also opens the door to more advanced dynamic voltage scaling (DVS) schemes, said Empower. Leveraging its faster switching speeds, the IVR can turn the supply voltage up or down when the processor demands it in nanoseconds instead of microseconds, reducing the power drained by the processor, explained Krishna. The power savings unlocked by more rapidly scaling the voltage can amount to more than 10% depending on the situation.
By integrating more of the magnetic and passive components in the system, Empower said IVRs can also be placed much closer to the load. The ideal location is directly under the SoC, as it shortens the power-delivery path as much as possible. That helps reduce impedance in the PDN, resulting in more power getting to the processor.
Krishna said that bringing the voltage regulator closer to the processor’s pins can curtail power losses on the PCB by a factor of 10, raising efficiency at the system level by up to 5%.
Empower’s latest family of power ICs, the EP71xx, integrates up to four voltage regulators in a compact 5- x 7- x 0.7-mm package that outputs a total current of 12 A. The multi-phase DC-DC converters run on a 3.3-V input with up to four independent outputs that can each supply part of the full current, delivering up to 27 W. Output voltages on each of the power rails are programmable from 0.5 to 2.5 V. The IVR also features ultra-fast transient response with a voltage drop of under 20 mV through a load step.
The EP71xx chips can be paired with everything from CPUs, GPUs, and SoCs to the DSPs inside pluggable connectors and active copper cables in data centers to a wide range of other networking, memory, storage, and wireless systems. Providing several outputs from a single IVR can also reduce the complexity of “power trees” used by FPGAs and other chips that feature several internal power domains, said Krishna.
On top of its fault protection and diagnostics, the IVRs add digital features including user-programmable power-up and power-down sequencing, user-configurable logic levels, and programmable I/O over the I2C interface.
While it is typically placed on the PCB, Empower is also mounting the EP71xx family of IVRs on a silicon interposer with magnetics and passives to create more of a complete voltage regulator module (VRM).
AI Power Delivery: Is Empower’s IVR the Answer?
Though not there yet, Empower said it’s working its way up to powering large AI chips and other high-performance (HPC) chips that push the limits of conventional power electronics and cooling.
The company is preparing to roll out another line of chips called Crescendo that can sling thousands of watts of power into big AI chips in the same class as NVIDIA’s Blackwell GPUs and Intel’s next-gen Gaudi. Integrating as much as possible into a single IVR not only saves space, but it also improves power efficiency by eliminating connection losses between the building blocks of the PDN as well as reducing EMI susceptibility.
Power delivery is bound to get even more complicated as more of the semiconductor industry moves into the age of chiplets. The complexities involved with routing power to all of the chiplets in 2.5D and 3D packages are causing a proliferation of external voltage converters, said Krishna. To address the problem, Empower rolled out a power “chiplet” based on its 1.8-V voltage regulator IC. The IVR can be placed in a package to handle the very last step of the power delivery, reducing parasitics and package ball count.
“We can actually be inside the package right next to the load,” noted Krishna, adding that it has also introduced a family of silicon chips called ECAPs that act as wide-bandwidth, ultra-low ESL capacitors. The ECAPs can be mounted inside the package with the IVR chiplet or outside it, depending on where they’re needed most.
“Instead of having multiple power rails running into the package, you only have one entering it,” said Krishna, before the IVRs inside feed the appropriate power to all of the different chiplets.
The inductors that must be paired with the IVR can be integrated directly into the package, embedded inside the PCB itself, or procured as off-the-shelf components and placed around the load on the PCB.
High-performance AI chips are already driving up power-per-rack specifications in the data center by 3X or more. And as systems engineers start to reevaluate how they deliver power to these huge chips, Empower could find itself in the right place at the right time with its IVR technology.