It seems like nonvolatile dual-inline memory modules (NVDIMMs) have suddenly lurched into the limelight after several years of being relegated to small niches. This article, excerpted from a new report by Objective Analysis, will cover the basics of NVDIMMs, reveal why they’ve spiked in popularity of late, and show how they will change the face of computer storage.
An NVDIMM Primer
If you’re already knee-deep in NVDIMMs, you can skip this part. However, we find that many members of the computing community, no matter what their technical level, have never come across NVDIMM technology, or don’t understand either how an NVDIMM is made or why it’s used. We’ll provide answers to those questions.
The NVDIMM exists to leverage the speed of the DRAM bus and of memory accesses. This is because the memory channel is the fastest bus in the system. It’s the bus that the memory modules, DIMMs, plug into, and so far it’s been used exclusively for the processor to communicate with DRAM. But DRAM loses its contents when power is removed, and that’s why computers need storage—storage is where the computer keeps bits in the event of a power loss.
The conventional way to add nonvolatile, or “persistent1” memory to a system is to use the hard-drive interface either for an hard-disk drive (HDD), or more recently, for a solid-state disk (SSD). Both the hardware and the software that communicate through this path are slow since they were designed to operate at speeds that are sufficient for hard drives. Although a lot of work has been done over the past 10 years to improve the speed of the I/O channel, it’s still a burden to flash memory, and promises to be even more of a burden for tomorrow’s high-speed nonvolatile memories.
Fortunately, some creative people devised ways to add new memory technologies to the memory channel that maintain their contents even when power is lost for long periods of time. Because of their nonvolatile nature, these DIMMs have been given the name “NVDIMMs.”
One straightforward way to make a DIMM persistent is to keep the DIMM’s data in DRAM, just like any standard DIMM, but then add a NAND flash chip to copy the contents of the DRAM when power fails. This is the architecture used for today’s most prevalent NVDIMM type and is manufactured by a number of companies, including Cypress’ AgigA, Micron Technology, SMART Modular, and Viking Technology. A microcontroller is added to the DIMM to move the data from the DRAM to the NAND flash, which it does when power is lost. The controller also moves the NAND chip’s contents back into the DRAM when power is restored.
Since this data migration occurs after the system’s power has failed, the NVDIMM must have a source of power to support the backup process. This type of NVDIMM design includes its own small backup power supply, which is nothing more than a small bank of supercapacitors that are often bolted inside the server’s chassis and connected to the NVDIMM with a wire. Finally, the NVDIMM must isolate itself from the dying server’s memory bus when the data is being backed up, so buffers are added between the memory channel and the DRAM chips.
All of these additional chips and supercapacitors add significant cost to the NVDIMM. Still, many users are willing to pay for these more expensive DIMMs because a system that uses them reboots extremely quickly once power is restored.
A second kind of NVDIMM simply ties multiple SSDs to the memory channel through a bridge in a manner similar to the way PCIe RAID cards or HBAs (Host Bus Adapters) have been used tie a number of SSDs or HDDs to the PCIe bus. The memory bus provides the system architect with persistence at lower latency and higher bandwidth than is possible with SATA or PCIe SSDs, but the lack of an interrupt pin on the memory channel slows the processor’s communications with the NVDIMM, and appears to have limited this configuration’s success. This kind of NVDIMM was sold by SanDisk and IBM a few years ago under the names ULLtraDIMM and eXFlash DIMM, based on a bridge chip designed by Diablo Technology.