SanDisk has been at the forefront of flash storage technology from its inception. It has had a range of consumer and embedded flash products and has moved into the enterprise space as well. This is where the non-volatile memory file system (NVMFS) comes into play.
Flash has always had a challenging time compared to RAM although it has done very well compared to hard disks as flash prices have fallen and capacities have grown. In the long past, from a technology standpoint, we had memories like magnetic core memory that delivered non-volatile main memory (see “Magnetic Cores To MRAM: Nonvolatile Tipping Point?”).
SRAM and DRAM are now the mainstays for a processor's primary memory but they are volatile. This means that software has to perform various programmatic gymnastics to get important data from the volatile memory to non-volatile storage. This has typically been done using block storage devices like hard disks and solid state disks (SSD).
There are now some DRAM products that combine DRAM and flash onto the same DIMM (see “Non-Volatile DIMMs and NVMe Spice Up The Flash Memory Summit”). They usually have a supercap that allows DRAM contents to be copied to on-board flash storage in the event of a power failure. The flash memory is read when the system is restarted. The challenge is packing enough of each type of memory onto the DIMM. Obviously it is less than having a DIMM of only one type or the other.
This approach incurs overhead in a number of areas from the disk controller to the software that has to partition transfers into blocks that the storage devices can handle. This often leads to software exercises such as double buffering and journaling to make sure that data can be consistently retrieved in the event of power loss either intentionally during a shutdown or accidentally.
Last year I gave one of our Best of Electronic Design awards to Diablo Technologies for their Memory Channel Storage (MCS) architecture (see “Large-Scale Flash Moves Next To The Microprocessor”). The DIMMs have only flash storage but a DDR DRAM interface. It eliminates the disk controller normally associated with flash storage and provides the highest speed interface to the flahs memory. SanDisk's UltraDIMM (Fig. 1) implements this MCS and others have incorporated it into their hardware such as IBM with its eXFlash DIMMs.
This file type includes high resolution graphics and schematics when applicable.
Diablo Technologies is adding a feature called NanoCommit (see “Memory Channel Flash Storage Provides Fast RAM Mirroring”). This is a software interface that tracks data written to DRAM and moves changes incrementally and in small blocks to flash.
Taking advantage of devices like UltraDIMM takes software and that is why this year SanDisk gets the honors for its non-volatile memory file system (NVMFS). NVMFS is another file system but it adds features like atomic writes (Fig. 2). ACID (Atomicity, Consistency, Isolation, Durability) is key to proper database operation and atomic writes are the basis of this support.
At the bottom of the stack is SanDisk's ioMemory and UltraDIMM's that provide the flash storage. The difference between a conventional file systems and NVMFS is that once data is written it is saved in non-volatile storage compared to a typical disk-based, double buffering approach (Fig. 3). The underlying system employs SanDisk's auto commit memory (ACM) that has byte-level granularity. This allows applications like Oracle's MySQL to use NVMFS without logging transactions that can cut performance in half.
Another advantage of this approach is the ability to implement NVM-Compression in NVMFS. It allows thin-provisioning of the underlying flash memory using pluggable compression algorithms. This eliminates MySQL bit packing and requires no rebalancing. Empty data within a file is unmapped using flash TRIM functions.
NVM-Compression can be implemented with minimal overhead especially compared to row compression (Fig. 4). Most SQL databases implement row compression where each row is compressed before it is written. NVM-Compression allows the row to be written without being compressed and the backend handles the compression for the database server.
NVMFS is part of the trend towards software-defined, multi-tier storage. It allows incorporation of a range of storage technologies into a hierarchy that delivers higher performance and capacity while limiting changes to applications.
Also check out OpenNVM. This is an open-source project that is defining new interfaces and system support such as flash-aware Linux swap support. There is also a native key-value interface to flash.
Flash storage technology is changing rapidly. Technologies like NVMFS provides significant benefits while changing how applications take advantage of non-volatile storage.
