PCI Express is absorbing functionality from other common interfaces like SAS and SATA, reducing the number of interfaces that systems will require. Ethernet has already hit that point, pushing out other networking alternatives. USB is effectively doing the same for low-speed peripherals. USB 3.0 is pushing the bandwidth envelope, but it may be challenged by PCI Express wherever high speed is critical. PCI Express scales. USB is limited to a single lane.
This is effectively what’s happening with SAS and SATA (see “The Cloud’s Infrastructure Needs More Than A Silver Lining”). These interfaces will exist for at least another decade. But SCSI Express and SATA Express, two PCI Express-based solutions, will replace their single-channel counterparts. SCSI Express and SATA Express can handle solid-state disks (SSDs), though they will be challenged by another PCI Express-based alternative.
NAND Flash Memory And PCI Express
NVM Express (NVMe) targets solid-state storage. It isn’t designed to be a general storage interface, and it isn’t designed to address hard-disk drives like SATA Express and SCSI Express (see “What’s The Difference Between SATA And NVMe?”). But it does compete with them because flash storage is so important. NVMe is also starting to show up while standards groups are still working on the other two.
The University of New Hampshire’s InterOperability Lab (UNH-IOL) is a neutral, third-party lab that’s tackling NVMe compliance testing. It’s set up to test a range of enterprise storage and networking technologies, and its work is timely because products are beginning to roll out like Integrated Device Technology’s (IDT) 16- and 32-channel NVMe controllers (see “Controllers Speed NVM Express Drive Delivery”).
IDT’s reference design highlights one of the new form factors that’s comparable to 2.5-in. drives, except the interface is PCI Express (see the figure). This particular interface also supports SAS and SATA, but they wouldn’t be utilized on the reference design since PCI Express is all that’s needed to handle NVMe. SATA is going to top out at 6 Gbits/s, and SAS is currently running at 12 Gbits/s. These speeds are more than adequate for hard drives, but flash memory can easily fill the bandwidth.
It may look like a standard solid-state disk, but IDT’s reference design highlights its 16-channel NVM Express controller. The connector has a PCI Express interface, not SAS or SATA.
NVMe’s PCI Express Gen 3 base runs at 8 Gbits/s per lane. The typical low-end connection for NVMe will be x4 lanes, which provides significantly more bandwidth than the single-lane storage interfaces. NVMe platforms with x8 and x16 interfaces provide even more bandwidth. PCI Express Gen 4 is on the horizon, promising even more bandwidth in the future.
Moving to a common interface like PCI Express makes sense now that interface and serializer-deserializer (SERDES) design is so challenging. SATA and SAS already use SERDES comparable to PCI Express, as do other interfaces like Serial RapidIO and InfiniBand.
PCI Express also simplifies virtualization and connectivity chores. NVMe is designed to support single-root I/O virtualization (SR-IOV). PCI Express switches are standard items that provide features like redundant host connections via non-transparent (NT) bridging as well as connectivity expansion. SAS and SATA define these functions as well, but they’re different and redundant as PCI Express becomes ubiquitous.
Ethernet: The Survivor
Anyone remember Arcnet or Token Ring? There were half a dozen network technologies, but Ethernet has outlived all of them. The big change was the migration from coax to twisted-pair cabling that’s now the backbone of homes and enterprises.
Compatibility is the key. Ethernet now spans speeds from 10 Mbits/s up to 100 Gbits/s, and a packet can traverse an Ethernet network from one speed to the other automatically. Even PCI Express doesn’t span that range because it doesn’t go that slow.
PCI Express won’t replace everything. USB is still the interface for keyboards and mice, but PCI Express could easily handle high-speed devices like hard drives and SSDs. Of course, PCI Express is usually the host for a USB root node.
So here’s to a device with a single external peripheral connection.
