Fabric backplanes are rolling off production lines. Also, maturing technologies continue to move systems from evaluation to deployment, creating very large blade clusters. This one-two punch sums up today's ever-evolving field of interconnects.
However, the software still needs to catch up to the hardware. Once that occurs, processor networks will be able to tackle problems that were impossible in the past, as well as offer higher reliability plus higher performance.
Newer, faster technologies such as Advanced Switching Interconnect (ASI), 10-Gigabit Ethernet, InfiniBand, and RapidIO are displacing established fabric technologies like Star-Fabric and Mercury's Raceway++ (see "Fabric Acceptance," p. 69). These new technologies provide the bandwidth and expandability that's unattainable with bus technologies such as PCI, PCI-X, and VME64. They also offer more expansion and flexibility than their replacement, PCI Express.
Full mesh systems yield excellent performance, but the increasing number of nodes makes them increasingly difficult to support. Cabling can be a nightmare due to multiple racks as well. A common compromise is the dual-star configuration usually employed in fabric backplanes for standard form factors such as VME, AdvancedTCA, MicroTCA, and CompactPCI (see the figure).
No single fabric architecture has won the market yet. Some fill certain niches. InfiniBand dominates supercomputer clustering. PCI Express, which has been pushing fabrics, can deliver the bandwidth necessary to feed fabrics without overloading the host. Its cousin ASI uses the same underlying hardware, but ASI has its own protocol.
ASI
Based on PCI Express, ASI benefits from PCI Express' success. The protocols for ASI and PCI Express differ, but the hardware interface remains consistent. ASI's ability to tunnel PCI Express now comes in handy as ASI emerges from the lab into the real world.
ASI trails Ethernet, InfiniBand, and RapidIO in deployment. That could change this year, though, as ASI chips start to ship en masse. It appeals to users who are now familiar with PCI Express. The latter's success is building more confidence in the ASI camp.
10G ETHERNET
Ethernet runs the world, and 10-Gigabit Ethernet (10G) pumps data into and out of clusters attached to the Internet very well. Still, it has a tough road to travel against the competition because its compatibility is both a blessing and a curse.
First out of the chute for Ethernet fabrics was 1-Gigabit Ethernet, and it's been the mainstay for the past few years. But it's only a matter of time before 10G Ethernet supersedes it. Both provide direct connections as corporate and Internet Ethernet backbones. Bandwidth is the primary reason for fabric-based solutions, and 10G offers more than its older sibling. IT managers are comfortable with Ethernet, and 10G is just faster, right?
Ethernet's TCP/IP overhead is becoming its biggest liability. To combat that problem, TCP/IP offload engines (TOEs) or faster hosts were developed. They're sure to make a big difference in 10G, which is saddled with much more overhead. Consequently, high-end network attached storage (NAS) and storage-area networks (SANs) can take advantage of 10G's capacity. On top of that, interest in iSCSI continues to rise, which will require TOEs to keep up at 10G speeds.
InfiniBand will compete with 10G, since both target compute clusters. InfiniBand's trump card, remote direct memory access (RDMA), is being put into the Ethernet deck. But it remains to be seen if Ethernet's overhead will be the edge Infini-Band needs to stay ahead.
INFINIBAND
It's back. InfiniBand got a bad rap between the initial hype and product shipment. Yet many designers who jumped ship are back in force now that InfiniBand has proven itself. All major switch vendors feature InfiniBand products in their catalogs. It's a definite change from two years ago.
Mellanox's delivery of double-datarate (DDR) InfiniBand, which pumps out 60 Gbits/s per connection, puts Infini-Band on top of the performance heap. Add RDMA support and extremely low latency, and it's the clear winner for super clusters. InfiniBand chips also are inexpensive and use little power.
Fabric backplanes are rolling off production lines. Also, maturing technologies continue to move systems from evaluation to deployment, creating very large blade clusters. This one-two punch sums up today's ever-evolving field of interconnects.
However, the software still needs to catch up to the hardware. Once that occurs, processor networks will be able to tackle problems that were impossible in the past, as well as offer higher reliability plus higher performance.
Newer, faster technologies such as Advanced Switching Interconnect (ASI), 10-Gigabit Ethernet, InfiniBand, and RapidIO are displacing established fabric technologies like Star-Fabric and Mercury's Raceway++ (see "Fabric Acceptance," p. 69). These new technologies provide the bandwidth and expandability that's unattainable with bus technologies such as PCI, PCI-X, and VME64. They also offer more expansion and flexibility than their replacement, PCI Express.
Full mesh systems yield excellent performance, but the increasing number of nodes makes them increasingly difficult to support. Cabling can be a nightmare due to multiple racks as well. A common compromise is the dual-star configuration usually employed in fabric backplanes for standard form factors such as VME, AdvancedTCA, MicroTCA, and CompactPCI (see the figure).
No single fabric architecture has won the market yet. Some fill certain niches. InfiniBand dominates supercomputer clustering. PCI Express, which has been pushing fabrics, can deliver the bandwidth necessary to feed fabrics without overloading the host. Its cousin ASI uses the same underlying hardware, but ASI has its own protocol.
ASI
Based on PCI Express, ASI benefits from PCI Express' success. The protocols for ASI and PCI Express differ, but the hardware interface remains consistent. ASI's ability to tunnel PCI Express now comes in handy as ASI emerges from the lab into the real world.
ASI trails Ethernet, InfiniBand, and RapidIO in deployment. That could change this year, though, as ASI chips start to ship en masse. It appeals to users who are now familiar with PCI Express. The latter's success is building more confidence in the ASI camp.
10G ETHERNET
Ethernet runs the world, and 10-Gigabit Ethernet (10G) pumps data into and out of clusters attached to the Internet very well. Still, it has a tough road to travel against the competition because its compatibility is both a blessing and a curse.
First out of the chute for Ethernet fabrics was 1-Gigabit Ethernet, and it's been the mainstay for the past few years. But it's only a matter of time before 10G Ethernet supersedes it. Both provide direct connections as corporate and Internet Ethernet backbones. Bandwidth is the primary reason for fabric-based solutions, and 10G offers more than its older sibling. IT managers are comfortable with Ethernet, and 10G is just faster, right?
Ethernet's TCP/IP overhead is becoming its biggest liability. To combat that problem, TCP/IP offload engines (TOEs) or faster hosts were developed. They're sure to make a big difference in 10G, which is saddled with much more overhead. Consequently, high-end network attached storage (NAS) and storage-area networks (SANs) can take advantage of 10G's capacity. On top of that, interest in iSCSI continues to rise, which will require TOEs to keep up at 10G speeds.
InfiniBand will compete with 10G, since both target compute clusters. InfiniBand's trump card, remote direct memory access (RDMA), is being put into the Ethernet deck. But it remains to be seen if Ethernet's overhead will be the edge Infini-Band needs to stay ahead.
INFINIBAND
It's back. InfiniBand got a bad rap between the initial hype and product shipment. Yet many designers who jumped ship are back in force now that InfiniBand has proven itself. All major switch vendors feature InfiniBand products in their catalogs. It's a definite change from two years ago.
Mellanox's delivery of double-datarate (DDR) InfiniBand, which pumps out 60 Gbits/s per connection, puts Infini-Band on top of the performance heap. Add RDMA support and extremely low latency, and it's the clear winner for super clusters. InfiniBand chips also are inexpensive and use little power.