Fig 1. SeaMicro packs 768 64-bit Atom cores connected by a 1.28-Tbit/s fabric into its 10U SM1000-64HD system.
Fig 2. SeaMicro’s system employs a hypercube interconnect processor backplane surrounded by processor boards, storage boards, and network adapters.
The latest version of SeaMicro’s SM10000-64HD server uses 768 1.66-GHz Intel x86-64 cores (Fig. 1). That’s 768 Intel Atoms in a 10U rack (Fig. 2) drawing less than 3.5 kW along with up to 64 2.5-in. SATA solid-state or hard-disk drives and 160 Gbits/s of Ethernet links (64 by 1-Gbit/s or 160 by 10-Gbit/s links). It also uses a 1.28-Tbit/s fabric organized as a hypercube providing multiple paths between nodes.
Launched in 2010, the initial SM10000 was interesting and useful. It used some of the initial Intel Atom chips that were available to provide a great 32-bit environment of up to 512 cores. The latest Atom chips in the system offer 64-bit virtualization support that greatly alters the system’s capability to provide advanced cloud services.
The SM10000-64HD incorporates the same SeaMicro fabric ASIC chips as the original. These chips connect to the Atom chipset via PCI Express. The system supports 64 boards, and the latest processor boards contain dual-core Intel N570 processor chips.
Balancing Act
The SM10000-64HD is just one way of providing users with lots of cores. It’s a very common niche that’s easy to understand. It also meshes nicely with the virtualization trend, although it does not try to deliver dozens of cores in a symmetric multiprocessing (SMP) environment.
Instead, it works best where a couple of cores can handle a particular chore and provides the high-speed interconnect to link services. A larger computing environment would split the database and Web servers so they run on different nodes as requirements grow, though a typical LAMP (Linux, Apache, MySQL, PHP/Perl) configuration often will be placed on a single node.
The processor boards provide compute support only. Storage and networking with the outside world are part of the system, but the processors access these services via the fabric. This is transparent to users and allows SeaMicro to provide SAN-like (storage-area network) storage and managed network connections with load balancing and redundancy support as part of the package.
SeaMicro uses multiple FPGAs for the storage and network interfaces. The FPGAs include the fabric interface controller found on the processor board ASICs along with the support logic for the SATA or Ethernet interfaces.
A centralized management system handles all 384 processing nodes as well as the storage and networking system. This includes the ability to partition the system by nodes and to associate these partitions with storage and networking resources. It is even possible to configure a node without disk space where the application is employing data from another node such as a database server.
The SM10000-64HD is finding a home in Internet service provider (ISP) racks providing LAMP hosting and cloud services. It is also being used in some interesting supercomputing applications that are not being discussed in detail right now. These applications are taking advantage of the high-speed fabric to provide a very large, low-cost, low-power cluster. Its connection bandwidth is significantly higher and more efficient than Ethernet or even InfiniBand.
The SM10000-64HD will not replace larger SMP NUMA systems, but it will definitely beat out the competition in the space that SeaMicro is targeting.