64-bit Cortex Platform To Take On x86 Servers In The Cloud

Cavium is well known for delivering high performance, multicore network processors and accelerators (see “48-core MIPS64 Chip Accelerates Enterprise Networks” on electronicdesign.com). These platforms were based on Imagination Technologies MIPS processors and were heavily augmented with high speed network interfaces and accelerators.

Cavium's new ThunderX family (Fig. 1) is based on ARM's 64-bit, ARMv8 Cortex-A57 architecture (see “Delivering 64-Bit Arm Platforms” on electronicdesign.com). The CN8xxx family utilizes Cavium's chip technologies including a heavy mix of high performance network support including multiple 10/40/100 Gbit/s Ethernet connections.

Electronicdesign Com Sites Electronicdesign com Files Uploads 2014 06 111811 Fig1sm

Figure 1. Cavium's ThunderX platform packs in as many as 48 ARM Cortex-A57 cores running at 2.5 GHz.

Cavium's MIPS-based solutions, that are still available, were capable of running operating systems such as Linux and could easily support a cloud infrastructure but they were targeted at network acceleration, not cloud computation chores. On the other hand, the cloud is where ThunderX is designed to work. Cavium likes to indicate that its family is “workload optimized.”

To that end, Cavium has delivered four different platforms with slightly different targets application areas. The include:

ThunderX_CP is optimized for private and public cloud web servers providing services such as content delivery, web caching and search. They have 10/40 Gbit Ethernet support.
ThunderX_ST is optimized for storage. They can handle applications such as Hadoop, block and object storage, distributed file storage and hot/warm/cold storage type workloads. They have multiple SATA 3 controllers, 10/40 Gbit Ethernet ports and multiple PCI Express Gen3 ports. They also support Cavium's scalable fabric.
ThunderX_SC is optimized for secure web front-end, security appliances and cloud RAN (C-RAN) type workloads. They have 10/40 Gbit Ethernet ports, multiple PCIe Express Gen3 ports, plus hardware accelerators that include Cavium’s industry leading, 4th generation NITROX and TurboDPI technology with acceleration for IPSec, SSL, anti-virus, anti-malware, firewall and DPI services.
ThunderX_NT is optimized for media servers, scale-out embedded applications and NFV type workloads. It has support for 10/40/100 Gbit Ethernet ports as well as multiple PCI Express Gen3 ports. It incorporates Cavium's scalable fabric as well as support for bandwidth provisioning , QoS, traffic shaping and tunnel termination.

Each can have up to 48 cores running at 2.5 GHz that support Cavium's virtSOC virtualization technology that includes I/O virtualization support. The family supports dual socket solutions (Fig. 2) using Cavium Coherent Processor Interconnect (CCPI). This allows creation of nodes with up to 96 cores. The cores share cache, memory and peripherals in the same fashion as dual socket Intel Xeon and AMD Operaton cores. AMD has announced its single socket Opteron A1100 multicore chip based on the ARM Cortex-A57 (see “AMD ARMs 64-Bit Servers”). It has also announced its long term support for the ARMv8 architecture with its Project SkyBridge (see “AMD's Project SkyBridge Unifies x86 And 64-bit ARM”).

Electronicdesign Com Sites Electronicdesign com Files Uploads 2014 06 111811 Fig2sm

Figure 2. The Cavium Coherent Processor Interconnect (CCPI) can be used to link a pair of chips into a 96-core system.

The dual socket CCPI systems allow creation of many core nodes. Developers can expand the number of nodes using Cavium's scalable fabric available on some chips in the family. The fabric provides high speed connectivity in an XY mesh with connections to the four nearest neighbors in a 2D fabric. The architecture also supports 3D fabric connections. Nodes connected via the fabric do not share I/O or memory but they can deliver messages to applications anywhere in the fabric. This provides a way to build systems with thousands of nodes. Fabric monitoring and SLA features allow management and tracking of very large clusters.

The basic multicore framework has a 78 Kbyte L1 instruction cache, a 32 Kbyte L1 data cache and a shared 16 Mbyte L2 cache. Systems can have up to four DDR3/4, 72-bit DRAM memory controllers that handle 2400 MHz memory. A dual socket system supports up to 1 Tbyte of DRAM. Of course, the system is optimized for power utilization.

The family is further partitioned into the CN88xx and CN87xx chips. The CN88xx has 24 to 48 cores per chip and includes support for the dual socket and Ethernet fabric. They also have support for 100 Gbit Ethernet. The CN87xx has 8 to 16 cores and only has 10/40 Gbit Ethernet support. They also lack the dual socket and fabric support. They CN87xx targets lower end servers while the CN88xx targets cloud services.

ThunderX is a major departure for Cavium and adoption of the platform by some major players will be required for success. Cavium is poised to do well in this area given it track record in network chips since the players in that space are the same ones providing cloud services. ThunderX will not run Windows but it will run most other operating systems found in the cloud such as Linux.

Cavium is not alone in this space. AMD has already been mentioned but others have Cortex-A57 offerings targeting the cloud server space. Cavium does have an advantage when it comes to the number of cores, fabric and network support it has incorporated into its chips.