In the summer of 2006, the PCI Industrial Computer Manufacturers Group (PICMG) approved the MicroTCA telecommunications architecture, a small-form-factor platform addressing tight size and cost constraints. MicroTCA takes advantage of mezzanine cards developed for the AdvancedTCA platform by eliminating the need for carrier cards. What results is a scalable low-cost platform for building next-generation packet networks.
MicroTCA is the latest generation of open-architecture platforms developed by PICMG for telecom equipment design. The first telecom platform, AdvancedTCA, merged together a hot-swappable, multiprotocol, switched-fabric backplane with large form-factor cards and high-power capability. Such a combination made it possible to create high-density, high-performance telecom systems.
Then in 2005 we saw the arrival of AdvancedMC mezzanine modules, which enhanced the modularity of AdvancedTCA systems. Developers were able to create blades combining individually hot-swappable interface modules, mixing and matching functions as needed.
MicroTCA takes this modularity a step further. It leverages AdvancedMC modules to meet the needs of compact, low-cost systems by connecting AdvancedMC cards directly to a backplane without the need for a carrier card. By eliminating the Advanced Telecommunications Computing Architecture (ATCA) carrier, MicroTCA can offer a wide variety of form factors for designers to choose from, including custom “pico” assemblies for applications that need only a few modules for the entire system.
Understanding AdvancedMC Modules
Direct connection of AdvancedMC modules to a backplane implies that an understanding of MicroTCA must start with an examination of the AdvancedMC module itself. These modules employ a field-replaceable, serial packet interface that can have up to 20 I/O channels running as fast as 12.5 Gbits/s per channel. Because these channels are protocol-agnostic, they can support a variety of packet-oriented communications protocols, including Ethernet, PCIExpress, and Serial RapidIO (sRIO).
Physically, the modules offer several footprints as well as power capacity that tops out at 80 W (Fig. 1). AdvancedMC modules are hot-swappable, so they can be individually field-replaced without taking the system offline. In addition, they support Integrated Peripheral Management Interface (IPMI) system management, which enhances availability and serviceability by allowing shelf management to identify faults and take corrective action at the module level.
IPMI utilizes an I2C-based physical interface that enables the monitoring of system health characteristics, such as voltages, temperatures, and fan speeds. It also supports automatic event notification, remote shutdown and restart, and dynamic power allocation to individual AdvancedMC modules.
In MicroTCA, AdvancedMC modules (which ride on carrier cards in ATCA systems) plug directly into the backplane without requiring any modification. To replace the control functions of the ATCA carrier, MicroTCA uses a MicroTCA Carrier Hub (MCH). This module provides the switched fabric, shelf management, and optional clock distribution for a chassis. It acts as a fabric-star hub that offers one or more high-speed serial lanes to each module, along with a central switch for the lanes. One MCH module can support as many as 12 AdvancedMC modules in a system.
The backplane of a MicroTCA system can be designed to support star, dual-star, and mesh topologies. For high-availability system designs, the backplane can provide redundant IPMI interfaces, allowing two MCH modules to be employed in a single chassis. In addition, the backplanes support the connectivity for redundant power sources, the intelligence being provided by the power modules themselves.
The backplane and modules aren’t the only elements of a MicroTCA system, which consists of at least one AdvancedMC, at least one MCH, and all of the interconnect, power, cooling, and mechanical resources needed to support them (Fig. 2). These resources include the sub-rack, power modules, backplane, and cooling units. Together, the cards and their support resources form a shelf that can operate as a standalone system or be combined with other shelves to create a larger system.
MicroTCA Management
The management of a MicroTCA system has two levels: the shelf manager and the carrier manager. The shelf manager monitors overall system health, serving as an aggregation point for hardware information from one or more carrier managers. It also watches over other system units, identifying anomalous conditions and taking corrective action where possible. It manages the cooling units, ensuring the proper airflow through the shelf, by passing commands to the units through the carrier manager.
Furthermore, the shelf manager serves as the interface to any higher-level system-management functions required by the design. Thus, it provides overall system health information for such purposes as indicating status or triggering alarms. It may be implemented on its own AdvancedMC module in the system as a software application or as a function of the MCH.
The carrier manager, which is MicroTCA’s management workhorse, resides on the MCH in the MicroTCA Carrier Management Controller (MCMC). The carrier manager controls the AdvancedMC cards, the power modules, and the cooling units. It gathers status to convey to the shelf controller. It also handles the activation of individual modules. The carrier manager also interacts with module management controllers (MMC) within the AdvancedMC cards and the enhanced module management controllers (EMMC) on the power modules to manage system power distribution as well as hot-swap functions.
One important function of the carrier manager is to prevent incompatible devices from harming each other or disrupting communication in a MicroTCA system. Known as Electronic Keying (E-Keying), this function uses information from the AdvancedMC module, which indicates the capabilities of each port the module implements, and information within the carrier manager on the backplane’s connectivity.
By coordinating the information from these sources, the carrier manager decides which ports on each AdvancedMC switch fabric to enable or disable. It then communicates this information to the modules, ensuring that only compatible interfaces are enabled and controlling when the module is powered on.
Another carrier-manager function is to ensure the proper powering up and down of modules. Power modules within the MicroTCA system provide two feeds: 3.3-V management power and 12-V payload power, both under the control of the carrier manager. AdvancedMC modules use the management power feed to power the circuits needed for interacting with the carrier manager.
Subsequently, they use the payload power feed to power the rest of the module, including its fabric interfaces, after completing the E-Keying sequence. The carrier manager sends commands to the power modules, and they enable or disable the payload power feeds to each individual AdvancedMC module.
Managing System Power
The split power scheme allows the carrier manager to implement hot-swapping and prevents an AdvancedMC module from accessing the fabric lanes if the module is malfunctioning. When a module is to be powered up, whether at initial system startup or as the result of system insertion, the carrier manager first enables the module’s management power feed. Thus, the module can communicate with the carrier manager and initialize its I/O interfaces to a state compatible with the backplane’s fabric lanes. Once the module is fully configured, the carrier manager can then enable payload power.
Similarly, bringing a redundant module online can simply be a matter of enabling its payload power. The steps in reverse will gracefully power-down a module that’s tagged for removal. If there’s a malfunction, the carrier manager simply disables the module’s payload power, preventing it from interacting with the backplane fabric.
The carrier manager also manages any power-supply redundancy that’s present within the design. Redundant power supplies in a MicroTCA system connect through power switches (pass devices) to a “wired-OR” connection (Fig. 3). Normally, both power sources are enabled. In the event of a power-supply failure, however, the carrier manager can disconnect the failed supply from the system by disabling the corresponding pass device. The wired-OR connection ensures that current continues to flow during a failure and that the failed supply won’t load down the power system.
For the MicroTCA carrier manager to control system power appropriately, however, it’s essential for the AMC module to be designed to fully implement the IPMI control structure. Some early adopters of the ATCA architecture simplified the development of their custom AMC modules by neglecting the IPMI elements not needed on their carrier boards. What resulted were AMC modules that didn’t function correctly in a MicroTCA environment. To ensure proper operation in MicroTCA, developers of custom AMC modules should fully implement the elements of the AMC specification.
Flexible System Footprint
One significant advantage offered by MicroTCA is that it allows ATCA’s functionality to be scaled to very small configurations. The AdvancedMC module’s physical definitions have proven generous enough to allow the implementation of complex telecom functions in a single module, while small enough to allow cost-efficient design partitioning. By using those modules directly, without a carrier card, MicroTCA gives designers considerable flexibility in design reuse.
An ATCA server blade, for instance, might use a processor AMC connected via serial ATA (SATA) or serial attached SCSI (SAS) to a hard-disk AMC or via sRIO to an E1/T1 interface AMC. The carrier board provides the module interconnections, and the whole assembly forms a server blade that would connect with other blades over Gigabit Ethernet (GbE) to form a large network server system for a central-office installation.
Using MicroTCA, the same design can be scaled down to the equivalent of a single blade to meet the needs of a local-area network without implementing a full ATCA system. The processor AMC, E1/T1 AMC, and disk-drive AMC plug directly into the MicroTCA backplane. The MCH automatically establishes the GbE Interface between all of the modules and the rest of the network.
The MCH can also provide sRIO connectivity between the E1/T1 AMC and the processor AMC, so that the mini-server has connectivity to the telecom network. Finally, point-to-point links on the backplane provide the SATA/SAS connection between the processor AMC and the disk-drive AMC. But in some cases, a dedicated AMC controller may manage disk arrays.
This design could take advantage of MicroTCA’s “pico” shelf size, which allows for a minimum configuration of one or two AdvancedMC modules in a fully functional shelf. The pico size is well-suited to systems with modest performance needs and a minimum of available space. Other small-footprint configurations, including custom designs, are also possible within the MicroTCA specification.
Additionally, MicroTCA can target traditional rack-mount enclosure installations, offering several standard options (Fig. 4). Rack-mounted shelves can accept compact, mid, or full single or double modules in mixed configurations. A special configuration, the cube, provides additional flexibility. Cubes can be designed to fit together to occupy a rack width, while remaining independent functions provide a finer degree of modularity than a full rack-wide shelf.
These different system footprints let designers trade off size and system capability to create the optimum mix for their applications. The range of performance levels achievable using MicroTCA, as defined by backplane bandwidth, covers a host of applications. These include wireless basestations, digital loop carriers, optical add/drop multiplexers (ADMs), and fiber-to-the-curb optical network units (Fig. 5).
When designing their systems, though, developers will need to pay careful attention to airflow and heat. The board density achieved by MicroTCA, along with the power ratings supported by AMC boards, means that MicroTCA systems can easily develop hotspots. Design efforts should include a full thermal profile for the system, including evaluating board placement as it affects airflow.
In addition, designers creating custom AMC modules should follow the power limits suggested in the AMC specification and not try to run their boards “hot” to squeeze out additional performance. Variations from the specifications make it harder to obtain MicroTCA’s full benefits.
Reuse Equals Savings
MicroTCA increases the market opportunities for vendors’ AdvancedMC modules, which can reduce costs for system developers. Because the carrier- and shelf-management functions of a MicroTCA system replicate the management of the ATCA architecture, MicroTCA development efforts can apply AdvancedMC modules and their support software without modification.
By extending the applicability of AdvancedMC modules to smaller, lower-cost systems, MicroTCA enables vendors to enjoy greater production volumes and realize economies of scale that ultimately bring down prices. Similarly, developers of custom AMC modules for their own ATCA systems can enjoy the cost-reduction benefits of reusing their module designs to create a product family with a range of performance levels.
Many aspects of the MicroTCA and AdvancedMC standards serve to enhance this reuse. For example, the standards allow three different fitting types for AdvancedMC connectors: compression, surface mount, and press fit. A common footprint was proposed for each of these connector types, with the goal of providing multiple sources for backplane connectors. This will allow competition in that market space and permit the replacement of one connector manufacturer for another.
AdvancedMC modules offer physical compatibility despite individual differences in the module’s manufacturing process. As a result, by providing a common interface requirement, telecom equipment manufacturer (TEM) vendors can provide similar functions. This allows for multiple sourcing, which in turn lowers developer risk when applying the modules. As a byproduct, it encourages the adoption of AdvancedMC. This common interface ability is particularly important in markets where developers don’t want to depend on a single vendor.
The reuse of ATCA elements, along with the size and performance scalability of MicroTCA, now extends the range of ATCA to cover nearly all telecom applications. The original ATCA specification covers the larger end of system needs, while MicroTCA provides a more compact architecture that offers considerable design flexibility for smaller installations. At the same time, by leveraging the modular software and hardware developed for ATCA, MicroTCA keeps development costs down while maintaining the performance and reliability required by telecom applications.