What’s the Difference Between an Embedded Hypervisor and Separation Microkernel with Virtualization?

Hypervisors are used widely in enterprise servers, and the number of offerings in the embedded space continues to ramp up. Hypervisors are used for virtualization and provide some level of isolation, but they’re not the only option. Microkernels originated the embedded world, and a separation microkernel is specially designed for isolation and security. The same virtual-machine technology used in hypervisors can be added to a microkernel to provide a virtualization solution when needed.

Both solutions provide the ability to run multiple operating systems (OSes) in a virtualized environment, including mixing OS types. Over time, the two technologies have grown closer together, but some significant differences still exist in terms of latency, determinism, and security.

A hypervisor, also known as a virtual-machine monitor, is software designed to create and run virtual machines (VMs), each of which abstracts the hardware platform and runs a guest OS. The hypervisor is responsible for isolating each VM so that actions by one VM can’t compromise another (Fig. 1). In keeping with their enterprise server origins, hypervisors and VMs are heavyweight constructs that typically consume a great deal of resources in terms of the code base, memory footprint, and execution latency.

Hypervisors can be categorized as Type 1 or Type 2. Type 1 hypervisors run natively on the host hardware (i.e., “bare metal”), whereas Type 2 hypervisors run on top of a host OS. Type 1 hypervisors generally provide higher performance by eliminating one layer of software. That assumes, however, that all or most applications require virtualization. If only a small percentage of applications require virtualization, the overall system performance can be higher with a Type 2 hypervisor.

For example, if most applications run on a real-time OS (RTOS) and only one or two other applications need to run on Linux or Microsoft Windows, then the real-time applications can run on the host OS and aren’t burdened by going through a hypervisor. Only the applications that require virtualization need to pay the virtualization performance penalty.

A microkernel is an OS where the only essential services are implemented in kernel mode, and all other services are implemented mainly in user space, including device drivers, file systems, networking stacks, and virtualization. This leads to increased modularity, flexibility, and robustness, as well as a smaller, trusted compute base (TCB).

One specific type of microkernel is a separation kernel, which allocates all exported resources under its control into partitions, and those partitions are isolated except for explicitly allowed information flows. Separation kernels that are designed for the highest security meet the Separation Kernel Protection Profile (SKPP) defined by the U.S. National Security Agency (NSA), which was created for the most hostile threat environments.

A separation microkernel that includes a virtualization layer has some similarities to a Type 2 hypervisor in that the virtualization layer runs on top of a host OS and can be applied selectively to only the applications that require virtualization. However, it differs in that the isolation function is provided by the separation microkernel (in this case, the host OS), and isolation is enforced even between different instances of the virtualization layer (Fig. 2).

Virtualization Features

Both hypervisors and microkernels with a virtualization layer make use of hardware acceleration for full virtualization. On Intel processors, for example, that includes Intel VT-x, EPT, and Intel VT-d. Intel VT-x provides hardware instructions for entering and exiting a virtual execution mode where the guest OS sees itself as running with full privilege, while the host OS remains protected. Extended Page Tables (EPT) provide virtualization of the page tables that hold the mappings from physical to virtual memory. Intel VT-d provides a hardware assist for remapping direct-memory-access (DMA) transfers and device-generated interrupts. Even with those hardware-acceleration technologies and similar ones for other processors, a fair amount of virtualization is still done in software.

Because of their enterprise heritage, hypervisors sometimes provide a higher limit on the number of VMs per host and broader support of virtualization management features. Those advanced management capabilities might include dynamic resource allocation, failover, and live migration. Such capabilities can vary greatly by implementation.

Latency and Determinism

Before common microprocessors supported hardware acceleration of some virtualization functionality, full virtualization solutions were notoriously low performance. Although modern microprocessors provide a range of virtualization support, a fair amount of I/O virtualization still needs to be handled in software, such as device emulation, bus emulation, and interrupt emulation and routing.

In general, virtualization has lower latency and higher performance with a Type 1 hypervisor because it doesn’t have to also go through a host OS. With a Type 1 hypervisor, however, every application must pay that virtualization penalty. That includes real-time and safety-critical applications.

In contrast, a microkernel with a virtualization layer can have real-time applications running directly on the host OS while the non-real-time applications run on top of a virtualization layer. The virtualization performance penalty is larger for this approach, but it’s only paid by the non-real-time applications. In that manner, all hard real-time and safety-critical applications get the low latency and determinism inherent in the host RTOS.

Some hypervisors attempt to further reduce latency by giving an application the ability to run without a guest OS in “bare-metal” mode, but that’s a misnomer. Even if the application doesn’t run on a guest OS, it’s still running on top of the hypervisor. If the goal is to run a real-time, safety-critical, or security-critical application with lower latency, running without an OS isn’t the way to get there. Almost all of those types of target applications require tasking services, semaphores, or message passing—none of which are available in bare-metal mode.

Security

Security can be the area of most significant difference between a hypervisor and a separation microkernel with a virtualization layer. Although security can be the primary reason for considering a hypervisor, it’s a myth that hypervisors are inherently secure just because they utilize hardware features to enforce virtual address spaces and virtual I/O to isolate VMs.

It would be more accurate to say that the primary security protection provided by most hypervisors is simply whatever the underlying hardware supports through the MMU and IOMMU. First, that hardware security only covers some of what’s needed to isolate VMs, and second, that hardware security only helps isolate VMs and does nothing to make the hypervisor itself more secure. Because hypervisors run below the guest operating system, a compromised hypervisor isn’t detectable by the VM. Such exploits even have a catchy name: hyperjacking.

The likelihood of vulnerabilities generally is proportional to the size of the attack surface. Hypervisors have the virtualization software running in kernel mode, making it part of the trusted compute base (TCB), and that virtualization code can be huge. Almost every call from the guest OS to the kernel needs to be trapped, examined, and determined if the guest OS is permitted such access.

For a virtualization solution to be efficient, it needs to virtualize sequences of instructions instead of single instructions. Such look-ahead capability is just one example of increasing the already large code base of a hypervisor in pursuit of minimizing the virtualization performance penalty.

By definition, a microkernel is built with as little as possible running in kernel mode, so the virtualization layer isn’t part of the TCB. That doesn’t mean that the VMs are unprotected. Rather, the separation microkernel provides the isolation for every partition, whether virtualized or not. Even if a vulnerability occurs in the virtualization layer, such a vulnerability can’t spread even to another instance of the virtualization layer for another VM. In that way, a separation microkernel prevents hyperjacking.

Microkernels using separation kernel technology can have the highest levels of security and isolation. The proof of that security level is certification to the SKPP published by the NSA or similar security standards such as Common Criteria EAL6. Some hypervisors include some separation kernel principles to improve security, but no commercial hypervisor has been certified to the SKPP or Common Criteria EAL6.

An example of a separation microkernel with an optional virtualization layer is the INTEGRITY-178 tuMP RTOS from Green Hills Software. INTEGRITY-178 is a safety- and security-critical OS that has been certified by the National Information Assurance Partnership (NIAP) to meet the SKPP, High Robustness, as well as Common Criteria EAL6+. The INTEGRITY-178 tuMP RTOS extends that pedigree to multicore processors.

Its optional virtualization layer runs within partitions isolated by the separation kernel and allows one or more guest OSes to simultaneously run alongside safety- and security-critical functions on the same microprocessor. The result is a system that optimizes for safety, security, and real-time performance while enabling applications that require additional general-purpose or legacy OS support. 

Richard Jaenicke is the Director of Marketing for safety- and security-critical products at Green Hills Software.

References

John Rushby, "The Design and Verification of Secure Systems," Eighth ACM Symposium on Operating System Principles, pp. 12-21, Asilomar, CA, December 1981.

Information Assurance Directorate, National Security Agency, "U.S. Government Protection Profile for Separation Kernels in Environments Requiring High Robustness," Version 1.03, June 2007.