Virtualization Host Extensions
The following diagram shows a simplified version of the software stack and Exception level that we looked at in the section on virtualizing exceptions:
You can see how a standalone hypervisor maps to the Arm Exception levels. The hypervisor is running at EL2 and the virtual machines (VMs) at EL0/1. This situation is more problematic for hosted hypervisors, as shown in the following diagram:
Traditionally, kernels run at EL1, but the virtualization controls are in EL2. This means that most of the Host OS is at EL1, with some stub code running in EL2 to access the virtualization controls. This arrangement can be inefficient, because it may involve additional context switching.
The Virtualization Host Extensions (VHE) were introduced in Armv8.1-A. VHE aims to allow hosted hypervisors to run efficiently on Arm platforms, by having the kernel of the Host OS run at EL2 instead of EL1. VHE also aims to allow the same kernel binary to run at EL2 as a Host OS, or at EL1 as a Guest OS. The kernel binary can be mostly unmodified between the two cases, and only a few lines of code are needed to handle the distinction during the early boot phase.
Note: The DynamIQ processors (Cortex-A55, Cortex-A75 and Cortex-A76) support VHE.
Running the Host OS at EL2
VHE is controlled by two bits in
HCR_EL2. These bits can be summarized as:
E2H: Controls whether VHE is enabled.
TGE: When VHE is enabled, controls whether EL0 is Guest or Host.
The following table summarizes the typical settings:
|Guest kernel (EL1)||1||0|
|Guest application (EL0)||1||0|
|Host kernel (EL2)||1||1*|
|Host application (EL0)||1||1|
* On an exception that exits from a VM into the hypervisor,
TGE would initially be 0. Software would have to set the bit before running the main part of the host kernel.
You can see these typical settings in the following diagram:
Virtual address space
The following diagram shows what the virtual address spaces of EL0/EL1 looked like before VHE was introduced:
As discussed in Memory Management, EL0/1 has two regions. By convention, the upper region is referred to as kernel space, and the lower region is referred to as user space. However, EL2 only has a single region at the bottom of the address range. This difference is because, traditionally, a hypervisor would not host applications. This means that the hypervisor does not need a split between kernel space and user space.
Note: The allocation of kernel space to the upper region, and user space to the lower region, is simply a convention. It is not mandated by the Arm architecture.
The EL0/1 virtual address space also supports Address Space Identifiers (ASID), but EL2 does not. This is because the hypervisor would not usually host applications.
To allow our Host OS to execute efficiently in EL2, we need to add the second region and ASID support. Setting
HCR_EL2.E2H addresses these issues, as you can see in the following diagram:
While in EL0,
HCR_EL2.TGE controls which virtual address space is used: either the EL1 space or the EL2 space. Which space is used depends on whether the application is running under the Host OS (
TGE==1) or the Guest OS (
Re-directing register accesses
We saw in the section on Virtualizing generic timers that enabling VHE changes the layout of the EL2 virtual address space. However, we still have a problem with the configuration of the MMU. This is because our kernel will try to access _EL1 registers, such as
TTBR0_EL1, rather than _EL2 registers such as
To run the same binary at EL2, we need to redirect the accesses from the EL1 registers to the EL2 equivalents. Setting E2H will do this, so that accesses to _EL1 system registers are redirected to their EL2 equivalents. This redirection illustrated in the following diagram:
However, this redirection leaves us with a new problem. A hypervisor still needs access to the real
_EL1 registers, so that it can implement task switching. To resolve this, a new set of register aliases are introduced with an
_EL02 suffix. When used at EL2, with
E2H==1, these give access to the EL1 register for context switching. You can see this in the following diagram:
HCR_EL2.IMO/FMO/AMO bits control whether physical exceptions are routed to EL1 or EL2. When executing in EL0 with
TGE ==1, all physical exceptions are routed to EL2, unless they are routed to EL3 by
SCR_EL3. This is the case regardless of the actual values of the HCR_EL2 routing bits. This is because the application is executing as a child of the Host OS, and not a Guest OS. Therefore, any exceptions should be routed to the Host OS that is running in EL2.