Trapping and emulation of instructions

Sometimes a hypervisor needs to emulate operations within a Virtual Machine (VM).  For example, software within a VM might try to configure low level processor controls relating to power management or cache coherency. Typically, you do not want to give the VM direct access to these controls, because they could be used to break isolation, or to affect other VMs in your system.

A trap causes an exception when a given action, such as reading a register, is performed. A hypervisor needs the ability to trap operations, like the ones that configure low level controls, in a VM and emulate them, without affecting other VMs.

The architecture includes trap controls for you to trap operations in a VM and emulate them. When a trap is set, performing a specific action that would normally be allowed causes an exception to a higher Exception level. A hypervisor can use these traps to emulate operations within a VM.

For example, executing a Wait For Interrupt (WFI) instruction usually puts the CPU into a low power state. By asserting the TWI bit, if HCR_EL2.TWI==1, then executing WFI at EL0 or EL1 will instead cause an exception to EL2.

Note: Traps are not just for virtualization. There are EL3 and EL1 controlled traps as well.  However, traps are particularly useful to virtualization software. This guide only discusses the traps that are typically associated with virtualization.

In our WFI example, an OS would usually execute a WFI as part of an idle loop. With a Guest OS within a VM, the hypervisor can trap this operation and schedule a different vCPU instead, as this diagram shows:

Example of trapping WFIs from EL1 to EL2

Presenting virtual values of registers

Another example of using traps is to present virtual values of registers. For example, ID_AA64MMFR0_EL1 reports support for memory system-related features in the processor. An OS might read this register as part of boot, to determine which features within the kernel to enable. A hypervisor might want to present a different value, called a virtual value, to the Guest OS.

To do this, the hypervisor enables the trap that covers reads of the register. On a trap exception, the hypervisor determines which trap was triggered, and then emulates the operation. In this example, the hypervisor populates the destination register with the virtual value of ID_AA64MMFR0_EL1, as shown here:

Example of trapping and emulating an operationTraps can also be used as part of lazy context switching. For example, an OS will typically initialize the Memory Management Unit (MMU) configuration registers (TTBR<n>_EL1, TCR_EL1 and MAIR_EL1) during boot, and then will not reprogram them again. A hypervisor can use this to optimize its context switching routine, by only restoring the registers on a context switch and not saving them.

However, the OS might do something unusual and reprogram the registers after boot. To avoid this causing any problems, the hypervisor can set the HCR_EL2.TVM trap. This setting causes any write to the MMU related registers to generate a trap into EL2, which allows the hypervisor to detect whether it needs to update its saved copies of those registers.

Note: The architecture uses the terms trapping and routing for separate, but related, concepts. To recap, a trap causes an exception when a given action, such as reading a register, is performed. Routing refers to the Exception level that an exception is taken to once it has been generated.

MIDR and MPIDR

Using a trap to virtualize an operation requires significant computation. The operation generates a trap exception to EL2, and the hypervisor determines the desired operation, emulates it and then returns to the guest. Feature registers, such as ID_AA64MMFR0_EL1, are not frequently accessed by operating systems. This means that the computation is acceptable when trapping accesses to these registers into a hypervisor to emulate a read.

For registers that are accessed more frequently, or in performance critical code, you want to avoid such compute load. Examples of these registers and their values include:

  • MIDR_EL1. The type of processor, for example Cortex-A53
  • MPIDR_EL1. The affinity, for example core 1 of processor 2

A hypervisor might want a Guest OS to see the virtual values of these registers, without having to trap each individual access. For these registers, the architecture provides an alternative to trapping:

  • VPIDR_EL2. This is the value to return for EL1 reads of MIDR_EL1.
  • VMPIDR_EL2. This is the value to return for EL1 reads of MPIDR_EL1.

The hypervisor can setup these registers before entering the VM. If software running within the VM reads MIDR_EL1 or MPIDR_EL1, the hardware will automatically return the virtual value, without the need for a trap.

Note: VMPIDR_EL2 and VPIDR_EL2 do not have defined reset values. They must be initialized by start-up code before entering EL1 for the first time. This is especially important in bare metal environments.

Previous Next