Virtualizing exceptions

Interrupts are used by hardware in the system to signal events to software. For example, a GPU might send an interrupt to signal that it has completed rendering a frame.

A system that uses virtualization is more complex. Some interrupts might be handled by the hypervisor itself. Other interrupts might come from devices allocated to a Virtual Machine (VM), and need to be handled by software within that VM. Also, the VM that is targeted by an interrupt might not be running at the time that the interrupt is received.

This means that you need mechanisms to support the handling of some interrupts in EL2 by the hypervisor. You also need mechanisms for forwarding other interrupts to a specific VM or specific Virtual CPU (vCPU) within a VM.

To enable these mechanisms, the architecture includes support for virtual interrupts: vIRQs, vFIQs, and vSErrors. These virtual interrupts behave like their physical counterparts (IRQs, FIQs, and SErrors), but can only be signaled while executing in EL0 and EL1. It is not possible to receive a virtual interrupt while executing in EL2 or EL3.

Note: To recap, support for virtualization in Secure state was introduced in Armv8.4-A. For a virtual interrupt to be signaled in Secure EL0/1, Secure EL2 needs to be supported and enabled. Otherwise virtual interrupts are not signaled in Secure state.

Enabling virtual interrupts

To signal virtual interrupts to EL0/1, a hypervisor must set the corresponding routing bit in HCR_EL2. For example, to enable vIRQ signaling, a hypervisor must set HCR_EL2.IMO.  This setting routes physical IRQ exceptions to EL2, and enables signaling of the virtual exception to EL1.

Virtual interrupts are controlled per interrupt type. In theory, a VM could be configured to receive physical FIQs and virtual IRQs. In practice, this is unusual. A VM is usually configured only to receive virtual interrupts.

Generating virtual interrupts

There are two mechanisms for generating virtual interrupts:

  1. Internally by the core, using controls in HCR_EL2.
  2. Using a GICv2, or later, interrupt controller.

Let's start with mechanism 1. There are three bits in HCR_EL2 that control virtual interrupt generation:

  • VI = Setting this bit registers a vIRQ.
  • VF = Setting this bit registers a vFIQ.
  • VSE = Setting this bit registers a vSError.

Setting one of these bits is equivalent to an interrupt controller asserting an interrupt signal into the vCPU.  The generated virtual interrupt is subject to PSTATE masking, just like a regular interrupt.

This mechanism is simple to use, but the disadvantage is that it only provides a way to generate the interrupt itself.  The hypervisor is then required to emulate the operation of the interrupt controller in the VM.  To recap, trapping and emulating operations in software involve overhead that is best avoided for frequent operations such as interrupts.

The second option is to use Arm's Generic Interrupt Controller (GIC) to generate virtual interrupts. From Arm GICv2, the GIC can signal both physical and virtual interrupts, by providing a physical CPU interface and a virtual CPU interface, as shown in the following diagram:

The GIC virtual and physical CPU interfaces 

These two interfaces are identical, except that one signals physical interrupts and the other one signals virtual interrupts. The hypervisor can map the virtual CPU interface into a VM, allowing software in that VM to communicate directly with the GIC. The advantage of this approach is that the hypervisor only needs to set up the virtual interface, and does not need to emulate it. This approach reduces the number of times that the execution needs to be trapped to EL2, and therefore reduces the overhead of virtualizing interrupts.

Note: Although Arm GICv2 can be used with Armv8-A designs, it is more common to see GICv3 or GICv4 used. 

Example of forwarding an interrupt to a vCPU

So far, we have looked at how virtual interrupts are enabled and generated. Let's see an example that shows the forwarding of a virtual interrupt to a vCPU. In this example, we will consider a physical peripheral that has been assigned to a VM, as shown in the following diagram:

Example sequence for forwarding a virtual interrupt 

The diagram illustrates these steps:

  1. The physical peripheral asserts its interrupt signal into the GIC.
  2. The GIC generates a physical interrupt exception, either IRQ or FIQ, which gets routed to EL2 by the configuration of HCR_EL2.IMO/FMO. The hypervisor identifies the peripheral and determines that it has been assigned to a VM. It checks which vCPU the interrupt should be forwarded to.
  3. The hypervisor configures the GIC to forward the physical interrupt as a virtual interrupt to the vCPU. The GIC will then assert the vIRQ or vFIQ signal, but the processor will ignore this signal while it is executing in EL2.
  4. The hypervisor returns control to the vCPU.
  5. Now that the processor is in the vCPU (EL0 or EL1), the virtual interrupt from the GIC can be taken. This virtual interrupt is subject to the PSTATE exception masks.

The example shows a physical interrupt being forwarded as a virtual interrupt. The example matches the assigned peripheral model described in the section on stage 2 translation.  For a virtual peripheral, a hypervisor can create a virtual interrupt without linking it to a physical interrupt.

Interrupt masking and virtual interrupts

In Exception Model, we introduce the interrupts mask bits in PSTATE, PSTATE.I for IRQs, PSTATE.F for FIQs and PSTATE.A for SErrors. When operating within a virtualized environment, these masks work in a slightly different way.

For example, for IRQs we have already seen that setting HCR_EL2.IMO does two things:

  • Routes physical IRQs to EL2
  • Enables signaling of vIRQs in EL0 and EL1

This setting also changes the way that the PSTATE.I mask is applied.  While in EL0 and EL1, if HCR_E2.IMO==1, PSTATE.I operates on vIRQs not pIRQs.

Previous Next