The Device memory type is used for describing peripherals, often referred to as Memory-Mapped I/O (MMIO). For our example address map, here we can see what would be typically be marked as Device:
To recap, the Normal memory type means that there are no side-effects to the access. For Device type memory, the opposite is true. The device memory type is used for locations that can have side-effects.
For example, a read to a FIFO would normally cause it to advance to the next piece of data. This means that the number of accesses to the FIFO is important, and therefore the processor must adhere to what is specified by the program.
Device regions are never cacheable, as it is very unlikely that you would ever want to cache accesses to a peripheral.
Speculative data accesses are not permitted to regions marked as Device. The processor can only access the location if it is architecturally accessed. That means that an instruction that has been architecturally executed has accessed the location.
Instructions should not be placed in regions marked as Device, and Arm recommends that Device regions are always marked as not executable. Otherwise it is possible that the processor might speculatively fetch instructions from it. Which could cause problems for read-sensitive devices like FIFOs.
Note: There is a subtle distinction here which is easy to miss. Marking a region as Device prevents speculative data accesses only. Marking a region as non-executable prevents speculative instruction accesses. So to prevent any speculative accesses, a region must be marked as both Device and non-executable.
Here are two examples:
- Device_GRE. This allows gathering, re-ordering, and early write acknowledgement.
- Device_nGnRnE. This does not allow gathering, re-ordering, and early write acknowledgement.
Of these optimizatons, we've already seen how re-ordering work but we've not yet introduced gathering or early write acknowledgement. Gathering allows memory accesses to similar locations to be merged into a single bus transaction, optimizing the access. Early write acknowledgement indicates to the memory system whether a buffer can send write acknowledgements at any point on the bus between the core and the peripheral at the address.
Note: Normal Non-cacheable and Device_GRE might appear at first glance to be the same, but this is not the case. Normal Non-cacheable still allows speculative data accesses, Device_GRE does not.
Does the processor really do something different for each type?
The memory type describes the set of allowable behaviors for a location. Taking just the Device type, we can represent the allowable behaviors as shown here:
Device_nGnRnE is the most restrictive sub-type, and has the fewest allowed behaviors. Device_GRE is the least restrictive, and therefore has the most allowed behaviors.
Importantly, all the behaviors allowed for Device_nGnRnE are also permitted for Device_GRE. As, for example, it is not a requirement for a Device_GRE memory to use Gathering – it is just allowed. Therefore, it would be permissible for the processor to treat Device_GRE as Device_nGnRnE.
This specific example is extreme and unlikely for A-class processors. But it is common for processors to not differentiate between all the types and sub-types, for example treating Device_GRE and Device_nGRE in the same way. This is only allowed if the type or sub-type is always made more restrictive.