A more complicated virtual address space

The example in this section shows a more complex set of mappings. Like with the single-level table at EL3 and multiple level table examples, this example runs at EL3.

So far, the examples that we have seen map a small number of blocks. For a simple test image, this might be enough. However, in a larger system we want to map more of the resources of the systems, and map these resources at a finer grain. 

In the examples package, the files are in:  <example dir>\el3_stage1_full_mem_map\.

System physical address map

The example targets the Base Platform Model from Arm. The address map of the Base Platform model it typical of a modern Arm-based SoC and is summarized in the following table:

Physical address Component Secure or Non-secure Attributes that the example assigns
0x0000_0000 to 0x03FF_FFFF Trusted ROM Secure Normal, Cacheable, Shareable, Read-only, Executable
0x0400_0000 to 0x05FF_FFFF - - Fault
0x0600_0000 to 0x07FF_FFFF Trusted DRAM Secure Normal, Cacheable, Shareable, Read/Write, Executable
0x0800_0000 to 0x0FFF_FFFF Flash Non-secure Normal, Cacheable, Shareable, Read-only, XN*
0x1000_0000 to 0x19FF_FFFF - - Fault
0x1A00_0000 to 0x7FFF_FFFF Peripherals Secure Device-nGnRnE, Read/Write, XN
0x8000_0000 to 0xFFFF_FFFF DRAM Non-secure** Normal, Cacheable, Shareable, Read/Write, XN*

* These are Non-secure memories. This example runs in EL3, which is part of Secure state. Typically, we want to prevent execution from Non-secure locations while in Secure state. This can also be prevented using SCR_EL3.CIF.

** The FVP model that we are using can be configured so that the DRAM is either Non-secure, or both Secure and Non-secure. In the previous examples, we configured the model to allow both. In this example, we configure the model to allow only Non-secure accesses, which is a more realistic configuration.

The following diagram shows what the preceding memory map might look like as a set of MMU mappings:

Set the first level of translation

The address map that we just described is 4GB (0x0..0xFFFF_FFFF), or 32-bits, in total. As described in Configure the translation regime, the size of the virtual address space is specified as 64-T0SZ. The example sets TCR_EL3.T0SZ to 32, to give a 32-bit virtual address space. This is shown in the following code: 

  // Set up TCR_EL3
  // ---------------
  MOV      x0, #32                // T0SZ=0b011001 Limits VA space to 32 bits
  ORR      x0, x0, #(0x1 << 8)    // IGRN0=0b01    Walks to TTBR0 are Inner WB/WA
  ORR      x0, x0, #(0x1 << 10)   // OGRN0=0b01    Walks to TTBR0 are Outer WB/WA
  ORR      x0, x0, #(0x3 << 12)   // SH0=0b11      Inner Shareable
                                  // TBI0=0b0      Top byte not ignored
                                  // TG0=0b00      4KB granule
                                  // IPS=0         32-bit PA space
  MSR      TCR_EL3, x0

With a 4KB granule, 4GB is too big for a single level 2 table. Remember that each level 2 table covers 1GB. Therefore, the starting level of translation is level 1. Each entry in the level 1 table covers 1GB of virtual address space. Because we have configured a 4GB address space, the level 1 table in this example only requires four entries. We do not need to provide memory, or values, for the other entries.

In the previous examples, we always had a full (512 entry) level 1 table. A translation table must be size aligned. In the previous examples, this meant that the table had to 4KB aligned, with 512 entries * 8 bytes per entry. In this example, the alignment requirement is only 32-byte aligned, with 4 entries * 8 bytes per entry. However, for simplicity the example still allocates a 4KB aligned table.

The code below shows the reserving of memory for the level 1 and level 2 tables:

.align 12

  .global tt_l1_base
tt_l1_base:
  .fill 32 , 1 , 0

  .align 12
  .global tt_l2_base
tt_l2_base:
  .fill 4096 , 1 , 0
Level 1 table

Here is the level 1 table that is generated for the example:

  //
  // Generate L1 table
  //
  
  LDR      x1, =tt_l1_base             // Address of L1 table

  // [0]: 0x0000,0000 - 0x3FFF,FFFF
  LDR      x2, =tt_l2_base             // Get address of L2 table
  LDR      x0, =TT_S1_TABLE            // Entry template for pointer to next level table
  ORR      x0, x0, x2                  // Combine template with L2 table Base address
  STR      x0, [x1]
  
  // [1]: 0x4000,0000 - 0x7FFF,FFFF
  LDR      x0, =TT_S1_DEVICE_nGnRnE    // Entry template
                                       // AP=0, RW
  ORR      x0, x0, #0x40000000         // 'OR' template with base physical address
  STR      x0, [x1, #8]

  // [2]: 0x8000,0000 - 0xBFFF,FFFF (DRAM on the VE and Base Platform)
  LDR      x0, =TT_S1_NORMAL_WBWA      // Entry template
  ORR      x0, x0, #TT_S1_INNER_SHARED // 'OR' with inner-shareable attribute
  ORR      x0, x0, #TT_S1_NS           // 'OR' with NS==1
  ORR      x0, x0, #TT_S1_PXN          // 'OR' with XN==1
                                       // AP=0, RW
  ORR      x0, x0, #0x80000000         // 'OR' template with base physical address
  STR      x0, [x1, #16]
  
  // [3]: 0xC000,0000 - 0xFFFF,FFFF (DRAM on the VE and Base Platform)
  LDR      x0, =TT_S1_NORMAL_WBWA      // Entry template
  ORR      x0, x0, #TT_S1_INNER_SHARED // 'OR' with inner-shareable attribute
  ORR      x0, x0, #TT_S1_NS           // 'OR' with NS==1
  ORR      x0, x0, #TT_S1_PXN          // 'OR' with XN==1
                                       // AP=0, RW
  ORR      x0, x0, #0xC0000000         // 'OR' template with base physical address
  STR      x0, [x1, #24]

The level 1 table has only has only four entries. This is because we have a 4GB virtual address space and each entry in this table covers 1GB.

For the first 1GB of the virtual address space, 0x0000_0000 to 0x3FFF_FFFF, we need to describe multiple regions. Therefore, the entry in the level 1 table must point to level 2 table, where we make more granular mappings.

For the next three entries, we use 1GB blocks. Although we can use smaller blocks, larger blocks are more efficient. This is because larger blocks require less memory for the translation tables, and it means that the TLB entries covers more addresses. 

Level 2 table

Here is the level 2 table that is created for the first 1GB of the virtual address space:

//
  // Generate L2 table
  //
  …
  LDR      x1, =tt_l2_base              // Address of L1 table

  // [0..31]: 0x0000,0000 - 0x03FF,FFFF (Trusted Boot ROM)
  LDR      x0, =TT_S1_NORMAL_WBWA       // Entry template
  ORR      x0, x0, #TT_S1_INNER_SHARED  // 'OR' with inner-shareable attribute
  ORR      x0, x0, #TT_S1_PRIV_RO       // 'OR' in Read-only
  ORR      x0, x0, xzr                  // 'OR' template with base physical address
  MOV      x2, #32
1:
  STR      x0, [x1], #8
  ADD      x0, x0, #0x200000            // Increment the physical address field
  SUB      x2, x2, #1
  CBNZ     x2, 1b
  
  // [32..47]: 0x0400,0000 - 0x05FF,FFFF (Fault)
  LDR      x0, =TT_S1_FAULT             // Entry template
  ORR      x0, x0, #0x04000000          // 'OR' template with base physical address
  MOV      x2, #16
1:
  STR      x0, [x1], #8
  ADD      x0, x0, #0x200000            // Increment the physical address field
  SUB      x2, x2, #1
  CBNZ     x2, 1b

  
  // [48..63]: 0x0600,0000 - 0x07FF,FFFF (Trusted DRAM)
  LDR      x0, =TT_S1_NORMAL_WBWA       // Entry template
  ORR      x0, x0, #TT_S1_INNER_SHARED  // 'OR' with inner-shareable attribute
                                        // RW
  ORR      x0, x0, #0x06000000          // 'OR' template with base physical address
  MOV      x2, #16
1:
  STR      x0, [x1], #8
  ADD      x0, x0, #0x200000            // Increment the physical address field
  SUB      x2, x2, #1
  CBNZ     x2, 1b
  
  // [64..127]: 0x0800,0000 - 0x0FFF,FFFF (Flash)
  LDR      x0, =TT_S1_NORMAL_WBWA       // Entry template
  ORR      x0, x0, #TT_S1_INNER_SHARED  // 'OR' with inner-shareable attribute
  ORR      x0, x0, #TT_S1_PRIV_RO       // 'OR' in Read-only
  ORR      x0, x0, #TT_S1_NS            // 'OR' with NS==1
  ORR      x0, x0, #TT_S1_PXN           // 'OR' with XN==1
  ORR      x0, x0, #0x08000000          // 'OR' template with base physical address
  MOV      x2, #64
1:
  STR      x0, [x1], #8
  ADD      x0, x0, #0x200000            // Increment the physical address field
  SUB      x2, x2, #1
  CBNZ     x2, 1b
 
 // [128..511]: 0x1000,0000 - 0x3FFF,FFFF (Fault)
Previous