1Foreword 2======== 3 4- This document describes the FF-A implementation from `[1]`_ for the 5 configuration where the SPMC resides at S-EL2 on platforms implementing the 6 FEAT_SEL2 architecture extension. 7- It is not an architecture specification and it might provide assumptions on 8 sections mandated as implementation-defined in the specification. 9- It covers the implications of TF-A used as a bootloader, and Hafnium used as a 10 reference code base for an SPMC. 11 12Terminology 13=========== 14 15- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 16 (or partitions) in the normal world. 17- The term SPMC refers to the S-EL2 component managing secure partitions in 18 the secure world when the FEAT_SEL2 architecture extension is implemented. 19- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 20 partition and implementing the FF-A ABI on platforms not implementing the 21 FEAT_SEL2 architecture extension. 22- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 23- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 24 25Sample reference stack 26====================== 27 28The following diagram illustrates a possible configuration when the 29FEAT_SEL2 architecture extension is implemented, showing the |SPMD| 30and |SPMC|, one or multiple secure partitions, with an optional 31Hypervisor: 32 33.. image:: ../resources/diagrams/Hafnium_overview_SPMD.png 34 35Integration with TF-A (Bootloader and SPMD) 36=========================================== 37 38The `TF-A project`_ provides the reference implementation for the secure monitor 39for Arm A class devices, executing at EL3. It includes the implementation of the 40|SPMD|, which manages the world-switch, to relay the FF-A calls to the |SPMC|. 41 42TF-A also serves as the system bootlader, and it was used in the reference 43implementation for the SPMC and SPs. 44SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 45Thus they are supplied as distinct signed entities within the FIP flash 46image. The FIP image itself is not signed hence this provides the ability 47to upgrade SPs in the field. 48 49TF-A build options 50------------------ 51 52This section explains the TF-A build options for an FF-A based SPM, in which SPMD 53is located at EL3. 54 55This is a step needed for integrating Hafnium as the S-EL2 SPMC and 56the TF-A as SPMD, together making the SPM component. 57 58- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 59 protocol from NWd to SWd back and forth. It is not possible to 60 enable another Secure Payload Dispatcher when this option is chosen. 61- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 62 level to being at S-EL2. It defaults to enabled (value 1) when 63 SPD=spmd is chosen.The context save/restore routine and exhaustive list 64 of registers is visible at `[4]`_. When set the reference software stack 65 assumes enablement of FEAT_PAuth, FEAT_BTI and FEAT_MTE architecture 66 extensions. 67- **SP_LAYOUT_FILE**: this option specifies a text description file 68 providing paths to SP binary images and manifests in DTS format 69 (see `Secure Partitions Layout File`_). It is required when ``SPMD_SPM_AT_SEL2`` 70 is enabled, i.e. when multiple secure partitions are to be loaded by BL2 on 71 behalf of the SPMC. 72- **BL32** option is re-purposed to specify the SPMC image. It can specify either 73 the Hafnium binary path (built for the secure world) or the path to a TEE 74 binary implementing FF-A interfaces. 75- **BL33** option to specify normal world loader such as U-Boot or the UEFI 76 framework payload, which would use FF-A calls during runtime to interact with 77 Hafnium as the SPMC. 78 79As a result of configuring ``SPD=spmd`` and ``SPMD_SPM_AT_SEL2`` TF-A provides 80context save/restore operations when entering/exiting an EL2 execution context. 81 82There are other build options that relate support other valid FF-A 83system configurations where the SPMC is implemented at S-EL1 and EL3. 84Note that they conflict with those needed to integrate with Hafnium as the SPMC. 85For more details refer to |TF-A| build options `[10]`_. 86 87Sample TF-A build command line when FEAT_SEL2 architecture extension is 88implemented and the SPMC is located at S-EL2, for Arm's FVP platform: 89 90.. code:: shell 91 92 make \ 93 CROSS_COMPILE=aarch64-none-elf- \ 94 PLAT=fvp \ 95 SPD=spmd \ 96 ARM_ARCH_MINOR=5 \ 97 BRANCH_PROTECTION=1 \ 98 ENABLE_FEAT_MTE2=1 \ 99 BL32=<path-to-hafnium-binary> \ 100 BL33=<path-to-bl33-binary> \ 101 SP_LAYOUT_FILE=sp_layout.json \ 102 all fip 103 104Sample TF-A build command line when FEAT_SEL2 architecture extension is 105implemented, the SPMC is located at S-EL2, and enabling secure boot: 106 107.. code:: shell 108 109 make \ 110 CROSS_COMPILE=aarch64-none-elf- \ 111 PLAT=fvp \ 112 SPD=spmd \ 113 ARM_ARCH_MINOR=5 \ 114 BRANCH_PROTECTION=1 \ 115 ENABLE_FEAT_MTE2=1 \ 116 BL32=<path-to-hafnium-binary> \ 117 BL33=<path-to-bl33-binary> \ 118 SP_LAYOUT_FILE=sp_layout.json \ 119 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 120 TRUSTED_BOARD_BOOT=1 \ 121 COT=dualroot \ 122 ARM_ROTPK_LOCATION=devel_rsa \ 123 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 124 GENERATE_COT=1 \ 125 all fip 126 127FVP model invocation 128-------------------- 129 130The FVP command line needs the following options to exercise the S-EL2 SPMC: 131 132+---------------------------------------------------+------------------------------------+ 133| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 134| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 135+---------------------------------------------------+------------------------------------+ 136| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 137| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 138| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 139| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 140| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 141| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 142| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 143| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 144+---------------------------------------------------+------------------------------------+ 145| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 146| - cluster1.has_branch_target_exception=1 | | 147+---------------------------------------------------+------------------------------------+ 148| - cluster0.has_pointer_authentication=2 | Implements FEAT_PAuth | 149| - cluster1.has_pointer_authentication=2 | | 150+---------------------------------------------------+------------------------------------+ 151| - cluster0.memory_tagging_support_level=2 | Implements FEAT_MTE2 | 152| - cluster1.memory_tagging_support_level=2 | | 153| - bp.dram_metadata.is_enabled=1 | | 154+---------------------------------------------------+------------------------------------+ 155 156Sample FVP command line invocation: 157 158.. code:: shell 159 160 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \ 161 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 162 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 163 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 164 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 165 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 166 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 \ 167 -C cluster0.has_pointer_authentication=2 -C cluster1.has_pointer_authentication=2 \ 168 -C cluster0.has_branch_target_exception=1 -C cluster1.has_branch_target_exception=1 \ 169 -C cluster0.memory_tagging_support_level=2 -C cluster1.memory_tagging_support_level=2 \ 170 -C bp.dram_metadata.is_enabled=1 \ 171 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B \ 172 -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 \ 173 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 \ 174 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 175 176SPMC Configuration 177================== 178 179This section details the configuration files required to deploy Hafnium as the SPMC, 180along with those required to configure each secure partion. 181 182SPMC Manifest 183------------- 184 185This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 186time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 187two different cases: 188 189The SPMC manifest is used by the SPMD to setup the environment required by the 190SPMC to run at S-EL2. SPs run at S-EL1 or S-EL0. 191 192.. code:: shell 193 194 attribute { 195 spmc_id = <0x8000>; 196 maj_ver = <0x1>; 197 min_ver = <0x1>; 198 exec_state = <0x0>; 199 load_address = <0x0 0x6000000>; 200 entrypoint = <0x0 0x6000000>; 201 binary_size = <0x60000>; 202 }; 203 204* *spmc_id* defines the endpoint ID value that SPMC can query through 205 ``FFA_ID_GET``. 206* *maj_ver/min_ver*. SPMD checks provided FF-A version versus its internal 207 version and aborts if not matching. 208* *exec_state* defines the SPMC execution state (AArch64 or AArch32). 209 Notice Hafnium used as a SPMC only supports AArch64. 210* *load_address* and *binary_size* are mostly used to verify secondary 211 entry points fit into the loaded binary image. 212* *entrypoint* defines the cold boot primary core entry point used by 213 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 214 215Other nodes in the manifest are consumed by Hafnium in the secure world. 216A sample can be found at `[7]`_: 217 218* The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 219 indicates a |FF-A| compliant SP. The *load_address* field specifies the load 220 address at which BL2 loaded the SP package. 221* The *cpus* node provides the platform topology and allows MPIDR to VMPIDR mapping. 222 Note the primary core is declared first, then secondary cores are declared 223 in reverse order. 224* The *memory* nodes provide platform information on the ranges of memory 225 available for use by SPs at runtime. These ranges relate to either 226 normal or device and secure or non-secure memory, depending on the *device_type* 227 field. The system integrator must exclude the memory used by other components 228 that are not SPs, such as the monitor, or the SPMC itself, the OS Kernel/Hypervisor, 229 NWd VMs, or peripherals that shall not be used by any of the SPs. The following are 230 the supported *device_type* fields: 231 232 * "memory": normal secure memory. 233 * "ns-memory": normal non-secure memory. 234 * "device-memory": device secure memory. 235 * "ns-device-memory": device non-secure memory. 236 237 The SPMC limits the SP's address space such that they can only refer to memory 238 inside of those ranges, either by defining memory region or device region nodes in 239 their manifest as well as memory starting at the load address until the limit 240 defined by the memory size. The SPMC also checks for overlaps between the regions. 241 Thus, the SPMC prevents rogue SPs from tampering with memory from other 242 components. 243 244.. code:: shell 245 246 memory@0 { 247 device_type = "memory"; 248 reg = <0x0 0x6000000 0x2000000 0x0 0xff000000 0x1000000>; 249 }; 250 251 memory@1 { 252 device_type = "ns-memory"; 253 reg = <0x0 0x90010000 0x70000000>; 254 }; 255 256 memory@2 { 257 device_type = "device-memory"; 258 reg = <0x0 0x1c090000 0x0 0x40000>, /* UART */ 259 <0x0 0x2bfe0000 0x0 0x20000>, /* SMMUv3TestEngine */ 260 <0x0 0x2a490000 0x0 0x20000>, /* SP805 Trusted Watchdog */ 261 <0x0 0x1c130000 0x0 0x10000>; /* Virtio block device */ 262 }; 263 264 memory@3 { 265 device_type = "ns-device-memory"; 266 reg = <0x0 0x1C1F0000 0x0 0x10000>; /* LCD */ 267 }; 268 269Above find an example representation of the referred memory description. The 270ranges are described in a list of unsigned 32-bit values, in which the first 271two addresses relate to the based physical address, followed by the respective 272page size. The first secure range defined in the node below has base address 273`0x0 0x6000000` and size `0x2000000`; following there is another range with 274base address `0x0 0xff000000` and size `0x1000000`. 275 276* The *interrupt-controller* node contains the address ranges of GICD and GICR 277so that non-contiguous GICR frames can be probed during boot flow. The GICD 278address is defined in the first cell, followed by the GICR addresses. 279"redistributor-regions" is used to define the number of GICR addresses. 280 281This node is optional. When absent, the default configuration assumes there is 282one redistributor region. The default GICD memory range is from ``GICD_BASE`` 283to ``GICD_BASE + GICD_SIZE``. The default GICR memory range is from 284``GICR_BASE`` to ``GICR_BASE + GICR_FRAMES * GIC_REDIST_SIZE_PER_PE``. 285 286.. code:: shell 287 288 gic: interrupt-controller@0x30000000 { 289 compatible = "arm,gic-v3"; 290 #address-cells = <2>; 291 #size-cells = <1>; 292 #redistributor-regions = <4>; 293 reg = <0x00 0x30000000 0x10000>, // GICD 294 <0x00 0x301C0000 0x400000>, // GICR 0: Chip 0 295 <0x10 0x301C0000 0x400000>, // GICR 1: Chip 1 296 <0x20 0x301C0000 0x400000>, // GICR 2: Chip 2 297 <0x30 0x301C0000 0x400000>; // GICR 3: Chip 3 298 }; 299 300The above is an example representation of the referred interrupt controller 301description. The cells are made up of three values. The first two 32-bit values 302make up a 64-bit value representing the address of the GIC redistributor. The 303third value represents the size of this region. In this example, 304redistributor-regions states there are 4 GICR cells. The address of GICR 0 is 305`0x00301C0000` and the size of that region is `0x400000`. 306 307Secure Partitions Configuration 308------------------------------- 309 310SP Manifests 311~~~~~~~~~~~~ 312 313An SP manifest describes SP attributes as defined in `[1]`_ 314(partition manifest at virtual FF-A instance) in DTS format. It is 315represented as a single file associated with the SP. A sample is 316provided by `[5]`_. A binding document is provided by `[6]`_. 317 318Platform topology 319~~~~~~~~~~~~~~~~~ 320 321The *execution-ctx-count* SP manifest field can take the value of one or the 322total number of PEs. The FF-A specification `[1]`_ recommends the 323following SP types: 324 325- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 326 implement the same number of ECs as the number of PEs in the platform. 327- Migratable UP SPs: a single execution context can run and be migrated on any 328 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 329 receive a direct message request originating from any physical core targeting 330 the single execution context. 331 332Secure Partition packages 333~~~~~~~~~~~~~~~~~~~~~~~~~ 334 335Secure partitions are bundled as independent package files consisting 336of: 337 338- a header 339- a DTB 340- an image payload 341 342The header starts with a magic value and offset values to SP DTB and 343image payload. Each SP package is loaded independently by BL2 loader 344and verified for authenticity and integrity. 345 346The SP package identified by its UUID (matching FF-A uuid property) is 347inserted as a single entry into the FIP at end of the TF-A build flow 348as shown: 349 350.. code:: shell 351 352 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 353 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 354 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 355 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 356 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 357 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 358 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 359 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 360 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 361 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 362 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 363 364.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 365 366Secure Partitions Layout File 367~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 368 369A json-formatted description file is passed to the build flow specifying paths 370to the SP binary image and associated DTS partition manifest file. The latter 371is processed by the dtc compiler to generate a DTB fed into the SP package. 372Each partition can be configured with the following fields: 373 374:code:`image` 375 - Specifies the filename and offset of the image within the SP package. 376 - Can be written as :code:`"image": { "file": "path", "offset": 0x1234 }` to 377 give both :code:`image.file` and :code:`image.offset` values explicitly, or 378 can be written as :code:`"image": "path"` to give :code:`image.file` and value 379 and leave :code:`image.offset` absent. 380 381 :code:`image.file` 382 - Specifies the filename of the image. 383 384 :code:`image.offset` 385 - Specifies the offset of the image within the SP package. 386 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC. 387 - Optional. Defaults to :code:`0x4000`. 388 389:code:`pm` 390 - Specifies the filename and offset of the partition manifest within the SP package. 391 - Can be written as :code:`"pm": { "file": "path", "offset": 0x1234 }` to 392 give both :code:`pm.file` and :code:`pm.offset` values explicitly, or 393 can be written as :code:`"pm": "path"` to give :code:`pm.file` and value 394 and leave :code:`pm.offset` absent. 395 396 :code:`pm.file` 397 - Specifies the filename of the partition manifest. 398 399 :code:`pm.offset` 400 - Specifies the offset of the partition manifest within the SP package. 401 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC. 402 - Optional. Defaults to :code:`0x1000`. 403 404:code:`image.offset` and :code:`pm.offset` can be leveraged to support SPs with 405S1 translation granules that differ from 4KB, and to configure the regions 406allocated within the SP package, as well as to comply with the requirements for 407the implementation of the boot information protocol (see `Passing boot data to 408the SP`_ for more details). 409 410:code:`owner` 411 - Specifies the SP owner, identifying the signing domain in case of dual root CoT. 412 - Possible values are :code:`SiP` (silicon owner) or :code:`Plat` (platform owner). 413 - Optional. Defaults to :code:`SiP`. 414 415:code:`uuid` 416 - Specifies the UUID of the partition. 417 - Optional. Defaults to the value of the :code:`uuid` field from the DTS partition manifest. 418 419:code:`physical-load-address` 420 - Specifies the :code:`load_address` field of the generated DTS fragment. 421 - Optional. Defaults to the value of the :code:`load-address` from the DTS partition manifest. 422 423.. code:: shell 424 425 { 426 "tee1" : { 427 "image": "tee1.bin", 428 "pm": "tee1.dts", 429 "owner": "SiP", 430 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 431 }, 432 433 "tee2" : { 434 "image": "tee2.bin", 435 "pm": "tee2.dts", 436 "owner": "Plat" 437 }, 438 439 "tee3" : { 440 "image": { 441 "file": "tee3.bin", 442 "offset":"0x2000" 443 }, 444 "pm": { 445 "file": "tee3.dts", 446 "offset":"0x6000" 447 }, 448 "owner": "Plat" 449 }, 450 } 451 452SPMC boot 453========= 454 455The SPMC is loaded by BL2 as the BL32 image. 456 457The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 458 459BL2 passes the SPMC manifest address to BL31 through a register. 460 461At boot time, the SPMD in BL31 runs from the primary core, initializes the core 462contexts and launches the SPMC (BL32) passing the following information through 463registers: 464 465- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 466- X1 holds the ``HW_CONFIG`` physical address. 467- X4 holds the currently running core linear id. 468 469Secure boot 470----------- 471 472The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 473SPMC manifest, secure partitions and verifies them for authenticity and integrity. 474Refer to TBBR specification `[3]`_. 475 476The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 477the use of two root keys namely S-ROTPK and NS-ROTPK: 478 479- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 480- BL33 may be signed by the OEM using NS-ROTPK. 481- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 482- A maximum of 4 partitions can be signed with the S-ROTPK key and 4 partitions 483 signed with the NS-ROTPK key. 484 485Also refer to `Secure Partitions Configuration`_ and `TF-A build options`_ sections. 486 487Boot phases 488----------- 489 490Primary core boot-up 491~~~~~~~~~~~~~~~~~~~~ 492 493Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 494core. The SPMC performs its platform initializations and registers the SPMC 495secondary physical core entry point physical address by the use of the 496`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 497at secure physical FF-A instance). 498 499The SPMC then creates secure partitions base on SP packages and manifests. Each 500secure partition is launched in sequence (`SP Boot order`_) on their "primary" 501execution context. If the primary boot physical core linear id is N, an MP SP is 502started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 503UP SP, it is started using its unique EC0 on PE[N]. 504 505The SP primary EC (or the EC used when the partition is booted as described 506above): 507 508- Performs the overall SP boot time initialization, and in case of a MP SP, 509 prepares the SP environment for other execution contexts. 510- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 511 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 512 entry point for other execution contexts. 513- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 514 ``FFA_ERROR`` in case of failure. 515 516Secondary cores boot-up 517~~~~~~~~~~~~~~~~~~~~~~~ 518 519Once the system is started and NWd brought up, a secondary physical core is 520woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 521calls into the SPMD on the newly woken up physical core. Then the SPMC is 522entered at the secondary physical core entry point. 523 524In the current implementation, the first SP is resumed on the coresponding EC 525(the virtual CPU which matches the physical core). The implication is that the 526first SP must be a MP SP. 527 528In a linux based system, once secure and normal worlds are booted but prior to 529a NWd FF-A driver has been loaded: 530 531- The first SP has initialized all its ECs in response to primary core boot up 532 (at system initialization) and secondary core boot up (as a result of linux 533 invoking PSCI_CPU_ON for all secondary cores). 534- Other SPs have their first execution context initialized as a result of secure 535 world initialization on the primary boot core. Other ECs for those SPs have to 536 be run first through ffa_run to complete their initialization (which results 537 in the EC completing with FFA_MSG_WAIT). 538 539Refer to `Power management`_ for further details. 540 541Loading of SPs 542-------------- 543 544At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 545below: 546 547.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 548 549Note this boot flow is an implementation sample on Arm's FVP platform. 550Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 551different boot flow. The flow restricts to a maximum of 8 secure partitions. 552 553SP Boot order 554~~~~~~~~~~~~~ 555 556SP manifests provide an optional boot order attribute meant to resolve 557dependencies such as an SP providing a service required to properly boot 558another SP. SPMC boots the SPs in accordance to the boot order attribute, 559lowest to the highest value. If the boot order attribute is absent from the FF-A 560manifest, the SP is treated as if it had the highest boot order value 561(i.e. lowest booting priority). The FF-A specification mandates this field 562is unique to each SP. 563 564It is possible for an SP to call into another SP through a direct request 565provided the latter SP has already been booted. 566 567Passing boot data to the SP 568~~~~~~~~~~~~~~~~~~~~~~~~~~~ 569 570In `[1]`_ , the section "Boot information protocol" defines a method for passing 571data to the SPs at boot time. It specifies the format for the boot information 572descriptor and boot information header structures, which describe the data to be 573exchanged between SPMC and SP. 574The specification also defines the types of data that can be passed. 575The aggregate of both the boot info structures and the data itself is designated 576the boot information blob, and is passed to a Partition as a contiguous memory 577region. 578 579Currently, the SPM implementation supports the FDT type which is used to pass the 580partition's DTB manifest. 581 582The region for the boot information blob is allocated through the SP package. 583 584.. image:: ../resources/diagrams/partition-package.png 585 586To adjust the space allocated for the boot information blob, the json description 587of the SP (see section `Secure Partitions Layout File`_) shall be updated to contain 588the manifest offset. If no offset is provided the manifest offset defaults to 0x1000, 589which is the page size in the Hafnium SPMC. 590 591The configuration of the boot protocol is done in the SPs manifest. As defined by 592the specification, the manifest field 'gp-register-num' configures the GP register 593which shall be used to pass the address to the partitions boot information blob when 594booting the partition. 595In addition, the Hafnium SPMC implementation requires the boot information arguments 596to be listed in a designated DT node: 597 598.. code:: shell 599 600 boot-info { 601 compatible = "arm,ffa-manifest-boot-info"; 602 ffa_manifest; 603 }; 604 605The whole secure partition package image (see `Secure Partition packages`_) is 606mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can 607retrieve the address for the boot information blob in the designated GP register, 608process the boot information header and descriptors, access its own manifest 609DTB blob and extract its partition manifest properties. 610 611SPMC Runtime 612============ 613 614Parsing SP partition manifests 615------------------------------ 616 617Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 618Note the current implementation may not implement all optional fields. 619 620The SP manifest may contain memory and device regions nodes: 621 622- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 623 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 624 specify RX/TX buffer regions in which case it is not necessary for an SP 625 to explicitly invoke the ``FFA_RXTX_MAP`` interface. The memory referred 626 shall be contained within the memory ranges defined in SPMC manifest. The 627 NS bit in the attributes field should be consistent with the security 628 state of the range that it relates to. I.e. non-secure memory shall be 629 part of a non-secure memory range, and secure memory shall be contained 630 in a secure memory range of a given platform. 631- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 632 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 633 additional resources (e.g. interrupts). 634 635For the SPMC, base addresses for memory and device region nodes are IPAs provided 636the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation regime. 637 638ote: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 639same set of page tables. It is still open whether two sets of page tables shall 640be provided per SP. The memory region node as defined in the specification 641provides a memory security attribute hinting to map either to the secure or 642non-secure EL1&0 Stage-2 table if it exists. 643 644Secure partitions scheduling 645---------------------------- 646 647The FF-A specification `[1]`_ provides two ways to allocate CPU cycles to 648secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 649 650- the FFA_MSG_SEND_DIRECT_REQ (or FFA_MSG_SEND_DIRECT_REQ2) interface. 651- the FFA_RUN interface. 652 653Additionally a secure interrupt can pre-empt the normal world execution and give 654CPU cycles by transitioning to EL3 and S-EL2. 655 656Mandatory interfaces 657-------------------- 658 659The following interfaces are exposed to SPs: 660 661- ``FFA_VERSION`` 662- ``FFA_FEATURES`` 663- ``FFA_RX_RELEASE`` 664- ``FFA_RXTX_MAP`` 665- ``FFA_RXTX_UNMAP`` 666- ``FFA_PARTITION_INFO_GET`` 667- ``FFA_ID_GET`` 668- ``FFA_MSG_WAIT`` 669- ``FFA_MSG_SEND_DIRECT_REQ`` 670- ``FFA_MSG_SEND_DIRECT_RESP`` 671- ``FFA_MEM_DONATE`` 672- ``FFA_MEM_LEND`` 673- ``FFA_MEM_SHARE`` 674- ``FFA_MEM_RETRIEVE_REQ`` 675- ``FFA_MEM_RETRIEVE_RESP`` 676- ``FFA_MEM_RELINQUISH`` 677- ``FFA_MEM_FRAG_RX`` 678- ``FFA_MEM_FRAG_TX`` 679- ``FFA_MEM_RECLAIM`` 680- ``FFA_RUN`` 681 682As part of the FF-A v1.1 support, the following interfaces were added: 683 684 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 685 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 686 - ``FFA_NOTIFICATION_BIND`` 687 - ``FFA_NOTIFICATION_UNBIND`` 688 - ``FFA_NOTIFICATION_SET`` 689 - ``FFA_NOTIFICATION_GET`` 690 - ``FFA_NOTIFICATION_INFO_GET`` 691 - ``FFA_SPM_ID_GET`` 692 - ``FFA_SECONDARY_EP_REGISTER`` 693 - ``FFA_MEM_PERM_GET`` 694 - ``FFA_MEM_PERM_SET`` 695 - ``FFA_MSG_SEND2`` 696 - ``FFA_RX_ACQUIRE`` 697 698As part of the FF-A v1.2 support, the following interfaces were added: 699 700- ``FFA_PARTITION_INFO_GET_REGS`` 701- ``FFA_MSG_SEND_DIRECT_REQ2`` 702- ``FFA_MSG_SEND_DIRECT_RESP2`` 703- ``FFA_CONSOLE_LOG`` 704 705FFA_VERSION 706~~~~~~~~~~~ 707 708``FFA_VERSION`` requires a *requested_version* parameter from the caller. 709The returned value depends on the caller: 710 711- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 712 specified in the SPMC manifest. 713- SP: the SPMC returns its own implemented version. 714- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 715 716FFA_FEATURES 717~~~~~~~~~~~~ 718 719FF-A features supported by the SPMC may be discovered by secure partitions at 720boot (that is prior to NWd is booted) or run-time. 721 722The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 723FFA_SUCCESS from the SPMD. 724 725S-EL1 partitions calling FFA_FEATURES at virtual FF-A instance with NPI and MEI 726interrupt feature IDs get FFA_SUCCESS. 727 728S-EL0 partitions are not supported for NPI: ``FFA_NOT_SUPPORTED`` will be 729returned. 730 731Physical FF-A instances are not supported for NPI and MEI: ``FFA_NOT_SUPPORTED`` 732will be returned. 733 734The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 735the response relayed back to the NWd. 736 737FFA_RXTX_MAP/FFA_RXTX_UNMAP 738~~~~~~~~~~~~~~~~~~~~~~~~~~~ 739 740When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 741receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 742regime as secure buffers in the MMU descriptors. 743 744When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 745SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 746descriptors. The provided addresses may be owned by a VM in the normal world, 747which is expected to receive messages from the secure world. The SPMC will in 748this case allocate internal state structures to facilitate RX buffer access 749synchronization (through FFA_RX_ACQUIRE interface), and to permit SPs to send 750messages. The addresses used must be contained in the SPMC manifest NS memory 751node (see `SPMC manifest`_). 752 753The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 754caller, either it being the Hypervisor or OS kernel, as well as a secure 755partition, and restores them in the VM's translation regime so that they can be 756used for memory sharing operations from the normal world again. 757 758The minimum and maximum buffer sizes supported by the FF-A instance can be 759queried by calling ``FFA_FEATURES`` with the ``FFA_RXTX_MAP`` function ID. 760 761FFA_PARTITION_INFO_GET 762~~~~~~~~~~~~~~~~~~~~~~ 763 764Partition info get call can originate: 765 766- from SP to SPMC 767- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 768 769FFA_PARTITION_INFO_GET_REGS 770~~~~~~~~~~~~~~~~~~~~~~~~~~~ 771 772This call can originate: 773 774- from SP to SPMC 775- from SPMC to SPMD 776- from Hypervsior or OS kernel to SPMC. The request is relayed by the SPMD. 777 778The primary use of this ABI is to return partition information via registers 779as opposed to via RX/TX buffers and is useful in cases where sharing memory is 780difficult. 781 782FFA_ID_GET 783~~~~~~~~~~ 784 785The FF-A id space is split into a non-secure space and secure space: 786 787- FF-A ID with bit 15 clear relates to VMs. 788- FF-A ID with bit 15 set related to SPs. 789- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 790 and SPMC. 791 792The SPMD returns: 793 794- The default zero value on invocation from the Hypervisor. 795- The ``spmc_id`` value specified in the SPMC manifest on invocation from 796 the SPMC (see `SPMC manifest`_) 797 798This convention helps the SPMC to determine the origin and destination worlds in 799an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 800transactions in its world switch routine. It must not be permitted for a VM to 801use a secure FF-A ID as origin world by spoofing: 802 803- A VM-to-SP direct request/response shall set the origin world to be non-secure 804 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 805 set). 806- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 807 for both origin and destination IDs. 808 809An incoming direct message request arriving at SPMD from NWd is forwarded to 810SPMC without a specific check. The SPMC is resumed through eret and "knows" the 811message is coming from normal world in this specific code path. Thus the origin 812endpoint ID must be checked by SPMC for being a normal world ID. 813 814An SP sending a direct message request must have bit 15 set in its origin 815endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 816 817The SPMC shall reject the direct message if the claimed world in origin endpoint 818ID is not consistent: 819 820- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 821 world ID", 822- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 823 824 825FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 826~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 827 828This is a mandatory interface for secure partitions consisting in direct request 829and responses with the following rules: 830 831- An SP can send a direct request to another SP. 832- An SP can receive a direct request from another SP. 833- An SP can send a direct response to another SP. 834- An SP cannot send a direct request to an Hypervisor or OS kernel. 835- An Hypervisor or OS kernel can send a direct request to an SP. 836- An SP can send a direct response to an Hypervisor or OS kernel. 837 838FFA_MSG_SEND_DIRECT_REQ2/FFA_MSG_SEND_DIRECT_RESP2 839~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 840 841The primary usage of these ABIs is to send a direct request to a specified 842UUID within an SP that has multiple UUIDs declared in its manifest. 843 844Secondarily, it can be used to send a direct request with an extended 845set of message payload arguments. 846 847FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 848~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 849 850The secure partitions notifications bitmap are statically allocated by the SPMC. 851Hence, this interface is not to be issued by secure partitions. 852 853At initialization, the SPMC is not aware of VMs/partitions deployed in the 854normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 855to be prepared to handle notifications for the provided VM ID. 856 857FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 858~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 859 860Pair of interfaces to manage permissions to signal notifications. Prior to 861handling notifications, an FF-A endpoint must allow a given sender to signal a 862bitmap of notifications. 863 864If the receiver doesn't have notification support enabled in its FF-A manifest, 865it won't be able to bind notifications, hence forbidding it to receive any 866notifications. 867 868FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 869~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 870 871FFA_NOTIFICATION_GET retrieves all pending global notifications and 872per-vCPU notifications targeted to the current vCPU. 873 874Hafnium maintains a global count of pending notifications which gets incremented 875and decremented when handling FFA_NOTIFICATION_SET and FFA_NOTIFICATION_GET 876respectively. A delayed SRI is triggered if the counter is non-zero when the 877SPMC returns to normal world. 878 879FFA_NOTIFICATION_INFO_GET 880~~~~~~~~~~~~~~~~~~~~~~~~~ 881 882Hafnium maintains a global count of pending notifications whose information 883has been retrieved by this interface. The count is incremented and decremented 884when handling FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET respectively. 885It also tracks notifications whose information has been retrieved individually, 886such that it avoids duplicating returned information for subsequent calls to 887FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 888reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 889 890FFA_SPM_ID_GET 891~~~~~~~~~~~~~~ 892 893Returns the FF-A ID allocated to an SPM component which can be one of SPMD 894or SPMC. 895 896At initialization, the SPMC queries the SPMD for the SPMC ID, using the 897FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using 898the FFA_SPM_ID_GET interface at the secure physical FF-A instance. 899 900Secure partitions call this interface at the virtual FF-A instance, to which 901the SPMC returns the priorly retrieved SPMC ID. 902 903The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the 904SPMD, which returns the SPMC ID. 905 906FFA_SECONDARY_EP_REGISTER 907~~~~~~~~~~~~~~~~~~~~~~~~~ 908 909When the SPMC boots, all secure partitions are initialized on their primary 910Execution Context. 911 912The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition 913from its first execution context, to provide the entry point address for 914secondary execution contexts. 915 916A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 917the NWd or by invocation of FFA_RUN. 918 919FFA_RX_ACQUIRE/FFA_RX_RELEASE 920~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 921 922The RX buffers can be used to pass information to an FF-A endpoint in the 923following scenarios: 924 925 - When it was targetted by a FFA_MSG_SEND2 invokation from another endpoint. 926 - Return the result of calling ``FFA_PARTITION_INFO_GET``. 927 - In a memory share operation, as part of the ``FFA_MEM_RETRIEVE_RESP``, 928 with the memory descriptor of the shared memory. 929 930If a normal world VM is expected to exchange messages with secure world, 931its RX/TX buffer addresses are forwarded to the SPMC via FFA_RXTX_MAP ABI, 932and are from this moment owned by the SPMC. 933The hypervisor must call the FFA_RX_ACQUIRE interface before attempting 934to use the RX buffer, in any of the aforementioned scenarios. A successful 935call to FFA_RX_ACQUIRE transfers ownership of RX buffer to hypervisor, such 936that it can be safely used. 937 938The FFA_RX_RELEASE interface is used after the FF-A endpoint is done with 939processing the data received in its RX buffer. If the RX buffer has been 940acquired by the hypervisor, the FFA_RX_RELEASE call must be forwarded to 941the SPMC to reestablish SPMC's RX ownership. 942 943An attempt from an SP to send a message to a normal world VM whose RX buffer 944was acquired by the hypervisor fails with error code FFA_BUSY, to preserve 945the RX buffer integrity. 946The operation could then be conducted after FFA_RX_RELEASE. 947 948FFA_MSG_SEND2 949~~~~~~~~~~~~~ 950 951Hafnium copies a message from the sender TX buffer into receiver's RX buffer. 952For messages from SPs to VMs, operation is only possible if the SPMC owns 953the receiver's RX buffer. 954 955Both receiver and sender need to enable support for indirect messaging, 956in their respective partition manifest. The discovery of support 957of such feature can be done via FFA_PARTITION_INFO_GET. 958 959On a successful message send, Hafnium pends an RX buffer full framework 960notification for the receiver, to inform it about a message in the RX buffer. 961 962The handling of framework notifications is similar to that of 963global notifications. Binding of these is not necessary, as these are 964reserved to be used by the hypervisor or SPMC. 965 966FFA_CONSOLE_LOG 967~~~~~~~~~~~~~~~ 968 969``FFA_CONSOLE_LOG`` allows debug logging to the UART console. 970Characters are packed into registers: 971 972- `w2-w7` (|SMCCC| 32-bit) 973- `x2-x7` (|SMCCC| 64-bit, before v1.2) 974- `x2-x17` (|SMCCC| 64-bit, v1.2 or later) 975 976Paravirtualized interfaces 977-------------------------- 978 979Hafnium SPMC implements the following implementation-defined interface(s): 980 981HF_INTERRUPT_ENABLE 982~~~~~~~~~~~~~~~~~~~ 983 984Enables or disables the given virtual interrupt for the calling execution 985context. Returns 0 on success, or -1 if the interrupt id is invalid. 986 987HF_INTERRUPT_GET 988~~~~~~~~~~~~~~~~ 989 990Returns the ID of the next pending virtual interrupt for the calling execution 991context, and acknowledges it (i.e. marks it as no longer pending). Returns 992HF_INVALID_INTID if there are no pending interrupts. 993 994HF_INTERRUPT_DEACTIVATE 995~~~~~~~~~~~~~~~~~~~~~~~ 996 997Drops the current interrupt priority and deactivates the given virtual and 998physical interrupt ID for the calling execution context. Returns 0 on success, 999or -1 otherwise. 1000 1001HF_INTERRUPT_RECONFIGURE 1002~~~~~~~~~~~~~~~~~~~~~~~~ 1003 1004An SP specifies the list of interrupts it owns through its partition manifest. 1005This paravirtualized interface allows an SP to reconfigure a physical interrupt 1006in runtime. It accepts three arguments, namely, interrupt ID, command and value. 1007The command & value pair signify what change is being requested by the current 1008Secure Partition for the given interrupt. 1009 1010SPMC returns 0 to indicate that the command was processed successfully or -1 if 1011it failed to do so. At present, this interface only supports the following 1012commands: 1013 1014 - ``INT_RECONFIGURE_TARGET_PE`` 1015 - Change the target CPU of the interrupt. 1016 - Value represents linear CPU index in the range 0 to (MAX_CPUS - 1). 1017 1018 - ``INT_RECONFIGURE_SEC_STATE`` 1019 - Change the security state of the interrupt. 1020 - Value must be either 0 (Non-secure) or 1 (Secure). 1021 1022 - ``INT_RECONFIGURE_ENABLE`` 1023 - Enable or disable the physical interrupt. 1024 - Value must be either 0 (Disable) or 1 (Enable). 1025 1026SPMC-SPMD direct requests/responses 1027----------------------------------- 1028 1029Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 1030Using those IDs in source/destination fields of a direct request/response 1031permits SPMD to SPMC communication and either way. 1032 1033- SPMC to SPMD direct request/response uses SMC conduit. 1034- SPMD to SPMC direct request/response uses ERET conduit. 1035 1036This is used in particular to convey power management messages. 1037 1038Notifications 1039------------- 1040 1041The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 1042communication mechanism with non-blocking semantics. It allows for one FF-A 1043endpoint to signal another for service provision, without hindering its current 1044progress. 1045 1046Hafnium currently supports 64 notifications. The IDs of each notification define 1047a position in a 64-bit bitmap. 1048 1049The signaling of notifications can interchangeably happen between NWd and SWd 1050FF-A endpoints. 1051 1052The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 1053VMs, and from VMs to SPs. An hypervisor component would only manage 1054notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 1055deployed in NWd, the Hypervisor or OS kernel must invoke the interface 1056FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 1057endpoint in the NWd that supports it. 1058 1059A sender can signal notifications once the receiver has provided it with 1060permissions. Permissions are provided by invoking the interface 1061FFA_NOTIFICATION_BIND. 1062 1063Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 1064they are considered to be in a pending sate. The receiver can retrieve its 1065pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 1066are considered to be handled. 1067 1068Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 1069that is in charge of donating CPU cycles for notifications handling. The 1070FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 1071which FF-A endpoints have pending notifications. The receiver scheduler is 1072called and informed by the FF-A driver, and it should allocate CPU cycles to the 1073receiver. 1074 1075There are two types of notifications supported: 1076 1077- Global, which are targeted to an FF-A endpoint and can be handled within any 1078 of its execution contexts, as determined by the scheduler of the system. 1079- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 1080 a specific execution context, as determined by the sender. 1081 1082The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 1083permissions to the sender. 1084 1085Notification signaling resorts to two interrupts: 1086 1087- Schedule Receiver Interrupt: non-secure physical interrupt to be handled by 1088 the FF-A driver within the receiver scheduler. At initialization the SPMC 1089 donates an SGI ID chosen from the secure SGI IDs range and configures it as 1090 non-secure. The SPMC triggers this SGI on the currently running core when 1091 there are pending notifications, and the respective receivers need CPU cycles 1092 to handle them. 1093- Notifications Pending Interrupt: virtual interrupt to be handled by the 1094 receiver of the notification. Set when there are pending notifications for the 1095 given secure partition. The NPI is pended when the NWd relinquishes CPU cycles 1096 to an SP. 1097 1098The notifications receipt support is enabled in the partition FF-A manifest. 1099 1100Memory Sharing 1101-------------- 1102 1103The Hafnium implementation aligns with FF-A v1.2 ALP0 specification, 1104'FF-A Memory Management Protocol' supplement `[11]`_. Hafnium supports 1105the following ABIs: 1106 1107 - ``FFA_MEM_SHARE`` - for shared access between lender and borrower. 1108 - ``FFA_MEM_LEND`` - borrower to obtain exclusive access, though lender 1109 retains ownership of the memory. 1110 - ``FFA_MEM_DONATE`` - lender permanently relinquishes ownership of memory 1111 to the borrower. 1112 1113The ``FFA_MEM_RETRIEVE_REQ`` interface is for the borrower to request the 1114memory to be mapped into its address space: for S-EL1 partitions the SPM updates 1115their stage 2 translation regime; for S-EL0 partitions the SPM updates their 1116stage 1 translation regime. On a successful call, the SPMC responds back with 1117``FFA_MEM_RETRIEVE_RESP``. 1118 1119The ``FFA_MEM_RELINQUISH`` interface is for when the borrower is done with using 1120a memory region. 1121 1122The ``FFA_MEM_RECLAIM`` interface is for the owner of the memory to reestablish 1123its ownership and exclusive access to the memory shared. 1124 1125The memory transaction descriptors are transmitted via RX/TX buffers. In 1126situations where the size of the memory transaction descriptor exceeds the 1127size of the RX/TX buffers, Hafnium provides support for fragmented transmission 1128of the full transaction descriptor. The ``FFA_MEM_FRAG_RX`` and ``FFA_MEM_FRAG_TX`` 1129interfaces are for receiving and transmitting the next fragment, respectively. 1130 1131If lender and borrower(s) are SPs, all memory sharing operations are supported. 1132 1133Hafnium also supports memory sharing operations between the normal world and the 1134secure world. If there is an SP involved, the SPMC allocates data to track the 1135state of the operation. 1136 1137An SP can not share, lend or donate memory to the NWd. 1138 1139The SPMC is also the designated allocator for the memory handle, when borrowers 1140include at least an SP. The SPMC doesn't support the hypervisor to be allocator 1141to the memory handle. 1142 1143Hafnium also supports memory lend and share targetting multiple borrowers. 1144This is the case for a lender SP to multiple SPs, and for a lender VM to 1145multiple endpoints (from both secure world and normal world). If there is 1146at least one borrower VM, the hypervisor is in charge of managing its 1147stage 2 translation on a successful memory retrieve. However, the hypervisor could 1148rely on the SPMC to keep track of the state of the operation, namely: 1149if all fragments to the memory descriptors have been sent, and if the retrievers 1150are still using the memory at any given moment. In this case, the hypervisor might 1151need to request the SPMC to obtain a description of the used memory regions. 1152For example, when handling an ``FFA_MEM_RECLAIM`` the hypervisor retrieve request 1153can be used to obtain that state information, do the necessary validations, 1154and update stage-2 memory translation of the lender. 1155Hafnium currently only supports one borrower from the NWd, in a multiple borrower 1156scenario as described. If there is only a single borrower VM, the SPMC will 1157return error to the lender on call to either share, lend or donate ABIs. 1158 1159The semantics of ``FFA_MEM_DONATE`` implies ownership transmission, 1160which should target only one partition. 1161 1162The memory share interfaces are backwards compatible with memory transaction 1163descriptors from FF-A v1.0. Starting from FF-A v1.1, with the introduction 1164of the `Endpoint memory access descriptor size` and 1165`Endpoint memory access descriptor access offset` fields (from Table 11.20 of the 1166FF-A v1.2 ALP0 specification), memory transaction descriptors are forward 1167compatible, so can be used internally by Hafnium as they are sent. 1168These fields must be valid for a memory access descriptor defined for a compatible 1169FF-A version to the SPMC FF-A version. For a transaction from an FF-A v1.0 endpoint 1170the memory transaction descriptor will be translated to an FF-A v1.1 descriptor for 1171Hafnium's internal processing of the operation. If the FF-A version of a 1172borrower is v1.0, Hafnium provides FF-A v1.0 compliant memory transaction 1173descriptors on memory retrieve response. 1174 1175In the section :ref:`SPMC Configuration` there is a mention of non-secure memory 1176range, that limit the memory region nodes the SP can define. Whatever is left of 1177the memory region node carve-outs, the SPMC utilizes the memory to create a set of 1178page tables it associates with the NWd. The memory sharing operations incoming from 1179the NWd should refer to addresses belonging to these page tables. The intent 1180is for SPs not to be able to get access to regions they are not intended to access. 1181This requires special care from the system integrator to configure the memory ranges 1182correctly, such that any SP can't be given access and interfere with execution of 1183other components. More information in the :ref:`Threat Model`. 1184 1185Hafnium SPMC supports memory management transactions for device memory regions. 1186Currently this is limited to only the ``FFA_MEM_LEND`` interface and 1187to a single borrower. The device memory region used in the transaction must have 1188been decalared in the SPMC manifest as described above. Memory defined in a device 1189region node is given the attributes Device-nGnRnE, since this is the most restrictive 1190memory type the memory must be lent with these attrbutes as well. 1191 1192In |RME| enabled platforms, there is the ability to change the |PAS| 1193of a given memory region `[12]`_. The SPMC can leverage this feature to fulfill the 1194semantics of the ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE`` from the NWd into the SWd. 1195Currently, there is the implementation for the FVP platform to issue a 1196platform-specific SMC call to the EL3 monitor to change the PAS of the regions being 1197lent/donated. This shall guarantee the NWd can't tamper with the memory whilst 1198the SWd software expects exclusive access. For any other platform, the API under 1199the 'src/memory_protect' module can be redefined to leverage an equivalent platform 1200specific mechanism. For reference, check the `SPMC FVP build configuration`_. 1201 1202PE MMU configuration 1203-------------------- 1204 1205With secure virtualization enabled (``HCR_EL2.VM = 1``) and for S-EL1 1206partitions, two IPA spaces (secure and non-secure) are output from the 1207secure EL1&0 Stage-1 translation. 1208The EL1&0 Stage-2 translation hardware is fed by: 1209 1210- A secure IPA when the SP EL1&0 Stage-1 MMU is disabled. 1211- One of secure or non-secure IPA when the secure EL1&0 Stage-1 MMU is enabled. 1212 1213``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 1214NS/S IPA translations. The following controls are set up: 1215``VSTCR_EL2.SW = 0`` , ``VSTCR_EL2.SA = 0``, ``VTCR_EL2.NSW = 0``, 1216``VTCR_EL2.NSA = 1``: 1217 1218- Stage-2 translations for the NS IPA space access the NS PA space. 1219- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 1220 1221Secure and non-secure IPA regions (rooted to by ``VTTBR_EL2`` and ``VSTTBR_EL2``) 1222use the same set of Stage-2 page tables within a SP. 1223 1224The ``VTCR_EL2/VSTCR_EL2/VTTBR_EL2/VSTTBR_EL2`` virtual address space 1225configuration is made part of a vCPU context. 1226 1227For S-EL0 partitions with VHE enabled, a single secure EL2&0 Stage-1 translation 1228regime is used for both Hafnium and the partition. 1229 1230Schedule modes and SP Call chains 1231--------------------------------- 1232 1233An SP execution context is said to be in SPMC scheduled mode if CPU cycles are 1234allocated to it by SPMC. Correspondingly, an SP execution context is said to be 1235in Normal world scheduled mode if CPU cycles are allocated by the normal world. 1236 1237A call chain represents all SPs in a sequence of invocations of a direct message 1238request. When execution on a PE is in the secure state, only a single call chain 1239that runs in the Normal World scheduled mode can exist. FF-A v1.1 spec allows 1240any number of call chains to run in the SPMC scheduled mode but the Hafnium 1241SPMC restricts the number of call chains in SPMC scheduled mode to only one for 1242keeping the implementation simple. 1243 1244Partition runtime models 1245------------------------ 1246 1247The runtime model of an endpoint describes the transitions permitted for an 1248execution context between various states. These are the four partition runtime 1249models supported (refer to `[1]`_ section 7): 1250 1251 - RTM_FFA_RUN: runtime model presented to an execution context that is 1252 allocated CPU cycles through FFA_RUN interface. 1253 - RTM_FFA_DIR_REQ: runtime model presented to an execution context that is 1254 allocated CPU cycles through FFA_MSG_SEND_DIRECT_REQ or FFA_MSG_SEND_DIRECT_REQ2 1255 interface. 1256 - RTM_SEC_INTERRUPT: runtime model presented to an execution context that is 1257 allocated CPU cycles by SPMC to handle a secure interrupt. 1258 - RTM_SP_INIT: runtime model presented to an execution context that is 1259 allocated CPU cycles by SPMC to initialize its state. 1260 1261If an endpoint execution context attempts to make an invalid transition or a 1262valid transition that could lead to a loop in the call chain, SPMC denies the 1263transition with the help of above runtime models. 1264 1265Interrupt management 1266-------------------- 1267 1268GIC ownership 1269~~~~~~~~~~~~~ 1270 1271The SPMC owns the GIC configuration. Secure and non-secure interrupts are 1272trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 1273IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 1274virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 1275 1276Abbreviations: 1277 1278 - NS-Int: A non-secure physical interrupt. It requires a switch to the normal 1279 world to be handled if it triggers while execution is in secure world. 1280 - Other S-Int: A secure physical interrupt targeted to an SP different from 1281 the one that is currently running. 1282 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1283 running. 1284 1285Non-secure interrupt handling 1286~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1287 1288This section documents the actions supported in SPMC in response to a non-secure 1289interrupt as per the guidance provided by FF-A v1.1 EAC0 specification. 1290An SP specifies one of the following actions in its partition manifest: 1291 1292 - Non-secure interrupt is signaled. 1293 - Non-secure interrupt is signaled after a managed exit. 1294 - Non-secure interrupt is queued. 1295 1296An SP execution context in a call chain could specify a less permissive action 1297than subsequent SP execution contexts in the same call chain. The less 1298permissive action takes precedence over the more permissive actions specified 1299by the subsequent execution contexts. Please refer to FF-A v1.1 EAC0 section 13008.3.1 for further explanation. 1301 1302Secure interrupt handling 1303~~~~~~~~~~~~~~~~~~~~~~~~~ 1304 1305This section documents the support implemented for secure interrupt handling in 1306SPMC as per the guidance provided by FF-A v1.1 EAC0 specification. 1307The following assumptions are made about the system configuration: 1308 1309 - In the current implementation, S-EL1 SPs are expected to use the para 1310 virtualized ABIs for interrupt management rather than accessing the virtual 1311 GIC interface. 1312 - Unless explicitly stated otherwise, this support is applicable only for 1313 S-EL1 SPs managed by SPMC. 1314 - Secure interrupts are configured as G1S or G0 interrupts. 1315 - All physical interrupts are routed to SPMC when running a secure partition 1316 execution context. 1317 - All endpoints with multiple execution contexts have their contexts pinned 1318 to corresponding CPUs. Hence, a secure virtual interrupt cannot be signaled 1319 to a target vCPU that is currently running or blocked on a different 1320 physical CPU. 1321 1322A physical secure interrupt could trigger while CPU is executing in normal world 1323or secure world. 1324The action of SPMC for a secure interrupt depends on: the state of the target 1325execution context of the SP that is responsible for handling the interrupt; 1326whether the interrupt triggered while execution was in normal world or secure 1327world. 1328 1329Secure interrupt signaling mechanisms 1330~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1331 1332Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 1333context that it has a pending virtual interrupt and to further run the SP 1334execution context, such that it can handle the virtual interrupt. SPMC uses 1335either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 1336to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 1337the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 1338running in S-EL2. 1339 1340+-----------+---------+---------------+---------------------------------------+ 1341| SP State | Conduit | Interface and | Description | 1342| | | parameters | | 1343+-----------+---------+---------------+---------------------------------------+ 1344| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 1345| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 1346| | | | resumes execution context of SP | 1347| | | | through ERET. | 1348+-----------+---------+---------------+---------------------------------------+ 1349| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 1350| | vIRQ | | is pending. It pends vIRQ signal and | 1351| | | | resumes execution context of SP | 1352| | | | through ERET. | 1353+-----------+---------+---------------+---------------------------------------+ 1354| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 1355| | | | not resume execution context of SP. | 1356+-----------+---------+---------------+---------------------------------------+ 1357| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 1358| | vIRQ | | execution context of SP through ERET. | 1359+-----------+---------+---------------+---------------------------------------+ 1360 1361Secure interrupt completion mechanisms 1362~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1363 1364A SP signals secure interrupt handling completion to the SPMC through the 1365following mechanisms: 1366 1367 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 1368 - ``FFA_RUN`` ABI if its was in BLOCKED state. 1369 1370This is a remnant of SPMC implementation based on the FF-A v1.0 specification. 1371In the current implementation, S-EL1 SPs use the para-virtualized HVC interface 1372implemented by SPMC to perform priority drop and interrupt deactivation (SPMC 1373configures EOImode = 0, i.e. priority drop and deactivation are done together). 1374The SPMC performs checks to deny the state transition upon invocation of 1375either FFA_MSG_WAIT or FFA_RUN interface if the SP didn't perform the 1376deactivation of the secure virtual interrupt. 1377 1378If the current SP execution context was preempted by a secure interrupt to be 1379handled by execution context of target SP, SPMC resumes current SP after signal 1380completion by target SP execution context. 1381 1382Actions for a secure interrupt triggered while execution is in normal world 1383~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1384 1385+-------------------+----------+-----------------------------------------------+ 1386| State of target | Action | Description | 1387| execution context | | | 1388+-------------------+----------+-----------------------------------------------+ 1389| WAITING | Signaled | This starts a new call chain in SPMC scheduled| 1390| | | mode. | 1391+-------------------+----------+-----------------------------------------------+ 1392| PREEMPTED | Queued | The target execution must have been preempted | 1393| | | by a non-secure interrupt. SPMC queues the | 1394| | | secure virtual interrupt now. It is signaled | 1395| | | when the target execution context next enters | 1396| | | the RUNNING state. | 1397+-------------------+----------+-----------------------------------------------+ 1398| BLOCKED, RUNNING | NA | The target execution context is blocked or | 1399| | | running on a different CPU. This is not | 1400| | | supported by current SPMC implementation and | 1401| | | execution hits panic. | 1402+-------------------+----------+-----------------------------------------------+ 1403 1404If normal world execution was preempted by a secure interrupt, SPMC uses 1405FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1406and further returns execution to normal world. 1407 1408The following figure describes interrupt handling flow when a secure interrupt 1409triggers while execution is in normal world: 1410 1411.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1412 1413A brief description of the events: 1414 1415 - 1) Secure interrupt triggers while normal world is running. 1416 - 2) FIQ gets trapped to EL3. 1417 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1418 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1419 vIRQ). 1420 - 5) Assuming SP1 vCPU is in WAITING state, SPMC signals virtual interrupt 1421 using FFA_INTERRUPT with interrupt id as an argument and resumes the SP1 1422 vCPU using ERET in SPMC scheduled mode. 1423 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1424 interrupt is not masked i.e., PSTATE.I = 0 1425 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1426 HVC call. SPMC clears the pending virtual interrupt state management 1427 and returns the pending virtual interrupt id. 1428 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1429 de-activation HVC call. SPMC de-activates the physical interrupt, 1430 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1431 - 9) SP1 performs secure interrupt completion through FFA_MSG_WAIT ABI. 1432 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1433 - 11) EL3 resumes normal world execution. 1434 1435Actions for a secure interrupt triggered while execution is in secure world 1436~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1437 1438+-------------------+----------+------------------------------------------------+ 1439| State of target | Action | Description | 1440| execution context | | | 1441+-------------------+----------+------------------------------------------------+ 1442| WAITING | Signaled | This starts a new call chain in SPMC scheduled | 1443| | | mode. | 1444+-------------------+----------+------------------------------------------------+ 1445| PREEMPTED by Self | Signaled | The target execution context reenters the | 1446| S-Int | | RUNNING state to handle the secure virtual | 1447| | | interrupt. | 1448+-------------------+----------+------------------------------------------------+ 1449| PREEMPTED by | Queued | SPMC queues the secure virtual interrupt now. | 1450| NS-Int | | It is signaled when the target execution | 1451| | | context next enters the RUNNING state. | 1452+-------------------+----------+------------------------------------------------+ 1453| BLOCKED | Signaled | Both preempted and target execution contexts | 1454| | | must have been part of the Normal world | 1455| | | scheduled call chain. Refer scenario 1 of | 1456| | | Table 8.4 in the FF-A v1.1 EAC0 spec. | 1457+-------------------+----------+------------------------------------------------+ 1458| RUNNING | NA | The target execution context is running on a | 1459| | | different CPU. This scenario is not supported | 1460| | | by current SPMC implementation and execution | 1461| | | hits panic. | 1462+-------------------+----------+------------------------------------------------+ 1463 1464The following figure describes interrupt handling flow when a secure interrupt 1465triggers while execution is in secure world. We assume OS kernel sends a direct 1466request message to SP1. Further, SP1 sends a direct request message to SP2. SP1 1467enters BLOCKED state and SPMC resumes SP2. 1468 1469.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1470 1471A brief description of the events: 1472 1473 - 1) Secure interrupt triggers while SP2 is running. 1474 - 2) SP2 gets preempted and execution traps to SPMC as IRQ. 1475 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1476 this secure interrupt. In this scenario, it is SP1. 1477 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1478 SPMC further resumes SP1 through ERET conduit. Note that SP1 remains in 1479 Normal world schedule mode. 1480 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1481 interrupt is not masked i.e., PSTATE.I = 0 1482 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1483 HVC call. SPMC clears the pending virtual interrupt state management 1484 and returns the pending virtual interrupt id. 1485 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1486 de-activation HVC call. SPMC de-activates the physical interrupt and 1487 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1488 - 9) Since SP1 direct request completed with FFA_INTERRUPT, it resumes the 1489 direct request to SP2 by invoking FFA_RUN. 1490 - 9) SPMC resumes the pre-empted vCPU of SP2. 1491 1492EL3 interrupt handling 1493~~~~~~~~~~~~~~~~~~~~~~ 1494 1495In GICv3 based systems, EL3 interrupts are configured as Group0 secure 1496interrupts. Execution traps to SPMC when a Group0 interrupt triggers while an 1497SP is running. Further, SPMC running at S-EL2 uses FFA_EL3_INTR_HANDLE ABI to 1498request EL3 platform firmware to handle a pending Group0 interrupt. 1499Similarly, SPMD registers a handler with interrupt management framework to 1500delegate handling of Group0 interrupt to the platform if the interrupt triggers 1501in normal world. 1502 1503 - Platform hook 1504 1505 - plat_spmd_handle_group0_interrupt 1506 1507 SPMD provides platform hook to handle Group0 secure interrupts. In the 1508 current design, SPMD expects the platform not to delegate handling to the 1509 NWd (such as through SDEI) while processing Group0 interrupts. 1510 1511Power management 1512---------------- 1513 1514In platforms with or without secure virtualization: 1515 1516- The NWd owns the platform PM policy. 1517- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1518- The EL3 PSCI library is in charge of the PM coordination and control 1519 (eventually writing to platform registers). 1520- While coordinating PM events, the PSCI library calls backs into the Secure 1521 Payload Dispatcher for events the latter has statically registered to. 1522 1523When using the SPMD as a Secure Payload Dispatcher: 1524 1525- A power management event is relayed through the SPD hook to the SPMC. 1526- In the current implementation only cpu on (svc_on_finish) and cpu off 1527 (svc_off) hooks are registered. 1528- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1529 The SPMC is entered through its secondary physical core entry point. 1530- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The PM event is 1531 signaled to the SPMC through a power management framework message. 1532 It consists in a SPMD-to-SPMC direct request/response (`SPMC-SPMD direct 1533 requests/responses`_) conveying the event details and SPMC response. 1534 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1535 updates its internal state to reflect the physical core is being turned off. 1536 In the current implementation no SP is resumed as a consequence. This behavior 1537 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1538 userspace. 1539 1540Arm architecture extensions for security hardening 1541-------------------------------------------------- 1542 1543Hafnium supports the following architecture extensions for security hardening: 1544 1545- Pointer authentication (FEAT_PAuth): the extension permits detection of forged 1546 pointers used by ROP type of attacks through the signing of the pointer 1547 value. Hafnium is built with the compiler branch protection option to permit 1548 generation of a pointer authentication code for return addresses (pointer 1549 authentication for instructions). The APIA key is used while Hafnium runs. 1550 A random key is generated at boot time and restored upon entry into Hafnium 1551 at run-time. APIA and other keys (APIB, APDA, APDB, APGA) are saved/restored 1552 in vCPU contexts permitting to enable pointer authentication in VMs/SPs. 1553- Branch Target Identification (FEAT_BTI): the extension permits detection of 1554 unexpected indirect branches used by JOP type of attacks. Hafnium is built 1555 with the compiler branch protection option, inserting land pads at function 1556 prologues that are reached by indirect branch instructions (BR/BLR). 1557 Hafnium code pages are marked as guarded in the EL2 Stage-1 MMU descriptors 1558 such that an indirect branch must always target a landpad. A fault is 1559 triggered otherwise. VMs/SPs can (independently) mark their code pages as 1560 guarded in the EL1&0 Stage-1 translation regime. 1561- Memory Tagging Extension (FEAT_MTE): the option permits detection of out of 1562 bound memory array accesses or re-use of an already freed memory region. 1563 Hafnium enables the compiler option permitting to leverage MTE stack tagging 1564 applied to core stacks. Core stacks are marked as normal tagged memory in the 1565 EL2 Stage-1 translation regime. A synchronous data abort is generated upon tag 1566 check failure on load/stores. A random seed is generated at boot time and 1567 restored upon entry into Hafnium. MTE system registers are saved/restored in 1568 vCPU contexts permitting MTE usage from VMs/SPs. 1569- Realm Management Extension (FEAT_RME): can be deployed in platforms that leverage 1570 RME for physical address isolation. The SPMC is capable of recovering from a 1571 Granule Protection Fault, if inadvertently accessing a region with the wrong security 1572 state setting. Also, the ability to change dynamically the physical address space of 1573 a region, can be used to enhance the handling of ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE``. 1574 More details in the section about `Memory Sharing`_. 1575 1576SIMD support 1577------------ 1578 1579In this section, the generic term |SIMD| is used to refer to vector and matrix 1580processing units offered by the Arm architecture. This concerns the optional 1581architecture extensions: Advanced SIMD (formerly FPU / NEON) / |SVE| / |SME|. 1582 1583The SPMC preserves the |SIMD| state according to the |SMCCC| (ARM DEN 0028F 15841.5F section 10 Appendix C: SME, SVE, SIMD and FP live state preservation by 1585the |SMCCC| implementation). 1586 1587The SPMC implements the |SIMD| support in the following way: 1588 1589- SPs are allowed to use Advanced SIMD instructions and manipulate 1590 the Advanced SIMD state. 1591- The SPMC saves and restores vCPU Advanced SIMD state when switching vCPUs. 1592- SPs are restricted from using |SVE| and |SME| instructions and manipulating 1593 associated system registers and state. Doing so, traps to the same or higher 1594 EL. 1595- Entry from the normal world into the SPMC and exit from the SPMC to the normal 1596 world preserve the |SIMD| state. 1597- Corollary to the above, the normal world is free to use any of the referred 1598 |SIMD| extensions and emit FF-A SMCs. The SPMC as a callee preserves the live 1599 |SIMD| state according to the rules mentioned in the |SMCCC|. 1600- This is also true for the case of a secure interrupt pre-empting the normal 1601 world while it is currently processing |SIMD| instructions. 1602- |SVE| and |SME| traps are enabled while S-EL2/1/0 run. Traps are temporarily 1603 disabled on the narrow window of the context save/restore operation within 1604 S-EL2. Traps are enabled again after those operations. 1605 1606Supported configurations 1607~~~~~~~~~~~~~~~~~~~~~~~~ 1608 1609The SPMC assumes Advanced SIMD is always implemented (despite being an Arm 1610optional architecture extension). The SPMC dynamically detects whether |SVE| 1611and |SME| are implemented in the platform, then saves and restores the |SIMD| 1612state according to the different combinations: 1613 1614+--------------+--------------------+--------------------+---------------+ 1615| FEAT_AdvSIMD | FEAT_SVE/FEAT_SVE2 | FEAT_SME/FEAT_SME2 | FEAT_SME_FA64 | 1616+--------------+--------------------+--------------------+---------------+ 1617| Y | N | N | N | 1618+--------------+--------------------+--------------------+---------------+ 1619| Y | Y | N | N | 1620+--------------+--------------------+--------------------+---------------+ 1621| Y | Y | Y | N | 1622+--------------+--------------------+--------------------+---------------+ 1623| Y | Y | Y | Y | 1624+--------------+--------------------+--------------------+---------------+ 1625| Y | N | Y | N | 1626+--------------+--------------------+--------------------+---------------+ 1627| Y | N | Y | Y | 1628+--------------+--------------------+--------------------+---------------+ 1629 1630Y: architectural feature implemented 1631N: architectural feature not implemented 1632 1633SIMD save/restore operations 1634~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1635 1636The SPMC considers the following SIMD registers state: 1637 1638- Advanced SIMD consists of 32 ``Vn`` 128b vectors. Vector's lower 128b is 1639 shared with the larger |SVE| / |SME| variable length vectors. 1640- |SVE| consists of 32 ``Zn`` variable length vectors, ``Px`` predicates, 1641 ``FFR`` fault status register. 1642- |SME| when Streaming SVE is enabled consists of 32 ``Zn`` variable length 1643 vectors, ``Px`` predicates, ``FFR`` fault status register (when FEAT_SME_FA64 1644 extension is implemented and enabled), ZA array (when enabled). 1645- Status and control registers (FPCR/FPSR) common to all above. 1646 1647For the purpose of supporting the maximum vector length (or Streaming SVE 1648vector length) supported by the architecture, the SPMC sets ``SCR_EL2.LEN`` 1649and ``SMCR_EL2.LEN`` to the maximum permitted value (2048 bits). This makes 1650save/restore operations independent from the vector length constrained by EL3 1651(by ``ZCR_EL3``), or the ``ZCR_EL2.LEN`` value set by the normal world itself. 1652 1653For performance reasons, the normal world might let the secure world know it 1654doesn't depend on the |SVE| or |SME| live state while doing an SMC. It does 1655so by setting the |SMCCC| SVE hint bit. In which case, the secure world limits 1656the normal world context save/restore operations to the Advanced SIMD state 1657even if either one of |SVE| or |SME|, or both, are implemented. 1658 1659The following additional design choices were made related to SME save/restore 1660operations: 1661 1662- When FEAT_SME_FA64 is implemented, ``SMCR_EL2.FA64`` is set and FFR register 1663 saved/restored when Streaming SVE mode is enabled. 1664- For power saving reasons, if Streaming SVE mode is enabled while entering the 1665 SPMC, this state is recorded, Streaming SVE state saved and the mode disabled. 1666 Streaming SVE is enabled again while restoring the SME state on exiting the 1667 SPMC. 1668- The ZA array state is left untouched while the SPMC runs. As neither SPMC 1669 and SPs alter the ZA array state, this is a conservative approach in terms 1670 of memory footprint consumption. 1671 1672SMMUv3 support in Hafnium 1673------------------------- 1674 1675An SMMU is analogous to an MMU in a CPU. It performs address translations for 1676Direct Memory Access (DMA) requests from system I/O devices. 1677The responsibilities of an SMMU include: 1678 1679- Translation: Incoming DMA requests are translated from bus address space to 1680 system physical address space using translation tables compliant to 1681 Armv8/Armv7 VMSA descriptor format. 1682- Protection: An I/O device can be prohibited from read, write access to a 1683 memory region or allowed. 1684- Isolation: Traffic from each individial device can be independently managed. 1685 The devices are differentiated from each other using unique translation 1686 tables. 1687 1688The following diagram illustrates a typical SMMU IP integrated in a SoC with 1689several I/O devices along with Interconnect and Memory system. 1690 1691.. image:: ../resources/diagrams/MMU-600.png 1692 1693SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1694support for SMMUv3 driver in both normal and secure world. A brief introduction 1695of SMMUv3 functionality and the corresponding software support in Hafnium is 1696provided here. 1697 1698SMMUv3 features 1699~~~~~~~~~~~~~~~ 1700 1701- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1702 translation support. It can either bypass or abort incoming translations as 1703 well. 1704- Traffic (memory transactions) from each upstream I/O peripheral device, 1705 referred to as Stream, can be independently managed using a combination of 1706 several memory based configuration structures. This allows the SMMUv3 to 1707 support a large number of streams with each stream assigned to a unique 1708 translation context. 1709- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1710 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1711 are supported by SMMUv3. 1712- SMMUv3 offers non-secure stream support with secure stream support being 1713 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1714 instance for secure and non-secure stream support. 1715- It also supports sub-streams to differentiate traffic from a virtualized 1716 peripheral associated with a VM/SP. 1717- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1718 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1719 for providing Secure Stage2 translation support to upstream peripheral 1720 devices. 1721 1722SMMUv3 Programming Interfaces 1723~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1724 1725SMMUv3 has three software interfaces that are used by the Hafnium driver to 1726configure the behaviour of SMMUv3 and manage the streams. 1727 1728- Memory based data strutures that provide unique translation context for 1729 each stream. 1730- Memory based circular buffers for command queue and event queue. 1731- A large number of SMMU configuration registers that are memory mapped during 1732 boot time by Hafnium driver. Except a few registers, all configuration 1733 registers have independent secure and non-secure versions to configure the 1734 behaviour of SMMUv3 for translation of secure and non-secure streams 1735 respectively. 1736 1737Peripheral device manifest 1738~~~~~~~~~~~~~~~~~~~~~~~~~~ 1739 1740Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1741These DMA devices are dependent on PE endpoint to initiate and receive memory 1742management transactions on their behalf. The acccess to the MMIO regions of 1743any such device is assigned to the endpoint during boot. 1744The :ref:`device node<device_region_node>` of the corresponding partition 1745manifest must specify these additional properties for each peripheral device in 1746the system: 1747 1748- smmu-id: This field helps to identify the SMMU instance that this device is 1749 upstream of. 1750- stream-ids: List of stream IDs assigned to this device. 1751 1752.. code:: shell 1753 1754 smmuv3-testengine { 1755 base-address = <0x00000000 0x2bfe0000>; 1756 pages-count = <32>; 1757 attributes = <0x3>; 1758 smmu-id = <0>; 1759 stream-ids = <0x0 0x1>; 1760 interrupts = <0x2 0x3>, <0x4 0x5>; 1761 exclusive-access; 1762 }; 1763 1764DMA isolation 1765------------- 1766 1767Hafnium, with help of SMMUv3 driver, enables the support for static DMA 1768isolation. The DMA device is explicitly granted access to a specific 1769memory region only if the partition requests it by declaring the following 1770properties of the DMA device in the :ref:`memory region node<memory_region_node>` 1771of the partition manifest: 1772 1773- smmu-id 1774- stream-ids 1775- stream-ids-access-permissions 1776 1777SMMUv3 driver uses a unqiue set of stage 2 translations for the DMA device 1778rather than those used on behalf of the PE endpoint. This ensures that the DMA 1779device has a limited visibility of the physical address space. 1780 1781.. code:: shell 1782 1783 smmuv3-memcpy-src { 1784 description = "smmuv3-memcpy-source"; 1785 pages-count = <4>; 1786 base-address = <0x00000000 0x7400000>; 1787 attributes = <0x3>; /* read-write */ 1788 smmu-id = <0>; 1789 stream-ids = <0x0 0x1>; 1790 stream-ids-access-permissions = <0x3 0x3>; 1791 }; 1792 1793SMMUv3 driver limitations 1794~~~~~~~~~~~~~~~~~~~~~~~~~ 1795 1796The primary design goal for the Hafnium SMMU driver is to support secure 1797streams. 1798 1799- Currently, the driver only supports Stage2 translations. No support for 1800 Stage1 or nested translations. 1801- Supports only AArch64 translation format. 1802- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1803 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1804- No support for independent peripheral devices. 1805 1806S-EL0 Partition support 1807----------------------- 1808The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1809FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1810with ARMv8.4 and FEAT_SEL2). 1811 1812S-EL0 partitions are useful for simple partitions that don't require full 1813Trusted OS functionality. It is also useful to reduce jitter and cycle 1814stealing from normal world since they are more lightweight than VMs. 1815 1816S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1817the SPMC. They are differentiated primarily by the 'exception-level' property 1818and the 'execution-ctx-count' property in the SP manifest. They are host apps 1819under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1820call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1821can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1822for memory regions. 1823 1824S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1825capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1826a S-EL0 partition to accept a direct message from secure world and normal world, 1827and generate direct responses to them. 1828All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not supported. 1829 1830Interrupt handling, Memory sharing, indirect messaging, and notifications features 1831in context of S-EL0 partitions are supported. 1832 1833References 1834========== 1835 1836.. _TF-A project: https://trustedfirmware-a.readthedocs.io/en/latest/ 1837 1838.. _SPMC FVP build configuration: https://github.com/TF-Hafnium/hafnium-project-reference/blob/main/BUILD.gn#L143 1839 1840.. _[1]: 1841 1842[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1843 1844.. _[2]: 1845 1846[2] `Secure Partition Manager using MM interface <https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partition-manager-mm.html>`__ 1847 1848.. _[3]: 1849 1850[3] `Trusted Boot Board Requirements 1851Client <https://developer.arm.com/documentation/den0006/d/>`__ 1852 1853.. _[4]: 1854 1855[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1856 1857.. _[5]: 1858 1859[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1860 1861.. _[6]: 1862 1863[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1864 1865.. _[7]: 1866 1867[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1868 1869.. _[8]: 1870 1871[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/ 1872 1873.. _[9]: 1874 1875[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1876 1877.. _[10]: 1878 1879[10] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/build-options.html# 1880 1881 .. _[11]: 1882 1883[11] https://developer.arm.com/documentation/den0140/a 1884 1885 .. _[12]: 1886 1887[12] https://developer.arm.com/documentation/den0129/latest/ 1888 1889-------------- 1890 1891*Copyright (c) 2020-2023, Arm Limited and Contributors. All rights reserved.* 1892
