1EL3 Secure Partition Manager 2**************************** 3 4.. contents:: 5 6Foreword 7======== 8 9This document describes the design of the EL3 SPMC based on the FF-A specification. 10EL3 SPMC provides reference FF-A compliant implementation without S-EL2 virtualization support, 11to help adopt and migrate to FF-A early. 12EL3 SPMC implementation in TF-A: 13 14- Manages a single S-EL1 Secure Partition 15- Provides a standard protocol for communication and memory sharing between FF-A endpoints. 16- Provides support for EL3 Logical Partitions to support easy migration from EL3 to S-EL1. 17 18Sample reference stack 19====================== 20 21The following diagram illustrates a possible configuration when the 22FEAT_SEL2 architecture extension is not implemented, showing the SPMD 23and SPMC at EL3, one S-EL1 secure partition, with an optional 24Hypervisor: 25 26.. image:: ../resources/diagrams/ff-a-spm-at-el3.png 27 28TF-A build options 29================== 30 31This section explains the TF-A build options involved in building 32an FF-A based SPM where the SPMD and SPMC are located at EL3: 33 34- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 35 protocol from NWd to SWd back and forth. It is not possible to 36 enable another Secure Payload Dispatcher when this option is chosen. 37- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being 38 at EL3. 39- **ARM_SPMC_MANIFEST_DTS**: this option specifies a manifest file 40 providing SP description. It is required when 41 ``SPMC_AT_EL3`` is enabled, the secure partitions are loaded 42 by BL2 on behalf of the SPMC. 43 44Notes: 45 46- BL32 option is re-purposed to specify the S-EL1 TEE or SP image. 47 BL32 option can be omitted if using TF-A Test Secure Payload as SP. 48- BL33 option can specify the TFTF binary or a normal world loader 49 such as U-Boot or the UEFI framework payload. 50 51Sample TF-A build command line when the SPMC is located at EL3: 52 53.. code:: shell 54 55 make \ 56 CROSS_COMPILE=aarch64-none-elf- \ 57 SPD=spmd \ 58 SPMD_SPM_AT_SEL2=0 \ 59 SPMC_AT_EL3=1 \ 60 BL32=<path-to-tee-binary> (opt for TSP) \ 61 BL33=<path-to-bl33-binary> \ 62 PLAT=fvp \ 63 all fip 64 65FVP model invocation 66==================== 67 68Sample FVP command line invocation: 69 70.. code:: shell 71 72 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \ 73 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 74 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 75 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 76 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 77 -C bp.pl011_uart2.out_file=fvp-uart2.log -C bp.vis.disable_visualisation=1 78 79 80Platform Guide 81============== 82 83- Platform Hooks See - `[4]`_ 84 85 - plat_spmc_shmem_begin 86 - plat_spmc_shmem_reclaim 87 88SPMC provides platform hooks related to memory management interfaces. 89These hooks can be used for platform specific implementations like 90for managing access control, programming TZ Controller or MPUs. 91These hooks are called by SPMC before the initial share request completes, 92and after the final reclaim has been completed. 93 94- Datastore 95 96 - plat_spmc_shmem_datastore_get 97 98 EL3 SPMC uses datastore for tracking memory transaction descriptors. 99 On FVP platform datastore is allocated from TZC DRAM section. 100 Other platforms need to allocate a similar secure memory region 101 to be used as shared memory datastore. 102 103 The accessor function is used during SPMC initialization to obtain 104 address and size of the datastore. 105 SPMC will also zero out the provided memory region. 106 107- Platform Defines See - `[5]`_ 108 109 - SECURE_PARTITION_COUNT 110 Number of Secure Partitions supported: must be 1. 111 112 - NS_PARTITION_COUNT 113 Number of NWd Partitions supported. 114 115 - MAX_EL3_LP_DESCS_COUNT 116 Number of Logical Partitions supported. 117 118Logical Secure Partition (LSP) 119============================== 120 121- The SPMC provides support for statically allocated EL3 Logical Secure Partitions 122 as per FF-A v1.1 specification. 123- The DECLARE_LOGICAL_PARTITION macro can be used to add a LSP. 124- For reference implementation See - `[2]`_ 125 126.. image:: ../resources/diagrams/ff-a-lsp-at-el3.png 127 128SPMC boot 129========= 130 131The SPMD and SPMC are built into the BL31 image along with TF-A's runtime components. 132BL2 loads the BL31 image as a part of (secure) boot process. 133 134The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 135 136BL2 passes the SPMC manifest address to BL31 through a register. 137 138At boot time, the SPMD in BL31 runs from the primary core, initializes the core 139contexts and launches the SPMC passing the following information through 140registers: 141 142- X0 holds the SPMC manifest blob address. 143- X4 holds the currently running core linear id. 144 145Parsing SP partition manifests 146------------------------------ 147 148SPMC consumes the SP manifest, as defined in `[7]`_. 149SP manifest fields align with Hafnium SP manifest for easy porting. 150 151.. code:: shell 152 153 compatible = "arm,ffa-manifest-1.0"; 154 155 ffa-version = <0x00010001>; /* 31:16 - Major, 15:0 - Minor */ 156 id = <0x8001>; 157 uuid = <0x6b43b460 0x74a24b78 0xade24502 0x40682886>; 158 messaging-method = <0x3>; /* Direct Messaging Only */ 159 exception-level = <0x2>; /* S-EL1 */ 160 execution-state = <0>; 161 execution-ctx-count = <8>; 162 gp-register-num = <0>; 163 power-management-messages = <0x7>; 164 165 166Passing boot data to the SP 167--------------------------- 168 169In `[1]`_ , the section "Boot information protocol" defines a method for passing 170data to the SPs at boot time. It specifies the format for the boot information 171descriptor and boot information header structures, which describe the data to be 172exchanged between SPMC and SP. 173The specification also defines the types of data that can be passed. 174The aggregate of both the boot info structures and the data itself is designated 175the boot information blob, and is passed to a Partition as a contiguous memory 176region. 177 178Currently, the SPM implementation supports the FDT type which is used to pass the 179partition's DTB manifest. 180 181The region for the boot information blob is statically allocated (4K) by SPMC. 182BLOB contains Boot Info Header, followed by SP Manifest contents. 183 184The configuration of the boot protocol is done in the SP manifest. As defined by 185the specification, the manifest field 'gp-register-num' configures the GP register 186which shall be used to pass the address to the partitions boot information blob when 187booting the partition. 188 189Supported interfaces 190==================== 191 192The following interfaces are exposed to SPs only: 193 194- ``FFA_MSG_WAIT`` 195- ``FFA_MEM_RETRIEVE_REQ`` 196- ``FFA_MEM_RETRIEVE_RESP`` 197- ``FFA_MEM_RELINQUISH`` 198- ``FFA_SECONDARY_EP_REGISTER`` 199 200The following interfaces are exposed to both NS Client and SPs: 201 202- ``FFA_VERSION`` 203- ``FFA_FEATURES`` 204- ``FFA_RX_RELEASE`` 205- ``FFA_RXTX_MAP`` 206- ``FFA_RXTX_UNMAP`` 207- ``FFA_PARTITION_INFO_GET`` 208- ``FFA_ID_GET`` 209- ``FFA_MSG_SEND_DIRECT_REQ`` 210- ``FFA_MSG_SEND_DIRECT_RESP`` 211- ``FFA_MEM_FRAG_TX`` 212- ``FFA_SPM_ID_GET`` 213 214The following additional interfaces are forwarded from SPMD to support NS Client: 215 216- ``FFA_RUN`` 217- ``FFA_MEM_LEND`` 218- ``FFA_MEM_SHARE`` 219- ``FFA_MEM_FRAG_RX`` 220- ``FFA_MEM_RECLAIM`` 221 222 223FFA_VERSION 224----------- 225 226``FFA_VERSION`` requires a *requested_version* parameter from the caller. 227SPMD forwards call to SPMC, the SPMC returns its own implemented version. 228SPMC asserts SP and SPMC are at same FF-A Version. 229 230FFA_FEATURES 231------------ 232 233FF-A features supported by the SPMC may be discovered by secure partitions at 234boot (that is prior to NWd is booted) or run-time. 235 236The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 237FFA_SUCCESS from the SPMD. 238 239The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 240the response relayed back to the NWd. 241 242 243FFA_RXTX_MAP 244------------ 245 246FFA_RXTX_UNMAP 247-------------- 248 249When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 250receive buffers described by their PAs to the EL3 translation regime 251as secure buffers in the MMU descriptors. 252 253When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 254SPMC EL3 translation regime and marked as NS buffers in the MMU 255descriptors. 256 257The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 258caller, either it being the Hypervisor or OS kernel, as well as a secure 259partition. 260 261FFA_PARTITION_INFO_GET 262---------------------- 263 264Partition info get call can originate: 265 266- from SP to SPMC 267- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 268 269The format (v1.0 or v1.1) of the populated data structure returned is based upon the 270FFA version of the calling entity. 271 272EL3 SPMC also supports returning only the count of partitions deployed. 273 274All LSPs and SP are discoverable from FFA_PARTITION_INFO_GET call made by 275either SP or NWd entities. 276 277FFA_ID_GET 278---------- 279 280The FF-A ID space is split into a non-secure space and secure space: 281 282- FF-A ID with bit 15 clear relates to VMs. 283- FF-A ID with bit 15 set related to SPs or LSPs. 284- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor 285 (or OS Kernel if Hyp is absent), SPMD and SPMC. 286 287This convention helps the SPM to determine the origin and destination worlds in 288an FF-A ABI invocation. In particular the SPM shall filter unauthorized 289transactions in its world switch routine. It must not be permitted for a VM to 290use a secure FF-A ID as origin world by spoofing: 291 292- A VM-to-SP direct request/response shall set the origin world to be non-secure 293 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 294 set). 295- Similarly, an SP-to-LSP direct request/response shall set the FF-A ID bit 15 296 for both origin and destination IDs. 297 298An incoming direct message request arriving at SPMD from NWd is forwarded to 299SPMC without a specific check. The SPMC is resumed through eret and "knows" the 300message is coming from normal world in this specific code path. Thus the origin 301endpoint ID must be checked by SPMC for being a normal world ID. 302 303An SP sending a direct message request must have bit 15 set in its origin 304endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 305 306The SPMC shall reject the direct message if the claimed world in origin endpoint 307ID is not consistent: 308 309- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 310 world ID", 311- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 312 313 314FFA_MSG_SEND_DIRECT_REQ 315----------------------- 316 317FFA_MSG_SEND_DIRECT_RESP 318------------------------ 319 320This is a mandatory interface for secure partitions participating in direct request 321and responses with the following rules: 322 323- An SP can send a direct request to LSP. 324- An LSP can send a direct response to SP. 325- An SP cannot send a direct request to an Hypervisor or OS kernel. 326- An Hypervisor or OS kernel can send a direct request to an SP or LSP. 327- An SP and LSP can send a direct response to an Hypervisor or OS kernel. 328- SPMD can send direct request to SPMC. 329 330FFA_SPM_ID_GET 331-------------- 332 333Returns the FF-A ID allocated to an SPM component which can be one of SPMD 334or SPMC. 335 336At initialization, the SPMC queries the SPMD for the SPMC ID, using the 337FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using 338the FFA_SPM_ID_GET interface at the secure physical FF-A instance. 339 340Secure partitions call this interface at the virtual FF-A instance, to which 341the SPMC returns the SPMC ID. 342 343The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the 344SPMD, which returns the SPMC ID. 345 346FFA_ID_GET 347---------- 348 349Returns the FF-A ID of the calling endpoint. 350 351FFA_MEM_SHARE 352------------- 353 354FFA_MEM_LEND 355------------ 356 357- If SP is borrower in the memory transaction, these calls are forwarded to SPMC. 358 SPMC performs Relayer responsibilities, caches the memory descriptors in the datastore, 359 and allocates FF-A memory handle. 360- If format of descriptor was v1.0, SPMC converts the descriptor to v1.1 before caching. 361 In case of fragmented sharing, conversion of memory descriptors happens after last 362 fragment has been received. 363- Multiple borrowers (including NWd endpoint) and fragmented memory sharing are supported. 364 365FFA_MEM_RETRIEVE_REQ 366-------------------- 367 368FFA_MEM_RETRIEVE_RESP 369--------------------- 370 371- Memory retrieve is supported only from SP. 372- SPMC fetches the cached memory descriptor from the datastore, 373- Performs Relayer responsiilities and sends FFA_MEM_RETRIEVE_RESP back to SP. 374- If descriptor size is more than RX buffer size, SPMC will send the descriptor in fragments. 375- SPMC will set NS Bit to 1 in memory descriptor response. 376 377FFA_MEM_FRAG_RX 378--------------- 379 380FFA_MEM_FRAG_TX 381--------------- 382 383FFA_MEM_FRAG_RX is to be used by: 384 385- SP if FFA_MEM_RETRIEVE_RESP returned descriptor with fragment length less than total length. 386- or by SPMC if FFA_MEM_SHARE/FFA_MEM_LEND is called with fragment length less than total length. 387 388SPMC validates handle and Endpoint ID and returns response with FFA_MEM_FRAG_TX. 389 390FFA_SECONDARY_EP_REGISTER 391------------------------- 392 393When the SPMC boots, secure partition is initialized on its primary 394Execution Context. 395 396The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition 397from its first execution context, to provide the entry point address for 398secondary execution contexts. 399 400A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 401the NWd or by invocation of FFA_RUN. 402 403Power management 404================ 405 406In platforms with or without secure virtualization: 407 408- The NWd owns the platform PM policy. 409- The Hypervisor or OS kernel is the component initiating PSCI service calls. 410- The EL3 PSCI library is in charge of the PM coordination and control 411 (eventually writing to platform registers). 412- While coordinating PM events, the PSCI library calls backs into the Secure 413 Payload Dispatcher for events the latter has statically registered to. 414 415When using the SPMD as a Secure Payload Dispatcher: 416 417- A power management event is relayed through the SPD hook to the SPMC. 418- In the current implementation CPU_ON (svc_on_finish), CPU_OFF 419 (svc_off), CPU_SUSPEND (svc_suspend) and CPU_SUSPEND_RESUME (svc_suspend_finish) 420 hooks are registered. 421 422Secure partitions scheduling 423============================ 424 425The FF-A specification `[1]`_ provides two ways to relinquinsh CPU time to 426secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 427 428- the FFA_MSG_SEND_DIRECT_REQ interface. 429- the FFA_RUN interface. 430 431Additionally a secure interrupt can pre-empt the normal world execution and give 432CPU cycles by transitioning to EL3. 433 434Partition Runtime State and Model 435================================= 436 437EL3 SPMC implements Partition runtime states are described in v1.1 FF-A specification `[1]`_ 438 439An SP can be in one of the following state: 440 441- RT_STATE_WAITING 442- RT_STATE_RUNNING 443- RT_STATE_PREEMPTED 444- RT_STATE_BLOCKED 445 446An SP will transition to one of the following runtime model when not in waiting state: 447 448- RT_MODEL_DIR_REQ 449- RT_MODEL_RUN 450- RT_MODEL_INIT 451- RT_MODEL_INTR 452 453Platform topology 454================= 455 456SPMC only supports a single Pinned MP S-EL1 SP. The *execution-ctx-count* 457SP manifest field should match the number of physical PE. 458 459Interrupt handling 460================== 461 462Secure Interrupt handling 463------------------------- 464 465- SPMC is capable of forwarding Secure interrupt to S-EL1 SP 466 which has preempted the normal world. 467- Interrupt is forwarded to SP using FFA_INTERRUPT interface. 468- Interrupt Number is not passed, S-EL1 SP can access the GIC registers directly. 469- Upon completion of Interrupt handling SP is expected to return to 470 SPMC using FFA_MSG_WAIT interface. 471- SPMC returns to normal world after interrupt handling is completed. 472 473In the scenario when secure interrupt occurs while the secure partition is running, 474the SPMC is not involved and the handling is implementation defined in the TOS. 475 476Non-Secure Interrupt handling 477----------------------------- 478 479The 'managed exit' scenario is the responsibility of the TOS and the SPMC is not involved. 480 481Test Secure Payload (TSP) 482========================= 483 484- TSP provides reference implementation of FF-A programming model. 485- TSP has the following support: 486 487 - SP initialization on all CPUs. 488 - Consuming Power Messages including CPU_ON, CPU_OFF, CPU_SUSPEND, CPU_SUSPEND_RESUME. 489 - Event Loop to receive Direct Requests. 490 - Sending Direct Response. 491 - Memory Sharing helper library. 492 - Ability to handle secure interrupt (timer). 493 494TSP Tests in CI 495--------------- 496 497- TSP Tests are exercised in the TF-A CI using prebuilt FF-A Linux Test driver in NWd. 498- Expected output: 499 500.. code:: shell 501 502 #ioctl 255 503 Test: Echo Message to SP. 504 Status: Completed Test Case: 1 505 Test Executed Successfully 506 507 Test: Message Relay vis SP to EL3 LSP. 508 Status: Completed Test Case: 2 509 Test Executed Successfully 510 511 Test: Memory Send. 512 Verified 1 constituents successfully 513 Status: Completed Test Case: 3 514 Test Executed Successfully 515 516 Test: Memory Send in Fragments. 517 Verified 256 constituents successfully 518 Status: Completed Test Case: 4 519 Test Executed Successfully 520 521 Test: Memory Lend. 522 Verified 1 constituents successfully 523 Status: Completed Test Case: 5 524 Test Executed Successfully 525 526 Test: Memory Lend in Fragments. 527 Verified 256 constituents successfully 528 Status: Completed Test Case: 6 529 Test Executed Successfully 530 531 Test: Memory Send with Multiple Endpoints. 532 random: fast init done 533 Verified 256 constituents successfully 534 Status: Completed Test Case: 7 535 Test Executed Successfully 536 537 Test: Memory Lend with Multiple Endpoints. 538 Verified 256 constituents successfully 539 Status: Completed Test Case: 8 540 Test Executed Successfully 541 542 Test: Ensure Duplicate Memory Send Requests are Rejected. 543 Status: Completed Test Case: 9 544 Test Executed Successfully 545 546 Test: Ensure Duplicate Memory Lend Requests are Rejected. 547 Status: Completed Test Case: 10 548 Test Executed Successfully 549 550 0 Tests Failed 551 552 Exiting Test Application - Total Failures: 0 553 554 555References 556========== 557 558.. _[1]: 559 560[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 561 562.. _[2]: 563 564[2] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fvp_el3_spmc_logical_sp.c 565 566.. _[3]: 567 568[3] `Trusted Boot Board Requirements 569Client <https://developer.arm.com/documentation/den0006/d/>`__ 570 571.. _[4]: 572 573[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fvp_el3_spmc.c 574 575.. _[5]: 576 577[5] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/include/platform_def.h 578 579.. _[6]: 580 581[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 582 583.. _[7]: 584 585[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_tsp_sp_manifest.dts 586 587.. _[8]: 588 589[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/ 590 591.. _[9]: 592 593[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 594 595-------------- 596 597*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.* 598