1.. _rdt_configuration: 2 3Enable Intel Resource Director Technology (RDT) Configurations 4############################################################### 5 6About Intel Resource Director Technology (RDT) 7********************************************** 8 9On x86 platforms that support Intel Resource Director Technology (RDT) 10allocation features, the ACRN hypervisor can help minimize cache misses by 11partitioning the shared cache among VMs. "Shared cache" refers to cache that is 12shared among multiple CPU cores. By default, VMs running on these cores are 13configured to use the entire cache, effectively sharing the cache among all VMs 14and without any partitioning. This design choice may cause too many cache misses 15for applications running in higher-priority VMs, such as real-time VMs (RTVM), 16negatively affecting their performance. The ACRN hypervisor can help minimize 17cache misses and improve performance by isolating a portion of the shared cache 18for a specific VM. 19 20ACRN supports the following features: 21 22Cache Allocation Technology (CAT) 23 The CAT support in the hypervisor isolates a portion of the cache for a VM from 24 other VMs. Generally, certain cache resources are allocated for the RTVMs to 25 reduce performance interference by other VMs attempting to use the same cache. 26 27 28Code and Data Prioritization (CDP) 29 The CDP feature in RDT is an extension of CAT that enables separate control over 30 code and data placement in the cache. The CDP support in the hypervisor isolates 31 a portion of the cache for code and another portion for data for the same VM. 32 33 34Virtual Cache Allocation Technology (vCAT) 35 ACRN also supports virtualizing CAT, referred to as vCAT. With 36 vCAT enabled, the hypervisor presents CAT to a selected set of VMs, allowing the 37 guest OSs to further isolate the cache used by higher-priority processes in 38 those VMs. 39 40Dependencies and Constraints 41***************************** 42 43Consider the following dependencies and constraints: 44 45* The hardware must support RDT for ACRN to enable RDT support in the 46 hypervisor. 47 48* The cache must be shared cache (cache shared across multiple CPU cores) and 49 not private cache (cache that is owned by only one CPU core). If the 50 cache is private, CAT, CDP, and vCAT have no benefit because the cache is 51 already exclusively used by one core. For this reason, the ACRN Configurator 52 will not allow you to configure private cache. 53 54* The ACRN Configurator displays CAT configuration parameters based on data 55 reported in the board configuration file. Some Intel platforms are known to 56 have L3 CAT even though it's not reported via the architectural CPUID 57 interface. In this situation, the board inspector detects availability of L3 58 CAT by trying to access the CAT MSRs directly. Versions of the board inspector 59 before v3.1 only used the CPUID interface and would indicate L3 CAT wasn't 60 supported in this circumstance. 61 62* The guest OS in a VM with vCAT enabled requires utilities in that OS for 63 further cache allocation configurations. An example is the `resctrl 64 <https://docs.kernel.org/x86/resctrl.html>`__ framework in Linux. 65 66Configuration Overview 67********************** 68 69You can allocate cache to each VM at the virtual CPU (vCPU) level. For example, 70you can create a post-launched real-time VM and assign three physical CPU 71cores to it. ACRN assigns a vCPU ID to each physical CPU. Furthermore, you can 72specify a vCPU as a real-time vCPU. Then you can allocate a portion of the cache 73to the real-time vCPU and allocate the rest of the cache to be shared among the 74other vCPUs. This type of configuration allows the real-time vCPU to use its 75assigned cache without interference from the other vCPUs, thus improving the 76performance of applications running on the real-time vCPU. The following 77documentation is a general overview of the configuration process. 78 79The :ref:`acrn_configurator_tool` provides a user interface to help you allocate 80cache to vCPUs. The configuration process requires setting VM parameters, then 81allocating cache to the VMs via an interface in the hypervisor parameters. This 82documentation presents the configuration process as a linear flow, but in 83reality you may find yourself moving back and forth between setting the 84hypervisor parameters and the VM parameters until you are satisfied with the 85entire configuration. 86 87For a real-time VM, you must set the following parameters in the VM's **Basic 88Parameters**: 89 90* **VM type**: Select **Real-time**. 91 92* **pCPU ID**: Select the physical CPU affinity for the VM. 93 94* **Virtual CPU ID**: Note the vCPU ID that the tool assigns to each physical 95 CPU. You will need to know the ID when you are ready to allocate cache. 96 97* **Real-time vCPU**: Select the Real-time vCPU check box next to each real-time 98 vCPU. The ACRN Configurator uses this information to create default cache 99 configurations, as you will see later in this documentation. If you change the 100 VM type from Real-time to Standard, the ACRN Configurator disables the 101 Real-time vCPU check box. 102 103 Some vCPUs of an RTVM may be used for housekeeping tasks and not need the 104 real-time performance optimization provided by advanced cache configuration 105 isolation. You indicate this by leaving this Real-time vCPU option unchecked. 106 Housekeeping tasks are generally related to kernel tasks to maintain its 107 internal state and services such as updating internal statistics for the 108 scheduler or timers, and User VM launch or exit. 109 110.. image:: images/configurator-rt01.png 111 :align: center 112 :class: drop-shadow 113 114To use vCAT for the VM, you must also set the following parameters in the VM's 115**Advanced Parameters**: 116 117* **Maximum virtual CLOS**: Select the maximum number of virtual CLOS masks. 118 This parameter defines the number of cache chunks that you will see in the 119 hypervisor parameters. 120 121* Select **VM Virtual Cache Allocation Tech**. 122 123.. image:: images/configurator-vcat01.png 124 :align: center 125 :class: drop-shadow 126 127Next, you can enable Intel RDT features in **Hypervisor Global Settings > 128Advanced Parameters > Memory Isolation for Performance**. You can enable one of 129the following combinations of features: 130 131* Cache Allocation Technology (CAT) alone 132 133* Cache Allocation Technology plus Code and Data Prioritization (CDP) 134 135* Cache Allocation Technology plus Virtual Cache Allocation Technology (vCAT) 136 137The following figure shows Cache Allocation Technology enabled: 138 139.. image:: images/configurator-cache01.png 140 :align: center 141 :class: drop-shadow 142 143When CDP or vCAT is enabled, CAT must be enabled too. The tool selects CAT if it's not already selected. 144 145.. image:: images/configurator-cache02.png 146 :align: center 147 :class: drop-shadow 148 149CDP and vCAT can't be enabled at the same time, so the tool clears the vCAT check box when CDP is selected and vice versa. 150 151Based on your selection, the tool displays the available cache in tables. 152Example: 153 154.. image:: images/configurator-cache03.png 155 :align: center 156 :class: drop-shadow 157 158The table title shows important information: 159 160* Cache level, such as Level 3 (L3) or Level 2 (L2) 161 162* Physical CPU cores that can access the cache 163 164The above example shows an L2 cache table. VMs assigned to any CPU cores 2-6 can 165have cache allocated to them. 166 167The table's left-most column shows the names of all VMs that are assigned to the 168CPU cores noted in the table title, as well as their vCPU IDs. The table 169categorizes the vCPUs as either standard or real-time. The real-time vCPUs are 170those that are set as real-time in the VM's parameters. All other vCPUs are 171considered standard. The above example shows one real-time vCPU (VM1 vCPU 2) and 172two standard vCPUs (VM0 vCPU 2 and 6). 173 174.. note:: 175 176 The Service VM is automatically assigned to all CPUs, so it appears in the standard category in all cache tables. 177 178The table's rows show the number of available cache chunks. You can see the 179size of each cache chunk in the note below the table. In the above example, 20 180cache chunks are available to allocate to the VMs, and each cache chunk is 64KB. 181All cache chunks are yellow, which means all of them are allocated to all VMs. 182All VMs share the entire cache. 183 184The **Apply basic real-time defaults** button creates a basic real-time 185configuration if real-time vCPUs exist. If there are no real-time vCPUs, the 186button will not do anything. 187 188If you select Cache Allocation Technology (CAT) alone, the **Apply basic 189real-time defaults** button allocates a different cache chunk to each real-time 190vCPU, making sure it doesn't overlap the cache of any other vCPU. The rest of 191the cache is shared among the standard vCPUs. In the following example, only VM1 192vCPU 2 can use cache chunk19, while all other vCPUs share the rest of the cache. 193 194.. image:: images/configurator-cache04.png 195 :align: center 196 :class: drop-shadow 197 198If you select CAT with Code and Data Prioritization, you can allocate different 199cache chunks to code or data on the same vCPU. The **Apply basic real-time 200defaults** button allocates one cache chunk to code on the real-time vCPU and a 201different cache chunk to data on the same vCPU, making sure the cache chunks 202don't overlap any others. In the following example, VM1 vCPU 2 can use cache 203chunk19 for code and chunk18 for data, while all other vCPUs share the rest of 204the cache. 205 206.. image:: images/configurator-cache05.png 207 :align: center 208 :class: drop-shadow 209 210To further customize the cache allocation, you can drag the right or left edges 211of the yellow boxes to cover the cache chunks that you want to allocate to 212specific VMs. 213 214.. note:: 215 216 If you have a real-time VM, ensure its cache chunks do not overlap with any 217 other VM's cache chunks. 218 219The tool helps you create valid configurations based on the underlying platform 220architecture. For example, it is only possible to assign consecutive cache 221chunks to a vCPU; there can be no gaps. Also, a vCPU must have access to at 222least one cache chunk. 223 224Learn More 225********** 226 227For details on the ACRN RDT high-level design, see :ref:`hv_rdt`. 228 229For details about RDT, see 230`Intel 64 and IA-32 Architectures Software Developer's Manual (SDM), Volume 3, 231(Section 17.19 Intel Resource Director Technology Allocation Features) 232<https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html>`_. 233 234For details on the ACRN vCAT high-level design, see :ref:`hv_vcat`. 235
