1.. _hv_vcat:
2
3Virtual Cache Allocation Technology (vCAT)
4###########################################
5
6vCAT refers to the virtualization of Cache Allocation Technology (CAT), one of the
7RDT (Resource Director Technology) technologies.
8
9ACRN vCAT is built on top of ACRN RDT: ACRN RDT provides a number of physical CAT resources
10(COS IDs + cache ways), ACRN vCAT exposes some number of virtual CAT resources to VMs
11and then transparently map them to the assigned physical CAT resources in the ACRN hypervisor;
12VM can take advantage of vCAT to prioritize and partition virtual cache ways for its own tasks.
13
14In current CAT implementation, one COS ID corresponds to one ``IA32_type_MASK_n`` (type: L2 or L3,
15n ranges from 0 to ``MAX_CACHE_CLOS_NUM_ENTRIES`` - 1) MSR and a bit in a capacity bitmask (CBM)
16corresponds to one cache way.
17
18On current generation systems, normally L3 cache is shared by all CPU cores on the same socket and
19L2 cache is generally just shared by the hyperthreads on a core. But when dealing with ACRN
20vCAT COS IDs assignment, it is assumed that all the L2/L3 caches (and therefore all COS IDs)
21are system-wide caches shared by all cores in the system, this is done for convenience and to simplify
22the vCAT configuration process. If vCAT is enabled for a VM (abbreviated as vCAT VM), there should not
23be any COS ID overlap between a vCAT VM and any other VMs. e.g. the vCAT VM has exclusive use of the
24assigned COS IDs.
25When assigning cache ways, however, the VM can be given exclusive, shared, or mixed access to the cache
26ways depending on particular performance needs. For example, use dedicated cache ways for RTVM, and use
27shared cache ways between low priority VMs.
28
29In ACRN, the CAT resources allocated for vCAT VMs are determined in :ref:`rdt_configuration`.
30
31For further details on the RDT, refer to the ACRN RDT high-level design :ref:`hv_rdt`.
32
33
34High Level ACRN vCAT Design
35***************************
36
37ACRN CAT virtualization support can be divided into two parts:
38
39- CAT Capability Exposure to Guest VM
40
41- CAT resources (COS IDs + cache ways) management
42
43The figure below shows high-level design of vCAT in ACRN:
44
45   .. figure:: images/vcat-hld.png
46      :align: center
47
48CAT Capability Exposure to Guest VM
49***********************************
50ACRN exposes CAT capability and resource to a Guest VM via vCPUID and vMSR, as explained
51in the following sections.
52
53vCPUID
54======
55
56CPUID Leaf 07H
57--------------
58
59- CPUID.(EAX=07H, ECX=0).EBX.PQE[bit 15]: Supports RDT capability if 1. This bit will be set for a vCAT VM.
60
61CPUID Leaf 10H
62--------------
63
64**CAT Resource Type and Capability Enumeration**
65
66- CPUID.(EAX=10H, ECX=0):EBX[1]: If 1, indicate L3 CAT support for a vCAT VM.
67- CPUID.(EAX=10H, ECX=0):EBX[2]: If 1, indicate L2 CAT support for a vCAT VM.
68- CPUID.(EAX=10H, ECX=1): CAT capability enumeration sub-leaf for L3. Reports L3 COS_MAX and CBM_LEN to a vCAT VM
69- CPUID.(EAX=10H, ECX=2): CAT capability enumeration sub-leaf for L2. Reports L2 COS_MAX and CBM_LEN to a vCAT VM
70
71vMSR
72====
73
74The following CAT MSRs will be virtualized for a vCAT VM:
75
76- IA32_PQR_ASSOC
77- IA32_type_MASK_0 ~ IA32_type_MASK_n
78
79By default, after reset, all CPU cores are assigned to COS 0 and all IA32_type_MASK_n MSRs
80are programmed to allow fill into all cache ways.
81
82
83CAT resources (COS IDs + cache ways) management
84************************************************
85
86All accesses to the CAT MSRs are intercepted by vMSR and control is passed to vCAT, which will perform
87the following actions:
88
89- Intercept IA32_PQR_ASSOC MSR to re-map virtual COS ID to physical COS ID.
90  Upon writes, store the re-mapped physical COS ID into its vCPU ``msr_store_area``
91  data structure guest part. It will be loaded to physical IA32_PQR_ASSOC on each VM-Enter.
92
93
94- Intercept IA32_type_MASK_n MSRs to re-map virtual CBM to physical CBM. Upon writes,
95  program re-mapped physical CBM into corresponding physical IA32_type_MASK_n MSR
96
97  Several vCAT P2V (physical to virtual) and V2P (virtual to physical)
98  mappings exist, as illustrated in the following pseudocode:
99
100.. code-block:: none
101
102       struct acrn_vm_config *vm_config = get_vm_config(vm_id)
103
104       max_pcbm = vm_config->max_type_pcbm (type: l2 or l3)
105       mask_shift = ffs64(max_pcbm)
106
107       vcosid = vmsr - MSR_IA32_type_MASK_0
108       pcosid = vm_config->pclosids[vcosid]
109
110       pmsr = MSR_IA32_type_MASK_0 + pcosid
111       pcbm = vcbm << mask_shift
112       vcbm = pcbm >> mask_shift
113
114Where
115       ``vm_config->pclosids[]``: array of physical COS IDs, where each corresponds to one ``vcpu_clos`` that
116       is defined in the scenario file
117
118       ``max_pcbm``: a bitmask that selects all the physical cache ways assigned to the VM, corresponds to
119       the nth ``CLOS_MASK`` that is defined in scenario file, where n = the first physical COS ID assigned
120       = ``vm_config->pclosids[0]``
121
122       ``ffs64(max_pcbm)``: find the first (least significant) bit set in ``max_pcbm`` and return
123       the index of that bit.
124
125       ``MSR_IA32_type_MASK_0``: 0xD10 for L2, 0xC90 for L3
126
127       ``vcosid``: virtual COS ID, always starts from 0
128
129       ``pcosid``: corresponding physical COS ID for a given ``vcosid``
130
131       ``vmsr``: virtual MSR address, passed to vCAT handlers by the
132       caller functions ``rdmsr_vmexit_handler()``/``wrmsr_vmexit_handler()``
133
134       ``pmsr``: physical MSR address
135
136       ``vcbm``: virtual CBM, passed to vCAT handlers by the
137       caller functions ``rdmsr_vmexit_handler()``/``wrmsr_vmexit_handler()``
138
139       ``pcbm``: physical CBM
140