1.. _ivshmem-hld:
2
3ACRN Shared Memory Based Inter-VM Communication
4###############################################
5
6ACRN supports inter-virtual machine communication based on a shared
7memory mechanism. The ACRN Device Model or hypervisor emulates a virtual
8PCI device (called an ``ivshmem`` device) to expose the base address and
9size of this shared memory.
10
11Inter-VM Communication Overview
12*******************************
13
14.. figure:: images/ivshmem-architecture.png
15   :align: center
16   :name: ivshmem-architecture-overview
17
18   ACRN Shared Memory Based Inter-VM Communication Architecture
19
20ACRN can emulate the ``ivshmem`` device in two ways:
21
22``ivshmem`` dm-land
23   The ``ivshmem`` device is emulated in the ACRN Device Model,
24   and the shared memory regions are reserved in the Service VM's
25   memory space.  This solution only supports communication between
26   post-launched User VMs.
27
28``ivshmem`` hv-land
29   The ``ivshmem`` device is emulated in the hypervisor, and the
30   shared memory regions are reserved in the hypervisor's
31   memory space.  This solution works for both pre-launched and
32   post-launched User VMs.
33
34While both solutions can be used at the same time, inter-VM communication
35may only be done between VMs using the same solution.
36
37ivshmem hv:
38   The **ivshmem hv** implements register virtualization
39   and shared memory mapping in the ACRN hypervisor.
40   A notification/interrupt mechanism is supported.
41
42ivshmem dm:
43   The **ivshmem dm** implements register virtualization
44   and shared memory mapping in the ACRN Device Model (``acrn-dm``).
45   It will support a notification/interrupt mechanism in the future.
46
47ivshmem server:
48   With **ivshmem server** support, VMs with ivshmem devices enabled can send
49   notifications (interrupts) to each other by writing the target peer ID (VM
50   ID) and vector index to the doorbell register of the ivshmem device. The
51   **ivshmem server** forwards this notification event to the target VM.
52
53   Two types of **ivshmem server** are defined in ACRN:
54
55   * User land **ivshmem server** is a daemon in user space to forward
56     notifications for **dm-land** ivshmem devices only, by co-working with
57     **ivshmem dm**. User land **ivshmem server** is not implemented.
58
59   * HV-land **ivshmem server** plays a role similar to the user land **ivshmem
60     server**, but it is a hypervisor module and forwards notifications (virtual
61     interrupts) to the target VM with  **hv-land** ivshmem devices enabled.
62
63.. figure:: images/ivshmem-hv-land-doorbell.png
64   :align: center
65   :name: ivshmem-hv-land-doorbell-overview
66
67   ACRN Ivshmem HV-Land Doorbell Overview
68
69Ivshmem Device Introduction
70***************************
71
72The ``ivshmem`` device is a virtual standard PCI device consisting of
73three Base Address Registers (BARs):
74
75* BAR0 is used for emulating interrupt-related registers.
76* BAR1 is used for emulating MSIX entry table.
77* BAR2 is used for exposing a shared memory region.
78
79The ``ivshmem`` device supports no extra capabilities.
80
81Configuration Space Definition
82
83+---------------+----------+----------+
84| Register      | Offset   | Value    |
85+===============+==========+==========+
86| Vendor ID     | 0x00     | 0x1AF4   |
87+---------------+----------+----------+
88| Device ID     | 0x02     | 0x1110   |
89+---------------+----------+----------+
90| Revision ID   | 0x08     | 0x1      |
91+---------------+----------+----------+
92| Class Code    | 0x09     | 0x5      |
93+---------------+----------+----------+
94
95
96MMIO Registers Definition
97
98.. list-table::
99   :widths: auto
100   :header-rows: 1
101
102   * - Register
103     - Offset
104     - Read/Write
105     - Description
106   * - IVSHMEM\_IRQ\_MASK\_REG
107     - 0x0
108     - R/W
109     - Interrupt Mask register is used for legacy interrupt.
110       IVSHMEM doesn't support legacy INTx interrupts, so this register
111       is reserved.
112   * - IVSHMEM\_IRQ\_STA\_REG
113     - 0x4
114     - R/W
115     - Interrupt Status register is used for legacy interrupt.
116       IVSHMEM doesn't support legacy INTx interrupts, so this register
117       is reserved.
118   * - IVSHMEM\_IV\_POS\_REG
119     - 0x8
120     - RO
121     - Inter-VM Position register is used to identify the device.
122       Its value is equal to the ID of the VM (to which the device belongs), in the context of ACRN.
123   * - IVSHMEM\_DOORBELL\_REG
124     - 0xC
125     - WO
126     - Doorbell register is used to trigger an interrupt to the peer VM.
127
128Usage
129*****
130
131For usage information, see :ref:`enable_ivshmem`.
132
133Inter-VM Communication Security Hardening (BKMs)
134************************************************
135
136Inter-VM communication enables you to create shared
137memory regions between post-launched User VMs.
138This mechanism is based on ivshmem v1.0 and exposes virtual PCI devices for the
139shared regions (in the Service VM's memory). This feature adopts a
140community-approved design for shared memory between VMs, following the same
141specification for KVM/QEMU (`Link <https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/specs/ivshmem-spec.txt;hb=HEAD>`_).
142
143Following the ACRN threat model, the policy definition for allocation and
144assignment of these regions is controlled by the Service VM, which is part of
145ACRN's Trusted Computing Base (TCB). However, to secure inter-VM communication
146between any user space applications that harness this channel, applications will
147face more requirements for the confidentiality, integrity, and authenticity of
148shared or transferred data. The application development team is
149responsibility for defining a threat model and security architecture for the
150application and utilizing custom or public libraries accordingly. This document
151provides an overview about potential hardening techniques from a user space
152application's perspective. Consider these techniques when defining the security
153architecture and threat model for your application.
154
155.. note:: This document is not a definitive guide on all security technologies
156   or how to implement security. We provide general pointers not bounded to a
157   specific OS or use case.
158
1591. **Secure Feature Configurability**
160
161   - ACRN ensures a minimal control plane for the configuration of the memory
162     region's boundaries and name handles. This capability is managed only by
163     the Service VM during the creation of the guest VM through the Device Model
164     (DM).
165   - Create different permissions or groups for the ``admin`` role to isolate it
166     from other entities that might have access to the Service VM. For example,
167     only admin permissions allow R/W/X on the DM binary.
168
1692. **Apply Access Control**
170
171   - Add restrictions based on behavior or subject and object rules around
172     information flow and accesses.
173   - In the Service VM, consider the ``/dev/shm`` device node as a critical
174     interface with special access requirements. Those requirements can be
175     fulfilled using any of the existing open source MAC technologies or even
176     ACLs depending on the OS compatibility (Ubuntu, Windows, etc.) and
177     integration complexity.
178   - In the User VM, the shared memory region can be accessed using the
179     ``mmap()`` of the UIO device node. Other complementary information can be
180     found under:
181
182     - ``/sys/class/uio/uioX/device/resource2`` --> shared memory base address
183     - ``/sys/class/uio/uioX/device/config`` --> shared memory size
184
185   - For Linux-based User VMs, we recommend using the standard ``UIO`` and
186     ``UIO_PCI_GENERIC`` drivers through the device node (for example,
187     ``/dev/uioX``).
188   - Reference: `AppArmor <https://wiki.ubuntuusers.de/AppArmor/>`_, `SELinux
189     <https://selinuxproject.org/page/Main_Page>`_, `UIO driver-API
190     <https://www.kernel.org/doc/html/v4.12/driver-api/uio-howto.html>`_
191
192
1933. **Crypto Support and Secure Applied Crypto**
194
195   - According to the application's threat model and the defined assets that
196     need to be shared securely, define the requirements for crypto algorithms.
197     Those algorithms should enable operations such as authenticated encryption
198     and decryption, secure key exchange, true random number generation, and
199     seed extraction. In addition, consider the landscape of your attack surface
200     and define the need for a security engine (for example, CSME services).
201   - Don't implement your own crypto functions. Use available compliant crypto
202     libraries as applicable, such as `Intel IPP
203     <https://github.com/intel/ipp-crypto>`_ or `TinyCrypt
204     <https://01.org/tinycrypt>`_.
205   - Utilize the platform/kernel infrastructure and services (for example,
206     :ref:`hld-security`, `Kernel Crypto backend/APIs
207     <https://www.kernel.org/doc/html/v5.4/crypto/index.html>`_, and `keyring
208     subsystem <https://www.man7.org/linux/man-pages/man7/keyrings.7.html>`_).
209   - Implement necessary flows for key lifecycle management, including wrapping,
210     revocation, and migration, depending on the crypto key type and
211     requirements for key persistence across system and power management events.
212   - Follow open source secure crypto coding guidelines for secure wrappers and
213     marshaling data structures: `Secure Applied Crypto <https://github.com
214     veorq/cryptocoding>`_
215   - References: `NIST Crypto Standards and Guidelines
216     <https://csrc.nist.gov/projects/cryptographic-standards-and-guidelines>`_,
217     `OpenSSL <https://www.openssl.org/>`_
218
219
2204. **Applications Passlisting**
221
222   - For use cases implemented in static environments (for example, Industrial
223     and Automotive usages), follow application approval techniques and disable
224     any third-party or native app stores.
225   - This mechanism can be chained with the access control policies to protect
226     access to passlisting rules and configuration files (refer to open source
227     or implement your custom solution).
228   - References: `NIST SP800-167
229     <https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-167.pdf>`_,
230     `fapolicyd <https://github.com/linux-application-whitelisting/fapolicyd>`_
231
232
2335. **Secure Boot and File System Integrity Verification**
234
235   - The previously highlighted technologies rely on the kernel, as a secure
236     component, to enforce such policies. We strongly recommend enabling secure
237     boot for the Service VM, and extend the secure boot chain to any
238     post-launched VM kernels.
239   - To ensure that no malware is introduced or persists, utilize the file
240     system (FS) verification methods on every boot to extend the secure boot
241     chain for post-launch VMs (kernel/FS).
242   - Reference: :ref:`how-to-enable-secure-boot-for-windows`
243   - Reference Stack:  `dm-verity
244     <https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/verity.html>`_
245
246.. note:: All the mentioned hardening techniques might require minor extra
247   development efforts.
248