1# Scheduler VM expectations
2
3Hafnium requires there to be a special 'primary' or 'scheduler' VM which is
4responsible for scheduling the other VMs. There are some particular expectations
5on this VM that are required for the rest of the system to function normally.
6
7[TOC]
8
9## Scheduling
10
11The scheduler VM is responsible for scheduling the vCPUs of all the other VMs.
12It should request information about the VMs in the system using the
13`FFA_PARTITION_INFO_GET` function, and then schedule their vCPUs as it wishes.
14The recommended way of doing this is to create a kernel thread for each vCPU,
15which will repeatedly run that vCPU by calling `FFA_RUN`.
16
17`FFA_RUN` will return one of several possible functions, which must be handled
18as follows:
19
20### `FFA_INTERRUPT`
21
22The vCPU has been preempted but still has work to do. If the scheduling quantum
23has not expired, the scheduler MUST call `FFA_RUN` on the vCPU to allow it to
24continue.
25
26If `w1` is non-zero, then Hafnium would like `FFA_RUN` to be called on the vCPU
27specified there. The scheduler MUST either wake the vCPU in question up if it is
28blocked, or preempt and re-run it if it is already running somewhere. This gives
29Hafnium a chance to update any CPU state which might have changed. The scheduler
30should call `FFA_RUN` again on the sending VM as usual.
31
32### `FFA_YIELD`
33
34The vCPU has voluntarily yielded the CPU. The scheduler SHOULD take a scheduling
35decision to give cycles to those that need them but MUST call `FFA_RUN` on the
36vCPU at a later point.
37
38### `FFA_MSG_WAIT`
39
40The vCPU is blocked waiting for a message. The scheduler MUST take it off the
41run queue and not call `FFA_RUN` on the vCPU until it has either:
42
43*   injected an interrupt
44*   sent it a message
45*   received `FFA_INTERRUPT` for it from another vCPU
46*   the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
47    specified duration has expired.
48
49### `FFA_MSG_SEND`
50
51A message has been sent by the vCPU. If the recipient is the scheduler VM itself
52then it can handle it as it pleases. Otherwise the scheduler MUST run a vCPU
53from the recipient VM and priority SHOULD be given to those vCPUs that are
54waiting for a message. The scheduler should call FFA_RUN again on the sending
55VM as usual.
56
57### `FFA_RX_RELEASE`
58
59The vCPU has made the mailbox writable and there are pending waiters. The
60scheduler MUST call `hf_mailbox_waiter_get()` repeatedly and notify all waiters
61by injecting an `HF_MAILBOX_WRITABLE_INTID` interrupt. The scheduler should call
62FFA_RUN again on the sending VM as usual.
63
64### `HF_FFA_RUN_WAIT_FOR_INTERRUPT`
65
66_This is a Hafnium-specific function not part of the FF-A standard._
67
68The vCPU is blocked waiting for an interrupt. The scheduler MUST take it off the
69run queue and not call `FFA_RUN` on the vCPU until it has either:
70
71*   injected an interrupt
72*   received `FFA_INTERRUPT` for it from another vCPU
73*   the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
74    specified duration has expired.
75
76### `FFA_ERROR`
77
78#### `FFA_ABORTED`
79
80The vCPU has aborted triggering the whole VM to abort. The scheduler MUST treat
81this the same as `FFA_INTERRUPT` for all the other vCPUs of the VM. For this
82vCPU the scheduler SHOULD either never call FFA_RUN on the vCPU again, or treat
83it the same as `HF_FFA_RUN_WAIT_FOR_INTERRUPT`.
84
85#### Any other error code
86
87This should not happen if the scheduler VM has called `FFA_RUN` correctly, but
88in case there is some other error it should be logged. The scheduler SHOULD
89either try again or suspend the vCPU indefinitely.
90
91## Interrupt handling
92
93The scheduler VM is responsible for handling all hardware interrupts. Many of
94these will be intended for the scheduler VM itself and it can handle them as
95usual. However, it must also:
96
97*   Enable, handle and ignore interrupts for the non-secure hypervisor physical
98    timer (PPI 10, IRQ 26).
99*   Forward interrupts intended for secondary VMs to an appropriate vCPU of the
100    VM by calling `hf_interrupt_inject` and then running the vCPU as usual with
101    `FFA_RUN`. (If the vCPU is already running at the time that
102    `hf_interrupt_inject` is called then it must be preempted and run again so
103    that Hafnium can inject the interrupt.)
104