1# Scheduler VM expectations
2
3Hafnium requires there to be a special 'primary' or 'scheduler' VM which is
4responsible for scheduling the other VMs. There are some particular expectations
5on this VM that are required for the rest of the system to function normally.
6
7## Scheduling
8
9The scheduler VM is responsible for scheduling the vCPUs of all the other VMs.
10It should request information about the VMs in the system using the
11`FFA_PARTITION_INFO_GET` function, and then schedule their vCPUs as it wishes.
12The recommended way of doing this is to create a kernel thread for each vCPU,
13which will repeatedly run that vCPU by calling `FFA_RUN`.
14
15`FFA_RUN` will return one of several possible functions, which must be handled
16as follows:
17
18### `FFA_INTERRUPT`
19
20The vCPU has been preempted but still has work to do. If the scheduling quantum
21has not expired, the scheduler MUST call `FFA_RUN` on the vCPU to allow it to
22continue.
23
24If `w1` is non-zero, then Hafnium would like `FFA_RUN` to be called on the vCPU
25specified there. The scheduler MUST either wake the vCPU in question up if it is
26blocked, or preempt and re-run it if it is already running somewhere. This gives
27Hafnium a chance to update any CPU state which might have changed. The scheduler
28should call `FFA_RUN` again on the sending VM as usual.
29
30### `FFA_YIELD`
31
32The vCPU has voluntarily yielded the CPU. The scheduler SHOULD take a scheduling
33decision to give cycles to those that need them but MUST call `FFA_RUN` on the
34vCPU at a later point.
35
36### `FFA_MSG_WAIT`
37
38The vCPU is blocked waiting for a message. The scheduler MUST take it off the
39run queue and not call `FFA_RUN` on the vCPU until it has either:
40
41*   injected an interrupt
42*   sent it a message
43*   received `FFA_INTERRUPT` for it from another vCPU
44*   the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
45    specified duration has expired.
46
47### `FFA_MSG_SEND`
48
49A message has been sent by the vCPU. If the recipient is the scheduler VM itself
50then it can handle it as it pleases. Otherwise the scheduler MUST run a vCPU
51from the recipient VM and priority SHOULD be given to those vCPUs that are
52waiting for a message. The scheduler should call `FFA_RUN` again on the sending
53VM as usual.
54
55### `FFA_RX_RELEASE`
56
57The vCPU has made the mailbox writable and there are pending waiters. The
58scheduler MUST call `hf_mailbox_waiter_get()` repeatedly and notify all waiters
59by injecting an `HF_MAILBOX_WRITABLE_INTID` interrupt. The scheduler should call
60`FFA_RUN` again on the sending VM as usual.
61
62### `HF_FFA_RUN_WAIT_FOR_INTERRUPT`
63
64_This is a Hafnium-specific function not part of the FF-A standard._
65
66The vCPU is blocked waiting for an interrupt. The scheduler MUST take it off the
67run queue and not call `FFA_RUN` on the vCPU until it has either:
68
69*   injected an interrupt
70*   received `FFA_INTERRUPT` for it from another vCPU
71*   the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
72    specified duration has expired.
73
74### `FFA_ERROR`
75
76#### `FFA_ABORTED`
77
78The vCPU has aborted triggering the whole VM to abort. The scheduler MUST treat
79this the same as `FFA_INTERRUPT` for all the other vCPUs of the VM. For this
80vCPU the scheduler SHOULD either never call `FFA_RUN` on the vCPU again, or treat
81it the same as `HF_FFA_RUN_WAIT_FOR_INTERRUPT`.
82
83#### Any other error code
84
85This should not happen if the scheduler VM has called `FFA_RUN` correctly, but
86in case there is some other error it should be logged. The scheduler SHOULD
87either try again or suspend the vCPU indefinitely.
88
89## Interrupt handling
90
91The scheduler VM is responsible for handling all hardware interrupts. Many of
92these will be intended for the scheduler VM itself and it can handle them as
93usual. However, it must also:
94
95*   Enable, handle and ignore interrupts for the non-secure hypervisor physical
96    timer (PPI 10, IRQ 26).
97*   Forward interrupts intended for secondary VMs to an appropriate vCPU of the
98    VM by calling `hf_interrupt_inject` and then running the vCPU as usual with
99    `FFA_RUN`. (If the vCPU is already running at the time that
100    `hf_interrupt_inject` is called then it must be preempted and run again so
101    that Hafnium can inject the interrupt.)
102