1Xen PV Console notes
2------------------------------------------------------------------------
3                                                      Stefano Stabellini
4                                        stefano.stabellini@eu.citrix.com
5
6
7Xen traditionally provided a single pv console to pv guests, storing the
8relevant information in xenstore under /local/domain/$DOMID/console.
9
10Now many years after the introduction of the pv console we have
11multiple pv consoles support for pv and hvm guests; multiple pv
12console backends (qemu and xenconsoled, see limitations below) and
13emulated serial cards too.
14
15This document tries to describe how the whole system works and how the
16different components interact with each other.
17
18The first PV console path in xenstore remains:
19
20/local/domain/$DOMID/console
21
22The virtual UART console path in xenstore is defined as:
23
24/local/domain/$DOMID/vuart/0
25
26The vuart console provides access to a virtual SBSA UART on ARM systems.
27To enable vuart the following line has to be added to the guest configuration
28file:
29
30vuart = "sbsa_uart"
31
32In Linux you can select the virtual SBSA UART by using the "ttyAMA0"
33console instead of "hvc0".
34
35The other PV consoles follow the conventional xenstore device path and
36live in:
37
38/local/domain/$DOMID/device/console/$DEVID.
39
40PV consoles have
41* (optional) string names;
42* 'connection' information describing to what they should be
43  connected; and
44* a 'type' indicating which daemon should process the data.
45
46We call a PV console with a name a "channel", in reference to the libvirt
47concept with the same name and purpose. The file "channels.txt" describes
48how to use channels and includes a registry of well-known channel names.
49
50If the PV console has a name (i.e. it is a "channel") then the name
51is written to the frontend directory:
52
53name = <name>
54
55If the PV console has no name (i.e. it is a regular console) then the "name"
56key is omitted.
57
58The toolstack writes 'connection' information in the xenstore backend in
59the keys
60* connection: either 'pty' or 'socket'
61* path: only present if connection = 'socket', the path of the socket to
62  glue the channel to
63
64An artifact of the current implementation, the toolstack will write an
65extra backend key
66* output: an identifier only meaningful for qemu/xenconsoled
67
68If the toolstack wants the console to be connected to a pty, it will write
69to the backend:
70
71connection = pty
72output = pty
73
74The backend will write the pty device name to the "tty" node in the
75console frontend.
76
77For the PV console the tty node is added at
78
79/local/domain/$DOMID/console/tty
80
81For the virtual UART console the tty node is added at
82
83/local/domain/$DOMID/vuart/0/tty
84
85If the toolstack wants a listening Unix domain socket to be created at path
86<path>, a connection accepted and data proxied to the console, it will write:
87
88connection = socket
89path = <path>
90output = chardev:<some internal identifier>
91
92where chardev:<some internal identifier> matches a qemu character device
93configured on the qemu command-line.
94
95The backend implementation daemon chosen for a particular console is specified
96by the toolstack in the "type" node in the xenstore frontend.
97For example:
98
99# xenstore-read /local/domain/26/console/1/type
100ioemu
101
102The supported values are only xenconsoled or ioemu; xenconsoled has
103several limitations: it can only be used for the first PV or virtual UART
104console and it can only connect to a pty.
105
106Emulated serials are provided by qemu-dm only to hvm guests; the number
107of emulated serials depends on how many "-serial" command line options
108are given to qemu. The output of a serial is specified as argument to
109the -serial command line option to qemu. Qemu writes the tty name to
110xenstore in the following path:
111
112/local/domain/$DOMID/serial/$SERIAL_NUM/tty
113
114xenconsole is the tool to connect to a PV or virtual UART console or an
115emulated serial that has a pty as output. Xenconsole takes a domid as
116parameter plus an optional console type (pv for PV consoles, vuart for
117virtual UART or serial for emulated serials) and console number.
118Depending on the type and console number, xenconsole will look for the tty
119node in different xenstore paths, as described above.  If the user doesn't
120specify the console type xenconsole will try to guess: if the guest is a pv
121guest it defaults to PV console, if the guest is an hvm guest it defaults to
122emulated serial.
123
124By default xl creates a pv console for hvm guests, plus an emulated
125serial if the user specified 'serial = "pty"' in the VM config file.
126Considering that xenconsole defaults to emulated serials for hvm guests,
127executing xl create -c "domain" causes xenconsole to attach to the
128emulated serial tty. This is most probably what the user wanted because
129currently no bootloaders support xen pv consoles so the only way to
130interact with a bootloader like grub over a console is to use the
131emulated serial.
132However the pv console is still easy to use with Linux PV on HVM guests:
133the user just need to pass "console=hvc0" to the kernel command line and
134then execute "xl console -t pv <domain>" to connect to it.
135
136When using stubdoms the serial cards are still emulated by qemu (this
137time running in the stubdom), the number of serial cards and where the
138output goes is still specified using qemu command line options.
139The difference is that for each emulated serial card there must be a pv
140console connection between the stubdom and dom0 to export the serial
141output from the stubdom to dom0. The pv console backend for stubdom's pv
142consoles is always ioemu because multiple pv consoles support is a
143requirement in this case, considering that minios has its own pv console
144too. In order to simplify the setup when using stubdoms the hvm guest
145can only have one pv console with xenstored as backend (the stubdom
146could provide pv console backends to the hvm guest but then it would
147need another pv console connection for each console backend to export
148the pty to dom0).
149
150The xenconsole program supports a very simple protocol to notify parent about
151its readiness. If xenconsole (the client) is exec'ed and has been given a fd
152(normally the write end of a pipe) via --start-notify-fd option, it will
153write 0x00 to that fd after connecting to the guest but before entering the
154event loop. Parent can read from the read end of the fd to know the status.
155