1.. SPDX-License-Identifier: CC-BY-4.0
2
3Continuous Integration
4======================
5
6Xen Project uses Gitlab-CI for automated testing. Test pipelines for official
7staging branches are at
8`<https://gitlab.com/xen-project/hardware/xen/-/pipelines>`_. Developers can
9schedule test pipelines in their repositories under
10`<https://gitlab.com/xen-project/people/>`_.
11
12Hardware runners
13****************
14
15Some of the tests are using dedicated hardware runners. Those are not available freely, but the access is granted to individual developers. To get access to them, ask on the ``#XenDevel:matrix.org`` Matrix channel.
16After getting access to relevant runners, few extra changes are necessary in settings of the relevant "xen" gitlab project (under your `<https://gitlab.com/xen-project/people/>`_ namespace):
17
181. Go to Settings -> CI/CD, expand the "Runners" section and enable relevant runners for your project.
192. Expand "Variables" section and add ``QUBES_JOBS=true`` variable for Qubes runners.
203. Go to Settings -> Repository, expand "Branch rules" section and add a rule for protected branches - only those branches will get tests on the hardware runners. It's okay to use a pattern for branch name, and it's okay to allow force push.
21
22How to Set Up a New Hardware Runner
23***********************************
24
25A hardware runner is a GitLab runner designed to execute Xen tests on real hardware, such as an embedded board or a laptop. The GitLab runner runs on a controller PC, which is connected to the target board used for Xen testing. The controller PC can be any system, from a workstation to a Raspberry Pi.
26
27Steps to Set Up the Controller PC:
28
291. Install the Docker service
30   - Follow the instructions at: [Docker Installation Guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
31
322. Install the GitLab Runner service
33   - Follow the instructions at: [GitLab Runner Installation Guide](https://docs.gitlab.com/runner/install/linux-repository/#install-gitlab-runner)
34
353. Register the runner with GitLab
36   - Generate a runner token with a specific tag to identify it.
37   - Run `gitlab-runner register` and enter the token. Accepting the default settings is sufficient.
38
39These steps are common for setting up any GitLab runner. However, for hardware runners, additional configurations are required to enable the controller PC to:
40
41- Power on and off the target board.
42- Connect to the serial interface of the target board.
43- Establish an Ethernet connection with the target board.
44- Run a TFTP server with a TFTP root directory accessible by GitLab container tests.
45  - This allows the test script to place binaries, such as Xen, in the TFTP root before powering on the target board.
46
47To enable the required capabilities, edit the GitLab Runner configuration file located at `/etc/gitlab-runner/config.toml`:
48
49- Map the TFTP boot directory.
50- Map the serial device of the target board.
51
52Example Configuration:
53
54    volumes = ["/scratch/gitlab-runner:/scratch/gitlab-runner"]
55    devices = ["/dev/ttyUSB0:/dev/ttyUSB0"]
56
57After making these changes, restart the GitLab Runner service:
58
59    gitlab-runner restart
60
61This completes the setup of the hardware runner. The system is now ready for executing Xen tests on real hardware. As an example, to execute tests on an AMD x86 embedded board, we currently use the following script:
62
63    automation/scripts/xilinx-smoke-dom0-x86_64.sh
64
65Other examples are available under automation/scripts, such as
66automation/scripts/qubes-x86-64.sh, and at external locations:
67`<https://www.qubes-os.org/news/2022/05/05/automated-os-testing-on-physical-laptops>`_,
68`<https://github.com/QubesOS/tests-hw-setup/blob/28aa8b86208a54fc2ac986f06c66c92230bf771e/states/gitlab-runner-conf.toml>`_.
69
70
71Selecting individual tests
72**************************
73
74Normally, all build and test jobs are scheduled in a pipeline. When working on a specific patches, it is sometimes useful to run only jobs relevant for the current work - both to save time and to save CI resources. This can be done by seeting ``SELECTED_JOBS_ONLY`` variable when starting the pipeline. The variable holds a regular expression, enclosed with ``/`` that matches jobs to be included. The variable can be set via the gitlab.com web UI or directly when pushing changes to gitlab::
75
76   git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/"
77
78Note if a test job requires some build job, both need to be included in the regex. For example, ``adl-smoke-x86-64-gcc-debug`` requires ``alpine-3.18-gcc-debug``, so to run just this test the command will look like this::
79
80   git push -o ci.variable=SELECTED_JOBS_ONLY="/adl-smoke-x86-64-gcc-debug|alpine-3.18-gcc-debug/"
81
82More details at `<https://docs.gitlab.co.jp/ee/user/project/push_options.html>`_.
83
84Alternatively, irrelevant jobs can be removed from respective yaml files in ``automation/gitlab-ci`` by adding temporary commit on top of the branch.
85