1.. _cmake-details:
2
3Build System (CMake)
4********************
5
6
7CMake is used to build your application together with the Zephyr kernel. A
8CMake build is done in two stages. The first stage is called
9**configuration**. During configuration, the CMakeLists.txt build scripts are
10executed. After configuration is finished, CMake has an internal model of the
11Zephyr build, and can generate build scripts that are native to the host
12platform.
13
14CMake supports generating scripts for several build systems, but only Ninja and
15Make are tested and supported by Zephyr. After configuration, you begin the
16**build** stage by executing the generated build scripts. These build scripts
17can recompile the application without involving CMake following
18most code changes. However, after certain changes, the configuration step must
19be executed again before building. The build scripts can detect some of these
20situations and reconfigure automatically, but there are cases when this must be
21done manually.
22
23Zephyr uses CMake's concept of a 'target' to organize the build. A
24target can be an executable, a library, or a generated file. For
25application developers, the library target is the most important to
26understand. All source code that goes into a Zephyr build does so by
27being included in a library target, even application code.
28
29Library targets have source code, that is added through CMakeLists.txt
30build scripts like this:
31
32.. code-block:: cmake
33
34   target_sources(app PRIVATE src/main.c)
35
36In the above :file:`CMakeLists.txt`, an existing library target named ``app``
37is configured to include the source file :file:`src/main.c`. The ``PRIVATE``
38keyword indicates that we are modifying the internals of how the library is
39being built. Using the keyword ``PUBLIC`` would modify how other
40libraries that link with app are built. In this case, using ``PUBLIC``
41would cause libraries that link with ``app`` to also include the
42source file :file:`src/main.c`, behavior that we surely do not want. The
43``PUBLIC`` keyword could however be useful when modifying the include
44paths of a target library.
45
46When introducing build system code using CMake or adding new CMake files,
47please follow the style guidelines outlined :ref:`here <cmake-style>`.
48
49
50Build and Configuration Phases
51==============================
52
53The Zephyr build process can be divided into two main phases: a configuration
54phase (driven by CMake) and a build phase (driven by Make or Ninja).
55
56.. _build_configuration_phase:
57
58Configuration Phase
59-------------------
60
61The configuration phase begins when the user invokes *CMake* to generate a
62build system, specifying a source application directory and a board target.
63
64.. figure:: build-config-phase.svg
65    :align: center
66    :alt: Zephyr's build configuration phase
67    :figclass: align-center
68    :width: 80%
69
70CMake begins by processing the :file:`CMakeLists.txt` file in the application
71directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr
72top-level directory, which in turn refers to :file:`CMakeLists.txt` files
73throughout the build tree (directly and indirectly). Its primary output is a
74set of Makefiles or Ninja files to drive the build process, but the CMake
75scripts also do some processing of their own, which is explained here.
76
77Note that paths beginning with :file:`build/` below refer to the build
78directory you create when running CMake.
79
80Devicetree
81   :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source
82   include*) files are collected from the target's architecture, SoC, board,
83   and application directories.
84
85   :file:`*.dtsi` files are included by :file:`*.dts` files via the C
86   preprocessor (often abbreviated *cpp*, which should not be confused with
87   C++). The C preprocessor is also used to merge in any devicetree
88   :file:`*.overlay` files, and to expand macros in :file:`*.dts`,
89   :file:`*.dtsi`, and :file:`*.overlay` files. The preprocessor output is
90   placed in :file:`build/zephyr/zephyr.dts.pre`.
91
92   The preprocessed devicetree sources are parsed by
93   :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to generate a
94   :file:`build/zephyr/include/generated/zephyr/devicetree_generated.h` header with
95   preprocessor macros.
96
97   Source code should access preprocessor macros generated from devicetree by
98   including the :zephyr_file:`devicetree.h <include/zephyr/devicetree.h>` header,
99   which includes :file:`devicetree_generated.h`.
100
101   :file:`gen_defines.py` also writes the final devicetree to
102   :file:`build/zephyr/zephyr.dts` in the build directory. This file's contents
103   may be useful for debugging.
104
105   If the devicetree compiler ``dtc`` is installed, it is run on
106   :file:`build/zephyr/zephyr.dts` to catch any extra warnings and errors
107   generated by this tool. The output from ``dtc`` is unused otherwise, and
108   this step is skipped if ``dtc`` is not installed.
109
110   The above is just a brief overview. For more information on devicetree, see
111   :ref:`dt-guide`.
112
113Kconfig
114   :file:`Kconfig` files define available configuration options for the
115   target architecture, SoC, board, and application, as well as dependencies
116   between options.
117
118   Kconfig configurations are stored in *configuration files*. The initial
119   configuration is generated by merging configuration fragments from the board
120   and application (e.g. :file:`prj.conf`).
121
122   The output from Kconfig is an :file:`autoconf.h` header with preprocessor
123   assignments, and a :file:`.config` file that acts both as a saved
124   configuration and as configuration output (used by CMake). The definitions in
125   :file:`autoconf.h` are automatically exposed at compile time, so there is no
126   need to include this header.
127
128   Information from devicetree is available to Kconfig, through the functions
129   defined in :zephyr_file:`kconfigfunctions.py
130   <scripts/kconfig/kconfigfunctions.py>`.
131
132   See :ref:`the Kconfig section of the manual <kconfig>` for more information.
133
134Build Phase
135-----------
136
137The build phase begins when the user invokes ``make`` or ``ninja``. Its
138ultimate output is a complete Zephyr application in a format suitable for
139loading/flashing on the desired target board (:file:`zephyr.elf`,
140:file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually,
141into four stages: the pre-build, first-pass binary, final binary, and
142post-processing.
143
144Pre-build
145+++++++++
146
147Pre-build occurs before any source files are compiled, because during
148this phase header files used by the source files are generated.
149
150Offset generation
151   Access to high-level data structures and members is sometimes
152   required when the definitions of those structures is not
153   immediately accessible (e.g., assembly language). The generation of
154   *offsets.h* (by *gen_offset_header.py*) facilitates this.
155
156System call boilerplate
157   The *gen_syscall.py* and *parse_syscalls.py*  scripts work
158   together to bind potential system call functions with their
159   implementations.
160
161.. figure:: build-build-phase-1.svg
162    :align: center
163    :alt: Zephyr's build stage I
164    :figclass: align-center
165    :width: 80%
166
167Intermediate binaries
168+++++++++++++++++++++
169
170Compilation proper begins with the first intermediate binary. Source files (C
171and assembly) are collected from various subsystems (which ones is
172decided during the configuration phase), and compiled into archives
173(with reference to header files in the tree, as well as those
174generated during the configuration phase and the pre-build stage(s)).
175
176.. figure:: build-build-phase-2.svg
177    :align: center
178    :alt: Zephyr's build stage II
179    :figclass: align-center
180    :width: 80%
181
182The exact number of intermediate binaries is decided during the configuration
183phase.
184
185If memory protection is enabled, then:
186
187Partition grouping
188   The *gen_app_partitions.py* script scans all the
189   generated archives and outputs linker scripts to ensure that
190   application partitions are properly grouped and aligned for the
191   target’s memory protection hardware.
192
193Then *cpp* is used to combine linker script fragments from the target’s
194architecture/SoC, the kernel tree, optionally the partition output if
195memory protection is enabled, and any other fragments selected during
196the configuration process, into a *linker.cmd* file. The compiled
197archives are then linked with *ld* as specified in the
198*linker.cmd*.
199
200Unfixed size binary
201   The unfixed size intermediate binary is produced when :ref:`usermode_api`
202   is enabled or :ref:`devicetree` is in use.
203   It produces a binary where sizes are not fixed and thus it may be used
204   by post-process steps that will impact the size of the final binary.
205
206.. figure:: build-build-phase-3.svg
207    :align: center
208    :alt: Zephyr's build stage III
209    :figclass: align-center
210    :width: 80%
211
212Fixed size binary
213   The fixed size intermediate binary is produced when :ref:`usermode_api`
214   is enabled or when generated IRQ tables are used,
215   :kconfig:option:`CONFIG_GEN_ISR_TABLES`
216   It produces a binary where sizes are fixed and thus the size must not change
217   between the intermediate binary and the final binary.
218
219.. figure:: build-build-phase-4.svg
220    :align: center
221    :alt: Zephyr's build stage IV
222    :figclass: align-center
223    :width: 80%
224
225Intermediate binaries post-processing
226+++++++++++++++++++++++++++++++++++++
227
228The binaries from the previous stage are incomplete, with empty and/or
229placeholder sections that must be filled in by, essentially, reflection.
230
231To complete the build procedure the following scripts are executed on the
232intermediate binaries to produce the missing pieces needed for the final
233binary.
234
235When :ref:`usermode_api` is enabled:
236
237Partition alignment
238   The *gen_app_partitions.py* script scans the unfixed size binary and
239   generates an app shared memory aligned linker script snippet where the
240   partitions are sorted in descending order.
241
242.. figure:: build-postprocess-1.svg
243    :align: center
244    :alt: Zephyr's intermediate binary post-process I
245    :figclass: align-center
246    :width: 80%
247
248When :ref:`devicetree` is used:
249
250Device dependencies
251   The *gen_device_deps.py* script scans the unfixed size binary to determine
252   relationships between devices that were recorded from devicetree data,
253   and replaces the encoded relationships with values that are optimized to
254   locate the devices actually present in the application.
255
256.. figure:: build-postprocess-2.svg
257    :align: center
258    :alt: Zephyr's intermediate binary post-process II
259    :figclass: align-center
260    :width: 80%
261
262When :kconfig:option:`CONFIG_GEN_ISR_TABLES` is enabled:
263   The *gen_isr_tables.py* script scans the fixed size binary and creates
264   an isr_tables.c source file with a hardware vector table and/or software
265   IRQ table.
266
267.. figure:: build-postprocess-3.svg
268    :align: center
269    :alt: Zephyr's intermediate binary post-process III
270    :figclass: align-center
271    :width: 80%
272
273When :ref:`usermode_api` is enabled:
274
275Kernel object hashing
276   The *gen_kobject_list.py* scans the *ELF DWARF*
277   debug data to find the address of the all kernel objects. This
278   list is passed to *gperf*, which generates a perfect hash function and
279   table of those addresses, then that output is optimized by
280   *process_gperf.py*, using known properties of our special case.
281
282.. figure:: build-postprocess-4.svg
283    :align: center
284    :alt: Zephyr's intermediate binary post-process IV
285    :figclass: align-center
286    :width: 80%
287
288When no intermediate binary post-processing is required then the first
289intermediate binary will be directly used as the final binary.
290
291Final binary
292++++++++++++
293
294The binary from the previous stage is incomplete, with empty and/or
295placeholder sections that must be filled in by, essentially, reflection.
296
297The link from the previous stage is repeated, this time with the missing
298pieces populated.
299
300.. figure:: build-build-phase-5.svg
301    :align: center
302    :alt: Zephyr's build final stage
303    :figclass: align-center
304    :width: 80%
305
306Post processing
307+++++++++++++++
308
309Finally, if necessary, the completed kernel is converted from *ELF* to
310the format expected by the loader and/or flash tool required by the
311target. This is accomplished in a straightforward manner with *objdump*.
312
313.. figure:: build-build-phase-6.svg
314    :align: center
315    :alt: Zephyr's build final stage post-process
316    :figclass: align-center
317    :width: 80%
318
319
320.. _build_system_scripts:
321
322Supporting Scripts and Tools
323============================
324
325The following is a detailed description of the scripts used during the build process.
326
327.. _gen_syscalls.py:
328
329:zephyr_file:`scripts/build/gen_syscalls.py`
330--------------------------------------------
331
332.. include:: ../../../scripts/build/gen_syscalls.py
333   :start-after: """
334   :end-before: """
335
336.. _gen_device_deps.py:
337
338:zephyr_file:`scripts/build/gen_device_deps.py`
339-----------------------------------------------
340
341.. include:: ../../../scripts/build/gen_device_deps.py
342   :start-after: """
343   :end-before: """
344
345.. _gen_kobject_list.py:
346
347:zephyr_file:`scripts/build/gen_kobject_list.py`
348------------------------------------------------
349
350.. include:: ../../../scripts/build/gen_kobject_list.py
351   :start-after: """
352   :end-before: """
353
354.. _gen_offset_header.py:
355
356:zephyr_file:`scripts/build/gen_offset_header.py`
357-------------------------------------------------
358
359.. include:: ../../../scripts/build/gen_offset_header.py
360   :start-after: """
361   :end-before: """
362
363.. _parse_syscalls.py:
364
365:zephyr_file:`scripts/build/parse_syscalls.py`
366----------------------------------------------
367
368
369.. include:: ../../../scripts/build/parse_syscalls.py
370   :start-after: """
371   :end-before: """
372
373.. _gen_idt.py:
374
375:zephyr_file:`arch/x86/gen_idt.py`
376----------------------------------
377
378.. include:: ../../../arch/x86/gen_idt.py
379   :start-after: """
380   :end-before: """
381
382.. _gen_gdt.py:
383
384:zephyr_file:`arch/x86/gen_gdt.py`
385----------------------------------
386
387.. include:: ../../../arch/x86/gen_gdt.py
388   :start-after: """
389   :end-before: """
390
391.. _gen_relocate_app.py:
392
393:zephyr_file:`scripts/build/gen_relocate_app.py`
394------------------------------------------------
395
396.. include:: ../../../scripts/build/gen_relocate_app.py
397   :start-after: """
398   :end-before: """
399
400.. _process_gperf.py:
401
402:zephyr_file:`scripts/build/process_gperf.py`
403---------------------------------------------
404
405.. include:: ../../../scripts/build/process_gperf.py
406   :start-after: """
407   :end-before: """
408
409:zephyr_file:`scripts/build/gen_app_partitions.py`
410--------------------------------------------------
411
412.. include:: ../../../scripts/build/gen_app_partitions.py
413   :start-after: """
414   :end-before: """
415
416.. _check_init_priorities.py:
417
418:zephyr_file:`scripts/build/check_init_priorities.py`
419-----------------------------------------------------
420
421.. include:: ../../../scripts/build/check_init_priorities.py
422   :start-after: """
423   :end-before: """
424