1.. SPDX-License-Identifier: GPL-2.0+ 2 3Buildman build tool 4=================== 5 6Quick-start 7----------- 8 9If you just want to quickly set up buildman so you can build something (for 10example Raspberry Pi 2): 11 12.. code-block:: bash 13 14 cd /path/to/u-boot 15 PATH=$PATH:`pwd`/tools/buildman 16 buildman --fetch-arch arm 17 buildman -k rpi_2 18 ls ../current/rpi_2 19 # u-boot.bin is the output image 20 21 22What is this? 23------------- 24 25This tool handles building U-Boot to check that you have not broken it 26with your patch series. It can build each individual commit and report 27which boards fail on which commits, and which errors come up. It aims 28to make full use of multi-processor machines. 29 30A key feature of buildman is its output summary, which allows warnings, 31errors or image size increases in a particular commit or board to be 32quickly identified and the offending commit pinpointed. This can be a big 33help for anyone working with >10 patches at a time. 34 35 36Caveats 37------- 38 39Buildman can be stopped and restarted, in which case it will continue 40where it left off. This should happen cleanly and without side-effects. 41If not, it is a bug, for which a patch would be welcome. 42 43Buildman gets so tied up in its work that it can ignore the outside world. 44You may need to press Ctrl-C several times to quit it. Also it will print 45out various exceptions when stopped. You may have to kill it since the 46Ctrl-C handling is somewhat broken. 47 48 49Theory of Operation 50------------------- 51 52(please read this section in full twice or you will be perpetually confused) 53 54Buildman is a builder. It is not make, although it runs make. It does not 55produce any useful output on the terminal while building, except for 56progress information (but see -v below). All the output (errors, warnings and 57binaries if you ask for them) is stored in output directories, which you can 58look at from a separate 'buildman -s' instance while the build is progressing, 59or when it is finished. 60 61Buildman is designed to build entire git branches, i.e. muliple commits. It 62can be run repeatedly on the same branch after making changes to commits on 63that branch. In this case it will automatically rebuild commits which have 64changed (and remove its old results for that commit). It is possible to build 65a branch for one board, then later build it for another board. This adds to 66the output, so now you have results for two boards. If you want buildman to 67re-build a commit it has already built (e.g. because of a toolchain update), 68use the -f flag. 69 70Buildman produces a concise summary of which boards succeeded and failed. 71It shows which commit introduced which board failure using a simple 72red/green colour coding (with yellow/cyan for warnings). Full error 73information can be requested, in which case it is de-duped and displayed 74against the commit that introduced the error. An example workflow is below. 75 76Buildman stores image size information and can report changes in image size 77from commit to commit. An example of this is below. 78 79Buildman starts multiple threads, and each thread builds for one board at 80a time. A thread starts at the first commit, configures the source for your 81board and builds it. Then it checks out the next commit and does an 82incremental build (i.e. not using 'make xxx_defconfig' unless you use -C). 83Eventually the thread reaches the last commit and stops. If a commit causes 84an error or warning, buildman will try it again after reconfiguring (but see 85-Q). Thus some commits may be built twice, with the first result silently 86discarded. Lots of errors and warnings will causes lots of reconfigures and your 87build will be very slow. This is because a file that produces just a warning 88would not normally be rebuilt in an incremental build. Once a thread finishes 89building all the commits for a board, it starts on the commits for another 90board. 91 92Buildman works in an entirely separate place from your U-Boot repository. 93It creates a separate working directory for each thread, and puts the 94output files in the working directory, organised by commit name and board 95name, in a two-level hierarchy (but see -P). 96 97Buildman is invoked in your U-Boot directory, the one with the .git 98directory. It clones this repository into a copy for each thread, and the 99threads do not affect the state of your git repository. Any checkouts done 100by the thread affect only the working directory for that thread. 101 102Buildman automatically selects the correct tool chain for each board. You 103must supply suitable tool chains (see --fetch-arch), but buildman takes care 104of selecting the right one. 105 106Buildman generally builds a branch (with the -b flag), and in this case 107builds the upstream commit as well, for comparison. So even if you have one 108commit in your branch, two commits will be built. Put all your commits in a 109branch, set the branch's upstream to a valid value, and all will be well. 110Otherwise buildman will perform random actions. Use -n to check what the 111random actions might be. 112 113Buildman effectively has two modes: without -s it builds, with -s it 114summarises the results of previous (or active) builds. 115 116If you just want to build the current source tree, leave off the -b flag. 117This will display results and errors as they happen. You can still look at 118them later using -se. Note that buildman will assume that the source has 119changed, and will build all specified boards in this case. 120 121Buildman is optimised for building many commits at once, for many boards. 122On multi-core machines, Buildman is fast because it uses most of the 123available CPU power. When it gets to the end, or if you are building just 124a few commits or boards, it will be pretty slow. As a tip, if you don't 125plan to use your machine for anything else, you can use -T to increase the 126number of threads beyond the default. 127 128 129Selecting which boards to build 130------------------------------- 131 132Buildman lets you build all boards, or a subset. Specify the subset by passing 133command-line arguments that list the desired build target, architecture, 134CPU, board name, vendor, SoC or options. Multiple arguments are allowed. Each 135argument will be interpreted as a regular expression, so behaviour is a superset 136of exact or substring matching. Examples are: 137 138- 'tegra20' - all boards with a Tegra20 SoC 139- 'tegra' - all boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...) 140- '^tegra[23]0$' - all boards with either Tegra20 or Tegra30 SoC 141- 'powerpc' - all PowerPC boards 142 143While the default is to OR the terms together, you can also make use of 144the '&' operator to limit the selection: 145 146- 'freescale & arm sandbox' - all Freescale boards with ARM architecture, plus 147 sandbox 148 149You can also use -x to specifically exclude some boards. For example: 150 151 buildman arm -x nvidia,freescale,.*ball$ 152 153means to build all arm boards except nvidia, freescale and anything ending 154with 'ball'. 155 156For building specific boards you can use the --boards (or --bo) option, which 157takes a comma-separated list of board target names and be used multiple times 158on the command line: 159 160.. code-block:: bash 161 162 buildman --boards sandbox,snow --boards firefly-rk3399 163 164It is convenient to use the -n option to see what will be built based on 165the subset given. Use -v as well to get an actual list of boards. 166 167Buildman does not store intermediate object files. It optionally copies 168the binary output into a directory when a build is successful (-k). Size 169information is always recorded. It needs a fair bit of disk space to work, 170typically 250MB per thread. 171 172 173Setting up 174---------- 175 176#. Get the U-Boot source. You probably already have it, but if not these 177 steps should get you started with a repo and some commits for testing. 178 179 .. code-block:: bash 180 181 cd /path/to/u-boot 182 git clone git://git.denx.de/u-boot.git . 183 git checkout -b my-branch origin/master 184 # Add some commits to the branch, reading for testing 185 186#. Create ~/.buildman to tell buildman where to find tool chains (see 187 buildman_settings_ for details). As an example:: 188 189 # Buildman settings file 190 191 [toolchain] 192 root: / 193 rest: /toolchains/* 194 eldk: /opt/eldk-4.2 195 arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux 196 aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux 197 198 [toolchain-prefix] 199 arc = /opt/arc/arc_gnu_2021.03_prebuilt_elf32_le_linux_install/bin/arc-elf32- 200 201 [toolchain-alias] 202 riscv = riscv32 203 sh = sh4 204 x86: i386 205 206 This selects the available toolchain paths. Add the base directory for 207 each of your toolchains here. Buildman will search inside these directories 208 and also in any '/usr' and '/usr/bin' subdirectories. 209 210 Make sure the tags (here root: rest: and eldk:) are unique. 211 212 The toolchain-alias section indicates that the i386 toolchain should be used 213 to build x86 commits. 214 215 Note that you can also specific exactly toolchain prefixes if you like:: 216 217 [toolchain-prefix] 218 arm: /opt/arm-eabi-4.6/bin/arm-eabi- 219 220 or even:: 221 222 [toolchain-prefix] 223 arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc 224 225 This tells buildman that you want to use this exact toolchain for the arm 226 architecture. This will override any toolchains found by searching using the 227 [toolchain] settings. 228 229 Since the toolchain prefix is an explicit request, buildman will report an 230 error if a toolchain is not found with that prefix. The current PATH will be 231 searched, so it is possible to use:: 232 233 [toolchain-prefix] 234 arm: arm-none-eabi- 235 236 and buildman will find arm-none-eabi-gcc in /usr/bin if you have it 237 installed. 238 239 Another example:: 240 241 [toolchain-wrapper] 242 wrapper: ccache 243 244 This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In 245 this example, ccache. It doesn't affect the toolchain scan. The wrapper is 246 added when CROSS_COMPILE environtal variable is set. The name in this 247 section is ignored. If more than one line is provided, only the last one 248 is taken. 249 250#. Make sure you have the required Python pre-requisites 251 252 Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and 253 urllib2. These should normally be available, but if you get an error like 254 this then you will need to obtain those modules:: 255 256 ImportError: No module named multiprocessing 257 258 259#. Check the available toolchains 260 261 Run this check to make sure that you have a toolchain for every architecture:: 262 263 $ ./tools/buildman/buildman --list-tool-chains 264 Scanning for tool chains 265 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-' 266 Tool chain test: OK, arch='x86', priority 1 267 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-' 268 Tool chain test: OK, arch='arm', priority 1 269 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux' 270 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.' 271 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin' 272 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc' 273 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin' 274 Tool chain test: OK, arch='i386', priority 4 275 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux' 276 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.' 277 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin' 278 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc' 279 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin' 280 Tool chain test: OK, arch='aarch64', priority 4 281 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux' 282 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.' 283 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin' 284 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc' 285 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin' 286 Tool chain test: OK, arch='microblaze', priority 4 287 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux' 288 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.' 289 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin' 290 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc' 291 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin' 292 Tool chain test: OK, arch='mips64', priority 4 293 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux' 294 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.' 295 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin' 296 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc' 297 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin' 298 Tool chain test: OK, arch='sparc64', priority 4 299 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi' 300 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.' 301 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin' 302 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' 303 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin' 304 Tool chain test: OK, arch='arm', priority 3 305 Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1 306 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux' 307 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.' 308 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin' 309 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc' 310 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin' 311 Tool chain test: OK, arch='sparc', priority 4 312 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux' 313 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.' 314 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin' 315 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc' 316 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin' 317 Tool chain test: OK, arch='mips', priority 4 318 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux' 319 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.' 320 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin' 321 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc' 322 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' 323 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin' 324 Tool chain test: OK, arch='x86_64', priority 4 325 Tool chain test: OK, arch='x86_64', priority 4 326 Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4 327 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux' 328 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.' 329 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin' 330 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc' 331 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin' 332 Tool chain test: OK, arch='m68k', priority 4 333 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux' 334 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.' 335 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin' 336 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc' 337 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin' 338 Tool chain test: OK, arch='powerpc', priority 4 339 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux' 340 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.' 341 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin' 342 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc' 343 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin' 344 Tool chain test: OK, arch='bfin', priority 6 345 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux' 346 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.' 347 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin' 348 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' 349 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin' 350 Tool chain test: OK, arch='sparc', priority 4 351 Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4 352 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux' 353 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.' 354 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin' 355 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' 356 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin' 357 Tool chain test: OK, arch='mips', priority 4 358 Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4 359 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux' 360 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.' 361 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin' 362 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' 363 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin' 364 Tool chain test: OK, arch='m68k', priority 4 365 Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4 366 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux' 367 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.' 368 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin' 369 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc' 370 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin' 371 Tool chain test: OK, arch='powerpc', priority 4 372 Tool chain test: OK, arch='or32', priority 4 373 - scanning path '/' 374 - looking in '/.' 375 - looking in '/bin' 376 - looking in '/usr/bin' 377 - found '/usr/bin/i586-mingw32msvc-gcc' 378 - found '/usr/bin/c89-gcc' 379 - found '/usr/bin/x86_64-linux-gnu-gcc' 380 - found '/usr/bin/gcc' 381 - found '/usr/bin/c99-gcc' 382 - found '/usr/bin/arm-linux-gnueabi-gcc' 383 - found '/usr/bin/aarch64-linux-gnu-gcc' 384 - found '/usr/bin/winegcc' 385 - found '/usr/bin/arm-linux-gnueabihf-gcc' 386 Tool chain test: OK, arch='i586', priority 11 387 Tool chain test: OK, arch='c89', priority 11 388 Tool chain test: OK, arch='x86_64', priority 4 389 Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4 390 Tool chain test: OK, arch='sandbox', priority 11 391 Tool chain test: OK, arch='c99', priority 11 392 Tool chain test: OK, arch='arm', priority 4 393 Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1 394 Tool chain test: OK, arch='aarch64', priority 4 395 Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4 396 Tool chain test: OK, arch='sandbox', priority 11 397 Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11 398 Tool chain test: OK, arch='arm', priority 4 399 Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1 400 List of available toolchains (34): 401 aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc 402 alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc 403 am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc 404 arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc 405 bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc 406 c89 : /usr/bin/c89-gcc 407 c99 : /usr/bin/c99-gcc 408 frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc 409 h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc 410 hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc 411 hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc 412 i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc 413 i586 : /usr/bin/i586-mingw32msvc-gcc 414 ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc 415 m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc 416 m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc 417 microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc 418 mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc 419 mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc 420 or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc 421 powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc 422 powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc 423 ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc 424 s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc 425 sandbox : /usr/bin/gcc 426 sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc 427 sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc 428 sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc 429 tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc 430 x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc 431 x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc 432 433 434 You can see that everything is covered, even some strange ones that won't 435 be used (c88 and c99). This is a feature. 436 437 438#. Install new toolchains if needed 439 440 You can download toolchains and update the [toolchain] section of the 441 settings file to find them. 442 443 To make this easier, buildman can automatically download and install 444 toolchains from kernel.org. First list the available architectures:: 445 446 $ ./tools/buildman/buildman --fetch-arch list 447 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/ 448 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/ 449 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/ 450 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/ 451 Available architectures: alpha am33_2.0 arm bfin cris crisv32 frv h8300 452 hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4 453 sparc sparc64 tilegx x86_64 xtensa 454 455 Then pick one and download it:: 456 457 $ ./tools/buildman/buildman --fetch-arch or32 458 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/ 459 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/ 460 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/ 461 Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz 462 Unpacking to: /home/sjg/.buildman-toolchains 463 Testing 464 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.' 465 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin' 466 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc' 467 Tool chain test: OK 468 469 Or download them all from kernel.org and move them to /toolchains directory: 470 471 .. code-block:: bash 472 473 ./tools/buildman/buildman --fetch-arch all 474 sudo mkdir -p /toolchains 475 sudo mv ~/.buildman-toolchains/*/* /toolchains/ 476 477 Buildman should now be set up to use your new toolchain. 478 479 At the time of writing, U-Boot has these architectures: 480 481 arc, arm, m68k, microblaze, mips, nios2, powerpc, sandbox, sh, x86, xtensa 482 483 484How to run it 485------------- 486 487First do a dry run using the -n flag: (replace <branch> with a real, local 488branch with a valid upstream): 489 490.. code-block:: bash 491 492 ./tools/buildman/buildman -b <branch> -n 493 494If it can't detect the upstream branch, try checking out the branch, and 495doing something like 'git branch --set-upstream-to upstream/master' 496or something similar. Buildman will try to guess a suitable upstream branch 497if it can't find one (you will see a message like "Guessing upstream as ..."). 498You can also use the -c option to manually specify the number of commits to 499build. 500 501As an example:: 502 503 Dry run, so not doing much. But I would do this: 504 505 Building 18 commits for 1059 boards (4 threads, 1 job per thread) 506 Build directory: ../lcd9b 507 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm 508 c18f1b4 tegra: Use const for pinmux_config_pingroup/table() 509 2f043ae tegra: Add display support to funcmux 510 e349900 tegra: fdt: Add pwm binding and node 511 424a5f0 tegra: fdt: Add LCD definitions for Tegra 512 0636ccf tegra: Add support for PWM 513 a994fe7 tegra: Add SOC support for display/lcd 514 fcd7350 tegra: Add LCD driver 515 4d46e9d tegra: Add LCD support to Nvidia boards 516 991bd48 arm: Add control over cachability of memory regions 517 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment 518 d92aff7 lcd: Add support for flushing LCD fb from dcache after update 519 dbd0677 tegra: Align LCD frame buffer to section boundary 520 0cff9b8 tegra: Support control of cache settings for LCD 521 9c56900 tegra: fdt: Add LCD definitions for Seaboard 522 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 523 cac5a23 tegra: Enable display/lcd support on Seaboard 524 49ff541 wip 525 526 Total boards to build for each commit: 1059 527 528This shows that it will build all 1059 boards, using 4 threads (because 529we have a 4-core CPU). Each thread will run with -j1, meaning that each 530make job will use a single CPU. The list of commits to be built helps you 531confirm that things look about right. Notice that buildman has chosen a 532'base' directory for you, immediately above your source tree. 533 534Buildman works entirely inside the base directory, here ../lcd9b, 535creating a working directory for each thread, and creating output 536directories for each commit and board. 537 538 539Suggested Workflow 540------------------ 541 542To run the build for real, take off the -n: 543 544.. code-block:: bash 545 546 ./tools/buildman/buildman -b <branch> 547 548Buildman will set up some working directories, and get started. After a 549minute or so it will settle down to a steady pace, with a display like this:: 550 551 Building 18 commits for 1059 boards (4 threads, 1 job per thread) 552 528 36 124 /19062 -18374 1:13:30 : SIMPC8313_SP 553 554This means that it is building 19062 board/commit combinations. So far it 555has managed to successfully build 528. Another 36 have built with warnings, 556and 124 more didn't build at all. It has 18374 builds left to complete. 557Buildman expects to complete the process in around an hour and a quarter. 558Use this time to buy a faster computer. 559 560 561To find out how the build went, ask for a summary with -s. You can do this 562either before the build completes (presumably in another terminal) or 563afterwards. Let's work through an example of how this is used:: 564 565 $ ./tools/buildman/buildman -b lcd9b -s 566 ... 567 01: Merge branch 'master' of git://git.denx.de/u-boot-arm 568 powerpc: + galaxy5200_LOWBOOT 569 02: tegra: Use const for pinmux_config_pingroup/table() 570 03: tegra: Add display support to funcmux 571 04: tegra: fdt: Add pwm binding and node 572 05: tegra: fdt: Add LCD definitions for Tegra 573 06: tegra: Add support for PWM 574 07: tegra: Add SOC support for display/lcd 575 08: tegra: Add LCD driver 576 09: tegra: Add LCD support to Nvidia boards 577 10: arm: Add control over cachability of memory regions 578 11: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment 579 12: lcd: Add support for flushing LCD fb from dcache after update 580 arm: + lubbock 581 13: tegra: Align LCD frame buffer to section boundary 582 14: tegra: Support control of cache settings for LCD 583 15: tegra: fdt: Add LCD definitions for Seaboard 584 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 585 17: tegra: Enable display/lcd support on Seaboard 586 18: wip 587 588This shows which commits have succeeded and which have failed. In this case 589the build is still in progress so many boards are not built yet (use -u to 590see which ones). But already we can see a few failures. The galaxy5200_LOWBOOT 591never builds correctly. This could be a problem with our toolchain, or it 592could be a bug in the upstream. The good news is that we probably don't need 593to blame our commits. The bad news is that our commits are not tested on that 594board. 595 596Commit 12 broke lubbock. That's what the '+ lubbock', in red, means. The 597failure is never fixed by a later commit, or you would see lubbock again, in 598green, without the +. 599 600To see the actual error:: 601 602 $ ./tools/buildman/buildman -b <branch> -se 603 ... 604 12: lcd: Add support for flushing LCD fb from dcache after update 605 arm: + lubbock 606 +common/libcommon.o: In function `lcd_sync': 607 +common/lcd.c:120: undefined reference to `flush_dcache_range' 608 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572 609 +make: *** [build/u-boot] Error 139 610 13: tegra: Align LCD frame buffer to section boundary 611 14: tegra: Support control of cache settings for LCD 612 15: tegra: fdt: Add LCD definitions for Seaboard 613 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 614 -common/lcd.c:120: undefined reference to `flush_dcache_range' 615 +common/lcd.c:125: undefined reference to `flush_dcache_range' 616 17: tegra: Enable display/lcd support on Seaboard 617 18: wip 618 619So the problem is in lcd.c, due to missing cache operations. This information 620should be enough to work out what that commit is doing to break these 621boards. (In this case pxa did not have cache operations defined). 622 623Note that if there were other boards with errors, the above command would 624show their errors also. Each line is shown only once. So if lubbock and snow 625produce the same error, we just see:: 626 627 12: lcd: Add support for flushing LCD fb from dcache after update 628 arm: + lubbock snow 629 +common/libcommon.o: In function `lcd_sync': 630 +common/lcd.c:120: undefined reference to `flush_dcache_range' 631 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572 632 +make: *** [build/u-boot] Error 139 633 634But if you did want to see just the errors for lubbock, use: 635 636.. code-block:: bash 637 638 ./tools/buildman/buildman -b <branch> -se lubbock 639 640If you see error lines marked with '-', that means that the errors were fixed 641by that commit. Sometimes commits can be in the wrong order, so that a 642breakage is introduced for a few commits and fixed by later commits. This 643shows up clearly with buildman. You can then reorder the commits and try 644again. 645 646At commit 16, the error moves: you can see that the old error at line 120 647is fixed, but there is a new one at line 126. This is probably only because 648we added some code and moved the broken line further down the file. 649 650As mentioned, if many boards have the same error, then -e will display the 651error only once. This makes the output as concise as possible. To see which 652boards have each error, use -l. So it is safe to omit the board name - you 653will not get lots of repeated output for every board. 654 655Buildman tries to distinguish warnings from errors, and shows warning lines 656separately with a 'w' prefix. Warnings introduced show as yellow. Warnings 657fixed show as cyan. 658 659The full build output in this case is available in:: 660 661 ../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/ 662 663Files: 664 665done 666 Indicates the build was done, and holds the return code from make. This is 0 667 for a good build, typically 2 for a failure. 668 669err 670 Output from stderr, if any. Errors and warnings appear here. 671 672log 673 Output from stdout. Normally there isn't any since buildman runs in silent 674 mode. Use -V to force a verbose build (this passes V=1 to 'make') 675 676toolchain 677 Shows information about the toolchain used for the build. 678 679sizes 680 Shows image size information. 681 682It is possible to get the build binary output there also. Use the -k option 683for this. In that case you will also see some output files, like: 684 685- System.map 686- toolchain 687- u-boot 688- u-boot.bin 689- u-boot.map 690- autoconf.mk 691- SPL/TPL versions like u-boot-spl and u-boot-spl.bin if available 692 693 694Checking Image Sizes 695-------------------- 696 697A key requirement for U-Boot is that you keep code/data size to a minimum. 698Where a new feature increases this noticeably it should normally be put 699behind a CONFIG flag so that boards can leave it disabled and keep the image 700size more or less the same with each new release. 701 702To check the impact of your commits on image size, use -S. For example:: 703 704 $ ./tools/buildman/buildman -b us-x86 -sS 705 Summary of 10 commits for 1066 boards (4 threads, 1 job per thread) 706 01: MAKEALL: add support for per architecture toolchains 707 02: x86: Add function to get top of usable ram 708 x86: (for 1/3 boards) text -272.0 rodata +41.0 709 03: x86: Add basic cache operations 710 04: x86: Permit bootstage and timer data to be used prior to relocation 711 x86: (for 1/3 boards) data +16.0 712 05: x86: Add an __end symbol to signal the end of the U-Boot binary 713 x86: (for 1/3 boards) text +76.0 714 06: x86: Rearrange the output input to remove BSS 715 x86: (for 1/3 boards) bss -2140.0 716 07: x86: Support relocation of FDT on start-up 717 x86: + coreboot-x86 718 08: x86: Add error checking to x86 relocation code 719 09: x86: Adjust link device tree include file 720 10: x86: Enable CONFIG_OF_CONTROL on coreboot 721 722 723You can see that image size only changed on x86, which is good because this 724series is not supposed to change any other board. From commit 7 onwards the 725build fails so we don't get code size numbers. The numbers are fractional 726because they are an average of all boards for that architecture. The 727intention is to allow you to quickly find image size problems introduced by 728your commits. 729 730Note that the 'text' region and 'rodata' are split out. You should add the 731two together to get the total read-only size (reported as the first column 732in the output from binutil's 'size' utility). 733 734A useful option is --step which lets you skip some commits. For example 735--step 2 will show the image sizes for only every 2nd commit (so it will 736compare the image sizes of the 1st, 3rd, 5th... commits). You can also use 737--step 0 which will compare only the first and last commits. This is useful 738for an overview of how your entire series affects code size. It will build 739only the upstream commit and your final branch commit. 740 741You can also use -d to see a detailed size breakdown for each board. This 742list is sorted in order from largest growth to largest reduction. 743 744It is even possible to go a little further with the -B option (--bloat). This 745shows where U-Boot has bloated, breaking the size change down to the function 746level. Example output is below:: 747 748 $ ./tools/buildman/buildman -b us-mem4 -sSdB 749 ... 750 19: Roll crc32 into hash infrastructure 751 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6 752 paz00 : all +23 bss -4 rodata -29 text +56 753 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64) 754 function old new delta 755 hash_command 80 160 +80 756 crc32_wd_buf - 56 +56 757 ext4fs_read_file 540 568 +28 758 insert_var_value_sub 688 692 +4 759 run_list_real 1996 1992 -4 760 do_mem_crc 168 68 -100 761 trimslice : all -9 bss +16 rodata -29 text +4 762 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 763 function old new delta 764 hash_command 80 160 +80 765 crc32_wd_buf - 56 +56 766 ext4fs_iterate_dir 672 668 -4 767 ext4fs_read_file 568 548 -20 768 do_mem_crc 168 68 -100 769 whistler : all -9 bss +16 rodata -29 text +4 770 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 771 function old new delta 772 hash_command 80 160 +80 773 crc32_wd_buf - 56 +56 774 ext4fs_iterate_dir 672 668 -4 775 ext4fs_read_file 568 548 -20 776 do_mem_crc 168 68 -100 777 seaboard : all -9 bss -28 rodata -29 text +48 778 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56) 779 function old new delta 780 hash_command 80 160 +80 781 crc32_wd_buf - 56 +56 782 ext4fs_read_file 548 568 +20 783 run_list_real 1996 2000 +4 784 do_nandboot 760 756 -4 785 do_mem_crc 168 68 -100 786 colibri_t20 : all -9 rodata -29 text +20 787 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28) 788 function old new delta 789 hash_command 80 160 +80 790 crc32_wd_buf - 56 +56 791 read_abs_bbt 204 208 +4 792 do_nandboot 760 756 -4 793 ext4fs_read_file 576 568 -8 794 do_mem_crc 168 68 -100 795 ventana : all -37 bss -12 rodata -29 text +4 796 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 797 function old new delta 798 hash_command 80 160 +80 799 crc32_wd_buf - 56 +56 800 ext4fs_iterate_dir 672 668 -4 801 ext4fs_read_file 568 548 -20 802 do_mem_crc 168 68 -100 803 harmony : all -37 bss -16 rodata -29 text +8 804 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16) 805 function old new delta 806 hash_command 80 160 +80 807 crc32_wd_buf - 56 +56 808 nand_write_oob_syndrome 428 432 +4 809 ext4fs_iterate_dir 672 668 -4 810 ext4fs_read_file 568 548 -20 811 do_mem_crc 168 68 -100 812 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336 813 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288) 814 function old new delta 815 crc32_wd_buf - 56 +56 816 do_fat_read_at 2872 2904 +32 817 hash_algo 16 - -16 818 do_mem_crc 168 68 -100 819 hash_command 420 160 -260 820 tec : all -449 bss -4 data -16 rodata -93 text -336 821 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288) 822 function old new delta 823 crc32_wd_buf - 56 +56 824 do_fat_read_at 2872 2904 +32 825 hash_algo 16 - -16 826 do_mem_crc 168 68 -100 827 hash_command 420 160 -260 828 plutux : all -481 bss +16 data -16 rodata -93 text -388 829 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340) 830 function old new delta 831 crc32_wd_buf - 56 +56 832 do_load_serial_bin 1688 1700 +12 833 hash_algo 16 - -16 834 do_fat_read_at 2904 2872 -32 835 do_mem_crc 168 68 -100 836 hash_command 420 160 -260 837 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4 838 MPC8610HPCD : all +55 rodata -29 text +84 839 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 840 function old new delta 841 hash_command - 176 +176 842 do_mem_crc 184 88 -96 843 MPC8641HPCN : all +55 rodata -29 text +84 844 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 845 function old new delta 846 hash_command - 176 +176 847 do_mem_crc 184 88 -96 848 MPC8641HPCN_36BIT: all +55 rodata -29 text +84 849 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 850 function old new delta 851 hash_command - 176 +176 852 do_mem_crc 184 88 -96 853 sbc8641d : all +55 rodata -29 text +84 854 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 855 function old new delta 856 hash_command - 176 +176 857 do_mem_crc 184 88 -96 858 xpedite517x : all -33 data -16 rodata -93 text +76 859 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64) 860 function old new delta 861 hash_command - 176 +176 862 hash_algo 16 - -16 863 do_mem_crc 184 88 -96 864 ... 865 866 867This shows that commit 19 has reduced codesize for arm slightly and increased 868it for powerpc. This increase was offset in by reductions in rodata and 869data/bss. 870 871Shown below the summary lines are the sizes for each board. Below each board 872are the sizes for each function. This information starts with: 873 874add 875 number of functions added / removed 876 877grow 878 number of functions which grew / shrunk 879 880bytes 881 number of bytes of code added to / removed from all functions, plus the total 882 byte change in brackets 883 884The change seems to be that hash_command() has increased by more than the 885do_mem_crc() function has decreased. The function sizes typically add up to 886roughly the text area size, but note that every read-only section except 887rodata is included in 'text', so the function total does not exactly 888correspond. 889 890It is common when refactoring code for the rodata to decrease as the text size 891increases, and vice versa. 892 893 894.. _buildman_settings: 895 896The .buildman settings file 897--------------------------- 898 899The .buildman file provides information about the available toolchains and 900also allows build flags to be passed to 'make'. It consists of several 901sections, with the section name in square brackets. Within each section are 902a set of (tag, value) pairs. 903 904'[global]' section 905 allow-missing 906 Indicates the policy to use for missing blobs. Note that the flags 907 ``--allow-missing`` (``-M``) and ``--no-allow-missing`` (``--no-a``) 908 override these setting. 909 910 always 911 Run with ``-M`` by default. 912 913 multiple 914 Run with ``-M`` if more than one board is being built. 915 916 branch 917 Run with ``-M`` if a branch is being built. 918 919 Note that the last two can be given together:: 920 921 allow-missing = multiple branch 922 923'[toolchain]' section 924 This lists the available toolchains. The tag here doesn't matter, but 925 make sure it is unique. The value is the path to the toolchain. Buildman 926 will look in that path for a file ending in 'gcc'. It will then execute 927 it to check that it is a C compiler, passing only the --version flag to 928 it. If the return code is 0, buildman assumes that it is a valid C 929 compiler. It uses the first part of the name as the architecture and 930 strips off the last part when setting the CROSS_COMPILE environment 931 variable (parts are delimited with a hyphen). 932 933 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc' 934 and CROSS_COMPILE will be set to powerpc-linux- when using it. 935 936 The tilde character ``~`` is supported in paths, to represent the home 937 directory. 938 939'[toolchain-prefix]' section 940 This can be used to provide the full toolchain-prefix for one or more 941 architectures. The full CROSS_COMPILE prefix must be provided. These 942 typically have a higher priority than matches in the '[toolchain]', due to 943 this prefix. 944 945 The tilde character ``~`` is supported in paths, to represent the home 946 directory. 947 948'[toolchain-alias]' section 949 This converts toolchain architecture names to U-Boot names. For example, 950 if an x86 toolchains is called i386-linux-gcc it will not normally be 951 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section 952 will tell buildman that the i386 and x86_64 toolchains can be used for 953 the x86 architecture. 954 955'[make-flags]' section 956 U-Boot's build system supports a few flags (such as BUILD_TAG) which 957 affect the build product. These flags can be specified in the buildman 958 settings file. They can also be useful when building U-Boot against other 959 open source software. 960 961 [make-flags] 962 at91-boards=ENABLE_AT91_TEST=1 963 snapper9260=${at91-boards} BUILD_TAG=442 964 snapper9g45=${at91-boards} BUILD_TAG=443 965 966 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260 967 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special 968 variable ${target} is available to access the target name (snapper9260 969 and snapper9g20 in this case). Variables are resolved recursively. Note 970 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-) 971 and underscore (_). 972 973 It is expected that any variables added are dealt with in U-Boot's 974 config.mk file and documented in the README. 975 976 Note that you can pass ad-hoc options to the build using environment 977 variables, for example: 978 979 SOME_OPTION=1234 ./tools/buildman/buildman my_board 980 981 982Quick Sanity Check 983------------------ 984 985If you have made changes and want to do a quick sanity check of the 986currently checked-out source, run buildman without the -b flag. This will 987build the selected boards and display build status as it runs (i.e. -v is 988enabled automatically). Use -e to see errors/warnings as well. 989 990 991Building Ranges 992--------------- 993 994You can build a range of commits by specifying a range instead of a branch 995when using the -b flag. For example:: 996 997 buildman -b upstream/master..us-buildman 998 999will build commits in us-buildman that are not in upstream/master. 1000 1001 1002Building Faster 1003--------------- 1004 1005By default, buildman doesn't execute 'make mrproper' prior to building the 1006first commit for each board. This reduces the amount of work 'make' does, and 1007hence speeds up the build. To force use of 'make mrproper', use -the -m flag. 1008This flag will slow down any buildman invocation, since it increases the amount 1009of work done on any build. An alternative is to use the --fallback-mrproper 1010flag, which retries the build with 'make mrproper' only after a build failure. 1011 1012One possible application of buildman is as part of a continual edit, build, 1013edit, build, ... cycle; repeatedly applying buildman to the same change or 1014series of changes while making small incremental modifications to the source 1015each time. This provides quick feedback regarding the correctness of recent 1016modifications. In this scenario, buildman's default choice of build directory 1017causes more build work to be performed than strictly necessary. 1018 1019By default, each buildman thread uses a single directory for all builds. When a 1020thread builds multiple boards, the configuration built in this directory will 1021cycle through various different configurations, one per board built by the 1022thread. Variations in the configuration will force a rebuild of affected source 1023files when a thread switches between boards. Ideally, such buildman-induced 1024rebuilds would not happen, thus allowing the build to operate as efficiently as 1025the build system and source changes allow. buildman's -P flag may be used to 1026enable this; -P causes each board to be built in a separate (board-specific) 1027directory, thus avoiding any buildman-induced configuration changes in any 1028build directory. 1029 1030U-Boot's build system embeds information such as a build timestamp into the 1031final binary. This information varies each time U-Boot is built. This causes 1032various files to be rebuilt even if no source changes are made, which in turn 1033requires that the final U-Boot binary be re-linked. This unnecessary work can 1034be avoided by turning off the timestamp feature. This can be achieved using 1035the `-r` flag, which enables reproducible builds by setting 1036`SOURCE_DATE_EPOCH=0` when building. 1037 1038Combining all of these options together yields the command-line shown below. 1039This will provide the quickest possible feedback regarding the current content 1040of the source tree, thus allowing rapid tested evolution of the code:: 1041 1042 ./tools/buildman/buildman -Pr tegra 1043 1044Note also the `--dtc-skip` option which uses the system device-tree compiler to 1045avoid needing to build it for each board. This can save 10-20% of build time. 1046An alternative is to set DTC=/path/to/dtc when running buildman. 1047 1048Checking configuration 1049---------------------- 1050 1051A common requirement when converting CONFIG options to Kconfig is to check 1052that the effective configuration has not changed due to the conversion. 1053Buildman supports this with the -K option, used after a build. This shows 1054differences in effective configuration between one commit and the next. 1055 1056For example:: 1057 1058 $ buildman -b kc4 -sK 1059 ... 1060 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig 1061 arm: 1062 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1 1063 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 1064 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1 1065 am335x_evm_usbspl : 1066 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1 1067 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 1068 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1 1069 44: Convert CONFIG_SPL_USB_HOST to Kconfig 1070 ... 1071 1072This shows that commit 44 enabled three new options for the board 1073am335x_evm_usbspl which were not enabled in commit 43. There is also a 1074summary for 'arm' showing all the changes detected for that architecture. 1075In this case there is only one board with changes, so 'arm' output is the 1076same as 'am335x_evm_usbspl'/ 1077 1078The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg 1079files which are produced by a build. If all you want is to check the 1080configuration you can in fact avoid doing a full build, using --config-only. 1081This tells buildman to configuration U-Boot and create the .cfg files, but not 1082actually build the source. This is 5-10 times faster than doing a full build. 1083 1084By default buildman considers the follow two configuration methods 1085equivalent:: 1086 1087 #define CONFIG_SOME_OPTION 1088 1089 CONFIG_SOME_OPTION=y 1090 1091The former would appear in a header filer and the latter in a defconfig 1092file. The achieve this, buildman considers 'y' to be '1' in configuration 1093variables. This avoids lots of useless output when converting a CONFIG 1094option to Kconfig. To disable this behaviour, use --squash-config-y. 1095 1096 1097Checking the environment 1098------------------------ 1099 1100When converting CONFIG options which manipulate the default environment, 1101a common requirement is to check that the default environment has not 1102changed due to the conversion. Buildman supports this with the -U option, 1103used after a build. This shows differences in the default environment 1104between one commit and the next. 1105 1106For example:: 1107 1108 $ buildman -b squash brppt1 -sU 1109 Summary of 2 commits for 3 boards (3 threads, 3 jobs per thread) 1110 01: Migrate bootlimit to Kconfig 1111 02: Squashed commit of the following: 1112 c brppt1_mmc: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0 1113 c brppt1_spi: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0 1114 + brppt1_nand: altbootcmd=run usbscript 1115 - brppt1_nand: altbootcmd=run usbscript 1116 (no errors to report) 1117 1118This shows that commit 2 modified the value of 'altbootcmd' for 'brppt1_mmc' 1119and 'brppt1_spi', removing a trailing semicolon. 'brppt1_nand' gained an a 1120value for 'altbootcmd', but lost one for ' altbootcmd'. 1121 1122The -U option uses the u-boot.env files which are produced by a build. 1123Internally, buildman writes out an out-env file into the build directory for 1124later comparison. 1125 1126defconfig fragments 1127------------------- 1128 1129Buildman provides some initial support for configuration fragments. It can scan 1130these when present in defconfig files and handle the resuiting Kconfig 1131correctly. Thus it is possible to build a board which has a ``#include`` in the 1132defconfig file. 1133 1134For now, Buildman simply includes the files to produce a single output file, 1135using the C preprocessor. It does not call the ``merge_config.sh`` script. The 1136redefined/redundant logic in that script could fairly easily be repeated in 1137Buildman, to detect potential problems. For now it is not clear that this is 1138useful. 1139 1140To specify the C preprocessor to use, set the ``CPP`` environment variable. The 1141default is ``cpp``. 1142 1143Note that Buildman does not support adding fragments to existing boards, e.g. 1144like:: 1145 1146 make qemu_riscv64_defconfig acpi.config 1147 1148This is partly because there is no way for Buildman to know which fragments are 1149valid on which boards. 1150 1151Building with clang 1152------------------- 1153 1154To build with clang (sandbox only), use the -O option to override the 1155toolchain. For example: 1156 1157.. code-block:: bash 1158 1159 buildman -O clang-7 --board sandbox 1160 1161 1162Building without LTO 1163-------------------- 1164 1165Link-time optimisation (LTO) is designed to reduce code size by globally 1166optimising the U-Boot build. Unfortunately this can dramatically slow down 1167builds. This is particularly noticeable when running a lot of builds. 1168 1169Use the -L (--no-lto) flag to disable LTO. 1170 1171.. code-block:: bash 1172 1173 buildman -L --board sandbox 1174 1175 1176Doing a simple build 1177-------------------- 1178 1179In some cases you just want to build a single board and get the full output, use 1180the -w option, for example: 1181 1182.. code-block:: bash 1183 1184 buildman -o /tmp/build --board sandbox -w 1185 1186This will write the full build into /tmp/build including object files. You must 1187specify the output directory with -o when using -w. 1188 1189 1190Support for IDEs (Integrated Development Environments) 1191------------------------------------------------------ 1192 1193Normally buildman summarises the output and shows information indicating the 1194meaning of each line of output. For example a '+' symbol appears at the start of 1195each error line. Also, buildman prints information about what it is about to do, 1196along with a summary at the end. 1197 1198When using buildman from an IDE, it is helpful to drop this behaviour. Use the 1199-I/--ide option for that. You might find -W helpful also so that warnings do 1200not cause the build to fail: 1201 1202.. code-block:: bash 1203 1204 buildman -o /tmp/build --board sandbox -wWI 1205 1206 1207Support for binary blobs 1208------------------------ 1209 1210U-Boot is moving to using Binman (see :doc:`../develop/package/binman`) for 1211dealing with the complexities of packaging U-Boot along with binary files from 1212other projects. These are called 'external blobs' by Binman. 1213 1214Typically a missing external blob causes a build failure. For build testing of 1215a lot of boards, or boards for which you do not have the blobs, you can use the 1216-M flag to allow missing blobs. This marks the build as if it succeeded, 1217although with warnings shown, including 'Some images are invalid'. If any boards 1218fail in this way, buildman exits with status 101. 1219 1220To convert warnings to errors, use -E. To make buildman return success with 1221these warnings, use -W. 1222 1223It is generally safe to default to enabling -M for all runs of buildman, so long 1224as you check the exit code. To do this, add:: 1225 1226 allow-missing = "always" 1227 1228to the top of the buildman_settings_ file. 1229 1230 1231Changing the configuration 1232-------------------------- 1233 1234Sometimes it is useful to change the CONFIG options for a build on the fly. This 1235can be used to build a board (or multiple) with a few changes to see the impact. 1236The -a option supports this: 1237 1238.. code-block:: bash 1239 1240 -a <cfg> 1241 1242where <cfg> is a CONFIG option (with or without the `CONFIG_` prefix) to enable. 1243For example: 1244 1245.. code-block:: bash 1246 1247 buildman -a CMD_SETEXPR_FMT 1248 1249will build with CONFIG_CMD_SETEXPR_FMT enabled. 1250 1251You can disable options by preceding them with tilde (~). You can specify the 1252-a option multiple times: 1253 1254.. code-block:: bash 1255 1256 buildman -a CMD_SETEXPR_FMT -a ~CMDLINE 1257 1258Some options have values, in which case you can change them: 1259 1260.. code-block:: bash 1261 1262 buildman -a 'BOOTCOMMAND="echo hello"' CONFIG_SYS_LOAD_ADDR=0x1000 1263 1264Note that you must put quotes around string options and the whole thing must be 1265in single quotes, to make sure the shell leave it alone. 1266 1267If you try to set an option that does not exist, or that cannot be changed for 1268some other reason (e.g. it is 'selected' by another option), then buildman 1269shows an error:: 1270 1271 $ buildman --board sandbox -a FRED 1272 Building current source for 1 boards (1 thread, 32 jobs per thread) 1273 0 0 0 /1 -1 (starting)errs 1274 Some CONFIG adjustments did not take effect. This may be because 1275 the request CONFIGs do not exist or conflict with others. 1276 1277 Failed adjustments: 1278 1279 FRED Missing expected line: CONFIG_FRED=y 1280 1281 1282One major caveat with this feature with branches (-b) is that buildman does not 1283name the output directories differently when you change the configuration, so 1284doing the same build again with different configuration will not trigger a 1285rebuild. You can use -f to work around that. 1286 1287 1288Other options 1289------------- 1290 1291Buildman has various other command-line options. Try --help to see them. 1292 1293To find out what toolchain prefix buildman will use for a build, use the -A 1294option. 1295 1296To request that compiler warnings be promoted to errors, use -E. This passes the 1297-Werror flag to the compiler. Note that the build can still produce warnings 1298with -E, e.g. the migration warnings:: 1299 1300 ===================== WARNING ====================== 1301 This board does not use CONFIG_DM_MMC. Please update 1302 ... 1303 ==================================================== 1304 1305When doing builds, Buildman's return code will reflect the overall result:: 1306 1307 0 (success) No errors or warnings found 1308 100 Errors found 1309 101 Warnings found (only if no -W) 1310 1311You can use -W to tell Buildman to return 0 (success) instead of 101 when 1312warnings are found. Note that it can be useful to combine -E and -W. This means 1313that all compiler warnings will produce failures (code 100) and all other 1314warnings will produce success (since 101 is changed to 0). 1315 1316If there are both warnings and errors, errors win, so buildman returns 100. 1317 1318The -y option is provided (for use with -s) to ignore the bountiful device-tree 1319warnings. Similarly, -Y tells buildman to ignore the migration warnings. 1320 1321Sometimes you might get an error in a thread that is not handled by buildman, 1322perhaps due to a failure of a tool that it calls. You might see the output, but 1323then buildman hangs. Failing to handle any eventuality is a bug in buildman and 1324should be reported. But you can use -T0 to disable threading and hopefully 1325figure out the root cause of the build failure. 1326 1327For situations where buildman is invoked from multiple running processes, it is 1328sometimes useful to have buildman wait until the others have finished. Use the 1329--process-limit option for this: --process-limit 1 will allow only one buildman 1330to process jobs at a time. 1331 1332To build a particular target, rather than the default U-Boot target, use the 1333`--target` option. This is unlikely to be useful unless you are building a 1334single board. 1335 1336Buildman normally builds out-of-tree, meaning that the source directory is not 1337disturbed by the build. Use `-i` to do an in-tree build instead. Note that this 1338does not affect the source directory, since buildman creates a separate git 1339'worktree' for each board. This means that it is possible to do an in-tree 1340build of an entire branch, or even a 'current source' build for multiple boards. 1341As a special case, you can use `-wi` to do an in-tree build in the current 1342directory. 1343 1344Build summary 1345------------- 1346 1347When buildman finishes it shows a summary, something like this:: 1348 1349 Completed: 5 total built, duration 0:00:21, rate 0.24 1350 1351This shows that a total of 5 builds were done across all selected boards, it 1352took 21 seconds and the builds happened at the rate of 0.24 per second. The 1353latter number depends on the speed of your machine and the efficiency of the 1354U-Boot build. 1355 1356 1357Using boards.cfg 1358---------------- 1359 1360This file is no-longer needed by buildman but it is still generated in the 1361working directory. This helps avoid a delay on every build, since scanning all 1362the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force 1363regeneration of the file - in that case buildman exits after writing the file 1364with exit code 2 if there was an error in the maintainer files. To use the 1365default filename, use a hyphen, i.e. `-R -`. 1366 1367You should use 'buildman -nv <criteria>' instead of greoing the boards.cfg file, 1368since it may be dropped altogether in future. 1369 1370 1371Checking maintainers 1372-------------------- 1373 1374Sometimes a board is added without a corresponding entry in a MAINTAINERS file. 1375Use the `--maintainer-check` option to check this:: 1376 1377 $ buildman --maintainer-check 1378 WARNING: board/mikrotik/crs3xx-98dx3236/MAINTAINERS: missing defconfig ending at line 7 1379 WARNING: no maintainers for 'clearfog_spi' 1380 1381Buildman returns with an exit code of 2 if there area any warnings. 1382 1383An experimental `--full-check option` also checks for boards which don't have a 1384CONFIG_TARGET_xxx where xxx corresponds to their defconfig filename. This is 1385not strictly necessary, but may be useful information. 1386 1387 1388Checking the command 1389-------------------- 1390 1391Buildman writes out the toolchain information to a `toolchain` file within the 1392output directory. It also writes the commands used to build U-Boot in an 1393`out-cmd` file. You can check these if you suspect something strange is 1394happening. 1395 1396TODO 1397---- 1398 1399Many improvements have been made over the years. There is still quite a bit of 1400scope for more though, e.g.: 1401 1402- easier access to log files 1403- 'hunting' for problems, perhaps by building a few boards for each arch, or 1404 checking commits for changed files and building only boards which use those 1405 files 1406 1407 1408Credits 1409------- 1410 1411Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving 1412the build speed by building all commits for a board instead of the other 1413way around. 1414 1415.. sectionauthor:: Simon Glass 1416.. sectionauthor:: Copyright (c) 2013 The Chromium OS Authors. 1417.. sectionauthor:: sjg@chromium.org 1418.. Halloween 2012 1419.. Updated 12-12-12 1420.. Updated 23-02-13 1421.. Updated 09-04-20 1422