1================================== 2The QEMU build system architecture 3================================== 4 5This document aims to help developers understand the architecture of the 6QEMU build system. As with projects using GNU autotools, the QEMU build 7system has two stages; first the developer runs the "configure" script 8to determine the local build environment characteristics, then they run 9"make" to build the project. This is about where the similarities with 10GNU autotools end, so try to forget what you know about them. 11 12The two general ways to perform a build are as follows: 13 14 - build artifacts outside of QEMU source tree entirely:: 15 16 cd ../ 17 mkdir build 18 cd build 19 ../qemu/configure 20 make 21 22 - build artifacts in a subdir of QEMU source tree:: 23 24 mkdir build 25 cd build 26 ../configure 27 make 28 29Most of the actual build process uses Meson under the hood, therefore 30build artifacts cannot be placed in the source tree itself. 31 32 33Stage 1: configure 34================== 35 36The configure script has five tasks: 37 38 - detect the host architecture 39 40 - list the targets for which to build emulators; the list of 41 targets also affects which firmware binaries and tests to build 42 43 - find the compilers (native and cross) used to build executables, 44 firmware and tests. The results are written as either Makefile 45 fragments (``config-host.mak``) or a Meson machine file 46 (``config-meson.cross``) 47 48 - create a virtual environment in which all Python code runs during 49 the build, and possibly install packages into it from PyPI 50 51 - invoke Meson in the virtual environment, to perform the actual 52 configuration step for the emulator build 53 54The configure script automatically recognizes command line options for 55which a same-named Meson option exists; dashes in the command line are 56replaced with underscores. 57 58Almost all QEMU developers that need to modify the build system will 59only be concerned with Meson, and therefore can skip the rest of this 60section. 61 62 63Modifying ``configure`` 64----------------------- 65 66``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore 67should be compatible with any POSIX shell. It is important to avoid 68using bash-isms to avoid breaking development platforms where bash is 69the primary host. 70 71The configure script provides a variety of functions to help writing 72portable shell code and providing consistent behavior across architectures 73and operating systems: 74 75``error_exit $MESSAGE $MORE...`` 76 Print $MESSAGE to stderr, followed by $MORE... and then exit from the 77 configure script with non-zero status. 78 79``has $COMMAND`` 80 Determine if $COMMAND exists in the current environment, either as a 81 shell builtin, or executable binary, returning 0 on success. The 82 replacement in Meson is ``find_program()``. 83 84``probe_target_compiler $TARGET`` 85 Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g., 86 ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working 87 compiler is present, return success and set variables ``$target_cc``, 88 ``$target_ar``, etc. to non-empty values. 89 90``write_target_makefile`` 91 Write a Makefile fragment to stdout, exposing the result of the most 92 ``probe_target_compiler`` call as the usual Make variables (``CC``, 93 ``AR``, ``LD``, etc.). 94 95 96Configure does not generally perform tests for compiler options beyond 97basic checks to detect the host platform and ensure the compiler is 98functioning. These are performed using a few more helper functions: 99 100``compile_object $CFLAGS`` 101 Attempt to compile a test program with the system C compiler using 102 $CFLAGS. The test program must have been previously written to a file 103 called $TMPC. 104 105``compile_prog $CFLAGS $LDFLAGS`` 106 Attempt to compile a test program with the system C compiler using 107 $CFLAGS and link it with the system linker using $LDFLAGS. The test 108 program must have been previously written to a file called $TMPC. 109 110``check_define $NAME`` 111 Determine if the macro $NAME is defined by the system C compiler. 112 113``do_compiler $CC $ARGS...`` 114 Attempt to run the C compiler $CC, passing it $ARGS... This function 115 does not use flags passed via options such as ``--extra-cflags``, and 116 therefore can be used to check for cross compilers. However, most 117 such checks are done at ``make`` time instead (see for example the 118 ``cc-option`` macro in ``pc-bios/option-rom/Makefile``). 119 120``write_c_skeleton`` 121 Write a minimal C program main() function to the temporary file 122 indicated by $TMPC. 123 124 125Python virtual environments and the build process 126------------------------------------------------- 127 128An important step in ``configure`` is to create a Python virtual 129environment (venv) during the configuration phase. The Python interpreter 130comes from the ``--python`` command line option, the ``$PYTHON`` variable 131from the environment, or the system PATH, in this order. The venv resides 132in the ``pyvenv`` directory in the build tree, and provides consistency 133in how the build process runs Python code. 134 135At this stage, ``configure`` also queries the chosen Python interpreter 136about QEMU's build dependencies. Note that the build process does *not* 137look for ``meson`` or ``sphinx-build`` binaries in the PATH; 138likewise, there are no options such as ``--meson`` or ``--sphinx-build``. 139This avoids a potential mismatch, where Meson and Sphinx binaries on the 140PATH might operate in a different Python environment than the one chosen 141by the user during the build process. On the other hand, it introduces 142a potential source of confusion where the user installs a dependency but 143``configure`` is not able to find it. When this happens, the dependency 144was installed in the ``site-packages`` directory of another interpreter, 145or with the wrong ``pip`` program. 146 147If a package is available for the chosen interpreter, ``configure`` 148prepares a small script that invokes it from the venv itself\ [#distlib]_. 149If not, ``configure`` can also optionally install dependencies in the 150virtual environment with ``pip``, either from wheels in ``python/wheels`` 151or by downloading the package with PyPI. Downloading can be disabled with 152``--disable-download``; and anyway, it only happens when a ``configure`` 153option (currently, only ``--enable-docs``) is explicitly enabled but 154the dependencies are not present. 155 156.. [#distlib] The scripts are created based on the package's metadata, 157 specifically the ``console_script`` entry points. This is the 158 same mechanism that ``pip`` uses when installing a package. 159 Currently, in all cases it would be possible to use ``python -m`` 160 instead of an entry point script, which makes this approach a 161 bit overkill. On the other hand, creating the scripts is 162 future proof and it makes the contents of the ``pyvenv/bin`` 163 directory more informative. Portability is also not an issue, 164 because the Python Packaging Authority provides a package 165 ``distlib.scripts`` to perform this task. 166 167The required versions of the packages are stored in a configuration file 168``pythondeps.toml``. The format is custom to QEMU, but it is documented 169at the top of the file itself and it should be easy to understand. The 170requirements should make it possible to use the version that is packaged 171by QEMU's supported distros. 172 173When dependencies are downloaded, instead, ``configure`` uses a "known 174good" version that is also listed in ``pythondeps.toml``. In this 175scenario, ``pythondeps.toml`` behaves like the "lock file" used by 176``cargo``, ``poetry`` or other dependency management systems. 177 178 179Bundled Python packages 180----------------------- 181 182Python packages that are **mandatory** dependencies to build QEMU, 183but are not available in all supported distros, are bundled with the 184QEMU sources. The only one is currently Meson (outdated in Ubuntu 18522.04 and openSUSE Leap). 186 187In order to include a new or updated wheel, modify and rerun the 188``python/scripts/vendor.py`` script. The script embeds the 189sha256 hash of package sources and checks it. The pypi.org web site 190provides an easy way to retrieve the sha256 hash of the sources. 191 192 193Stage 2: Meson 194============== 195 196The Meson build system describes the build and install process for: 197 1981) executables, which include: 199 200 - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc 201 202 - System emulators - ``qemu-system-$ARCH`` 203 204 - Userspace emulators - ``qemu-$ARCH`` 205 206 - Unit tests 207 2082) documentation 209 2103) ROMs, whether provided as binary blobs in the QEMU distributions 211 or cross compiled under the direction of the configure script 212 2134) other data files, such as icons or desktop files 214 215All executables are built by default, except for some ``contrib/`` 216binaries that are known to fail to build on some platforms (for example 21732-bit or big-endian platforms). Tests are also built by default, 218though that might change in the future. 219 220The source code is highly modularized, split across many files to 221facilitate building of all of these components with as little duplicated 222compilation as possible. Using the Meson "sourceset" functionality, 223``meson.build`` files group the source files in rules that are 224enabled according to the available system libraries and to various 225configuration symbols. Sourcesets belong to one of four groups: 226 227Subsystem sourcesets: 228 Various subsystems that are common to both tools and emulators have 229 their own sourceset, for example ``block_ss`` for the block device subsystem, 230 ``chardev_ss`` for the character device subsystem, etc. These sourcesets 231 are then turned into static libraries as follows:: 232 233 libchardev = static_library('chardev', chardev_ss.sources(), 234 build_by_default: false) 235 236 chardev = declare_dependency(objects: libchardev.extract_all_objects(recursive: false), 237 dependencies: chardev_ss.dependencies()) 238 239Target-independent emulator sourcesets: 240 Various general purpose helper code is compiled only once and 241 the .o files are linked into all output binaries that need it. 242 This includes error handling infrastructure, standard data structures, 243 platform portability wrapper functions, etc. 244 245 Target-independent code lives in the ``common_ss``, ``system_ss`` and 246 ``user_ss`` sourcesets. ``common_ss`` is linked into all emulators, 247 ``system_ss`` only in system emulators, ``user_ss`` only in user-mode 248 emulators. 249 250Target-dependent emulator sourcesets: 251 In the target-dependent set lives CPU emulation, some device emulation and 252 much glue code. This sometimes also has to be compiled multiple times, 253 once for each target being built. Target-dependent files are included 254 in the ``specific_ss`` sourceset. 255 256 Each emulator also includes sources for files in the ``hw/`` and ``target/`` 257 subdirectories. The subdirectory used for each emulator comes 258 from the target's definition of ``TARGET_BASE_ARCH`` or (if missing) 259 ``TARGET_ARCH``, as found in ``configs/targets/*.mak``. 260 261 Each subdirectory in ``hw/`` adds one sourceset to the ``hw_arch`` dictionary, 262 for example:: 263 264 arm_ss = ss.source_set() 265 arm_ss.add(files('boot.c'), fdt) 266 ... 267 hw_arch += {'arm': arm_ss} 268 269 The sourceset is only used for system emulators. 270 271 Each subdirectory in ``target/`` instead should add one sourceset to each 272 of the ``target_arch`` and ``target_system_arch``, which are used respectively 273 for all emulators and for system emulators only. For example:: 274 275 arm_ss = ss.source_set() 276 arm_system_ss = ss.source_set() 277 ... 278 target_arch += {'arm': arm_ss} 279 target_system_arch += {'arm': arm_system_ss} 280 281Module sourcesets: 282 There are two dictionaries for modules: ``modules`` is used for 283 target-independent modules and ``target_modules`` is used for 284 target-dependent modules. When modules are disabled the ``module`` 285 source sets are added to ``system_ss`` and the ``target_modules`` 286 source sets are added to ``specific_ss``. 287 288 Both dictionaries are nested. One dictionary is created per 289 subdirectory, and these per-subdirectory dictionaries are added to 290 the toplevel dictionaries. For example:: 291 292 hw_display_modules = {} 293 qxl_ss = ss.source_set() 294 ... 295 hw_display_modules += { 'qxl': qxl_ss } 296 modules += { 'hw-display': hw_display_modules } 297 298Utility sourcesets: 299 All binaries link with a static library ``libqemuutil.a``. This library 300 is built from several sourcesets; most of them however host generated 301 code, and the only two of general interest are ``util_ss`` and ``stub_ss``. 302 303 The separation between these two is purely for documentation purposes. 304 ``util_ss`` contains generic utility files. Even though this code is only 305 linked in some binaries, sometimes it requires hooks only in some of 306 these and depend on other functions that are not fully implemented by 307 all QEMU binaries. ``stub_ss`` links dummy stubs that will only be linked 308 into the binary if the real implementation is not present. In a way, 309 the stubs can be thought of as a portable implementation of the weak 310 symbols concept. 311 312 313The following files concur in the definition of which files are linked 314into each emulator: 315 316``configs/devices/*.mak`` 317 The files under ``configs/devices/`` control the boards and devices 318 that are built into each QEMU system emulation targets. They merely contain 319 a list of config variable definitions such as:: 320 321 include arm-softmmu.mak 322 CONFIG_XLNX_ZYNQMP_ARM=y 323 CONFIG_XLNX_VERSAL=y 324 325``*/Kconfig`` 326 These files are processed together with ``configs/devices/*.mak`` and 327 describe the dependencies between various features, subsystems and 328 device models. They are described in :ref:`kconfig` 329 330``configs/targets/*.mak`` 331 These files mostly define symbols that appear in the ``*-config-target.h`` 332 file for each emulator\ [#cfgtarget]_. However, the ``TARGET_ARCH`` 333 and ``TARGET_BASE_ARCH`` will also be used to select the ``hw/`` and 334 ``target/`` subdirectories that are compiled into each target. 335 336.. [#cfgtarget] This header is included by ``qemu/osdep.h`` when 337 compiling files from the target-specific sourcesets. 338 339These files rarely need changing unless you are adding a completely 340new target, or enabling new devices or hardware for a particular 341system/userspace emulation target 342 343 344Adding checks 345------------- 346 347Compiler checks can be as simple as the following:: 348 349 config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h')) 350 351A more complex task such as adding a new dependency usually 352comprises the following tasks: 353 354 - Add a Meson build option to meson_options.txt. 355 356 - Add code to perform the actual feature check. 357 358 - Add code to include the feature status in ``config-host.h`` 359 360 - Add code to print out the feature status in the configure summary 361 upon completion. 362 363Taking the probe for SDL2_Image as an example, we have the following 364in ``meson_options.txt``:: 365 366 option('sdl_image', type : 'feature', value : 'auto', 367 description: 'SDL Image support for icons') 368 369Unless the option was given a non-``auto`` value (on the configure 370command line), the detection code must be performed only if the 371dependency will be used:: 372 373 sdl_image = not_found 374 if not get_option('sdl_image').auto() or have_system 375 sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), 376 method: 'pkg-config') 377 endif 378 379This avoids warnings on static builds of user-mode emulators, for example. 380Most of the libraries used by system-mode emulators are not available for 381static linking. 382 383The other supporting code is generally simple:: 384 385 # Create config-host.h (if applicable) 386 config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found()) 387 388 # Summary 389 summary_info += {'SDL image support': sdl_image.found()} 390 391For the configure script to parse the new option, the 392``scripts/meson-buildoptions.sh`` file must be up-to-date; ``make 393update-buildoptions`` (or just ``make``) will take care of updating it. 394 395 396Support scripts 397--------------- 398 399Meson has a special convention for invoking Python scripts: if their 400first line is ``#! /usr/bin/env python3`` and the file is *not* executable, 401find_program() arranges to invoke the script under the same Python 402interpreter that was used to invoke Meson. This is the most common 403and preferred way to invoke support scripts from Meson build files, 404because it automatically uses the value of configure's --python= option. 405 406In case the script is not written in Python, use a ``#! /usr/bin/env ...`` 407line and make the script executable. 408 409Scripts written in Python, where it is desirable to make the script 410executable (for example for test scripts that developers may want to 411invoke from the command line, such as tests/qapi-schema/test-qapi.py), 412should be invoked through the ``python`` variable in meson.build. For 413example:: 414 415 test('QAPI schema regression tests', python, 416 args: files('test-qapi.py'), 417 env: test_env, suite: ['qapi-schema', 'qapi-frontend']) 418 419This is needed to obey the --python= option passed to the configure 420script, which may point to something other than the first python3 421binary on the path. 422 423By the time Meson runs, Python dependencies are available in the virtual 424environment and should be invoked through the scripts that ``configure`` 425places under ``pyvenv``. One way to do so is as follows, using Meson's 426``find_program`` function:: 427 428 sphinx_build = find_program( 429 fs.parent(python.full_path()) / 'sphinx-build', 430 required: get_option('docs')) 431 432 433Stage 3: Make 434============= 435 436The next step in building QEMU is to invoke make. GNU Make is required 437to build QEMU, and may be installed as ``gmake`` on some hosts. 438 439The output of Meson is a ``build.ninja`` file, which is used with the 440Ninja build tool. However, QEMU's build comprises other components than 441just the emulators (namely firmware and the tests in ``tests/tcg``) which 442need different cross compilers. The QEMU Makefile wraps both Ninja and 443the smaller build systems for firmware and tests; it also takes care of 444running ``configure`` again when the script changes. Apart from invoking 445these sub-Makefiles, the resulting build is largely non-recursive. 446 447Tests, whether defined in ``meson.build`` or not, are also ran by the 448Makefile with the traditional ``make check`` phony target, while benchmarks 449are run with ``make bench``. Meson test suites such as ``unit`` can be ran 450with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson" 451tests for all targets. 452 453If desired, it is also possible to use ``ninja`` and ``meson test``, 454respectively to build emulators and run tests defined in meson.build. 455The main difference is that ``make`` needs the ``-jN`` flag in order to 456enable parallel builds or tests. 457 458Useful make targets 459------------------- 460 461``help`` 462 Print a help message for the most common build targets. 463 464``print-VAR`` 465 Print the value of the variable VAR. Useful for debugging the build 466 system. 467 468 469Important files for the build system 470==================================== 471 472Statically defined files 473------------------------ 474 475The following key files are statically defined in the source tree, with 476the rules needed to build QEMU. Their behaviour is influenced by a 477number of dynamically created files listed later. 478 479``Makefile`` 480 The main entry point used when invoking make to build all the components 481 of QEMU. The default 'all' target will naturally result in the build of 482 every component. 483 484``*/meson.build`` 485 The meson.build file in the root directory is the main entry point for the 486 Meson build system, and it coordinates the configuration and build of all 487 executables. Build rules for various subdirectories are included in 488 other meson.build files spread throughout the QEMU source tree. 489 490``python/scripts/mkvenv.py`` 491 A wrapper for the Python ``venv`` and ``distlib.scripts`` packages. 492 It handles creating the virtual environment, creating scripts in 493 ``pyvenv/bin``, and calling ``pip`` to install dependencies. 494 495``tests/Makefile.include`` 496 Rules for external test harnesses like the TCG tests. 497 498``tests/docker/Makefile.include`` 499 Rules for Docker tests. Like ``tests/Makefile.include``, this file is 500 included directly by the top level Makefile, anything defined in this 501 file will influence the entire build system. 502 503``tests/vm/Makefile.include`` 504 Rules for VM-based tests. Like ``tests/Makefile.include``, this file is 505 included directly by the top level Makefile, anything defined in this 506 file will influence the entire build system. 507 508Dynamically created files 509------------------------- 510 511The following files are generated at run-time in order to control the 512behaviour of the Makefiles. This avoids the need for QEMU makefiles to 513go through any pre-processing as seen with autotools, where configure 514generates ``Makefile`` from ``Makefile.in``. 515 516Built by configure: 517 518``config-host.mak`` 519 When configure has determined the characteristics of the build host it 520 will write the paths to various tools to this file, for use in ``Makefile`` 521 and to a smaller extent ``meson.build``. 522 523 ``config-host.mak`` is also used as a dependency checking mechanism. If make 524 sees that the modification timestamp on configure is newer than that on 525 ``config-host.mak``, then configure will be re-run. 526 527``config-meson.cross`` 528 529 A Meson "cross file" (or native file) used to communicate the paths to 530 the toolchain and other configuration options. 531 532``config.status`` 533 534 A small shell script that will invoke configure again with the same 535 environment variables that were set during the first run. It's used to 536 rerun configure after changes to the source code, but it can also be 537 inspected manually to check the contents of the environment. 538 539``Makefile.prereqs`` 540 541 A set of Makefile dependencies that order the build and execution of 542 firmware and tests after the container images and emulators that they 543 need. 544 545``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak`` 546 547 Configuration variables used to build the firmware and TCG tests, 548 including paths to cross compilation toolchains. 549 550``pyvenv`` 551 552 A Python virtual environment that is used for all Python code running 553 during the build. Using a virtual environment ensures that even code 554 that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter 555 and packages. 556 557Built by Meson: 558 559``config-host.h`` 560 Used by C code to determine the properties of the build environment 561 and the set of enabled features for the entire build. 562 563``${TARGET-NAME}-config-devices.mak`` 564 TARGET-NAME is the name of a system emulator. The file is 565 generated by Meson using files under ``configs/devices`` as input. 566 567``${TARGET-NAME}-config-target.mak`` 568 TARGET-NAME is the name of a system or usermode emulator. The file is 569 generated by Meson using files under ``configs/targets`` as input. 570 571``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h`` 572 Used by C code to determine the properties and enabled 573 features for each target. enabled. They are generated from 574 the contents of the corresponding ``*.mak`` files using Meson's 575 ``configure_file()`` function; each target can include them using 576 the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively. 577 578``build.ninja`` 579 The build rules. 580 581 582Built by Makefile: 583 584``Makefile.ninja`` 585 A Makefile include that bridges to ninja for the actual build. The 586 Makefile is mostly a list of targets that Meson included in build.ninja. 587 588``Makefile.mtest`` 589 The Makefile definitions that let "make check" run tests defined in 590 meson.build. The rules are produced from Meson's JSON description of 591 tests (obtained with "meson introspect --tests") through the script 592 scripts/mtest2make.py. 593