Lines Matching full:the
2 The QEMU build system architecture
5 This document aims to help developers understand the architecture of the
6 QEMU build system. As with projects using GNU autotools, the QEMU build
7 system has two stages; first the developer runs the "configure" script
8 to determine the local build environment characteristics, then they run
9 "make" to build the project. This is about where the similarities with
12 The two general ways to perform a build are as follows:
29 Most of the actual build process uses Meson under the hood, therefore
30 build artifacts cannot be placed in the source tree itself.
36 The configure script has five tasks:
38 - detect the host architecture
40 - list the targets for which to build emulators; the list of
43 - find the compilers (native and cross) used to build executables,
44 firmware and tests. The results are written as either Makefile
49 the build, and possibly install packages into it from PyPI
51 - invoke Meson in the virtual environment, to perform the actual
52 configuration step for the emulator build
54 The configure script automatically recognizes command line options for
55 which a same-named Meson option exists; dashes in the command line are
58 Almost all QEMU developers that need to modify the build system will
59 only be concerned with Meson, and therefore can skip the rest of this
69 the primary host.
71 The configure script provides a variety of functions to help writing
76 Print $MESSAGE to stderr, followed by $MORE... and then exit from the
80 Determine if $COMMAND exists in the current environment, either as a
81 shell builtin, or executable binary, returning 0 on success. The
85 Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
91 Write a Makefile fragment to stdout, exposing the result of the most
92 ``probe_target_compiler`` call as the usual Make variables (``CC``,
97 basic checks to detect the host platform and ensure the compiler is
101 Attempt to compile a test program with the system C compiler using
102 $CFLAGS. The test program must have been previously written to a file
106 Attempt to compile a test program with the system C compiler using
107 $CFLAGS and link it with the system linker using $LDFLAGS. The test
111 Determine if the macro $NAME is defined by the system C compiler.
114 Attempt to run the C compiler $CC, passing it $ARGS... This function
117 such checks are done at ``make`` time instead (see for example the
121 Write a minimal C program main() function to the temporary file
125 Python virtual environments and the build process
129 environment (venv) during the configuration phase. The Python interpreter
130 comes from the ``--python`` command line option, the ``$PYTHON`` variable
131 from the environment, or the system PATH, in this order. The venv resides
132 in the ``pyvenv`` directory in the build tree, and provides consistency
133 in how the build process runs Python code.
135 At this stage, ``configure`` also queries the chosen Python interpreter
136 about QEMU's build dependencies. Note that the build process does *not*
137 look for ``meson`` or ``sphinx-build`` binaries in the PATH;
139 This avoids a potential mismatch, where Meson and Sphinx binaries on the
140 PATH might operate in a different Python environment than the one chosen
141 by the user during the build process. On the other hand, it introduces
142 a potential source of confusion where the user installs a dependency but
143 ``configure`` is not able to find it. When this happens, the dependency
144 was installed in the ``site-packages`` directory of another interpreter,
145 or with the wrong ``pip`` program.
147 If a package is available for the chosen interpreter, ``configure``
148 prepares a small script that invokes it from the venv itself\ [#distlib]_.
149 If not, ``configure`` can also optionally install dependencies in the
151 or by downloading the package with PyPI. Downloading can be disabled with
154 the dependencies are not present.
156 .. [#distlib] The scripts are created based on the package's metadata,
157 specifically the ``console_script`` entry points. This is the
161 bit overkill. On the other hand, creating the scripts is
162 future proof and it makes the contents of the ``pyvenv/bin``
164 because the Python Packaging Authority provides a package
167 The required versions of the packages are stored in a configuration file
168 ``pythondeps.toml``. The format is custom to QEMU, but it is documented
169 at the top of the file itself and it should be easy to understand. The
170 requirements should make it possible to use the version that is packaged
175 scenario, ``pythondeps.toml`` behaves like the "lock file" used by
183 but are not available in all supported distros, are bundled with the
184 QEMU sources. The only one is currently Meson (outdated in Ubuntu
187 In order to include a new or updated wheel, modify and rerun the
188 ``python/scripts/vendor.py`` script. The script embeds the
189 sha256 hash of package sources and checks it. The pypi.org web site
190 provides an easy way to retrieve the sha256 hash of the sources.
196 The Meson build system describes the build and install process for:
210 3) ROMs, whether provided as binary blobs in the QEMU distributions
211 or cross compiled under the direction of the configure script
218 though that might change in the future.
220 The source code is highly modularized, split across many files to
222 compilation as possible. Using the Meson "sourceset" functionality,
223 ``meson.build`` files group the source files in rules that are
224 enabled according to the available system libraries and to various
229 their own sourceset, for example ``block_ss`` for the block device subsystem,
230 ``chardev_ss`` for the character device subsystem, etc. These sourcesets
241 the .o files are linked into all output binaries that need it.
245 Target-independent code lives in the ``common_ss``, ``system_ss`` and
251 In the target-dependent set lives CPU emulation, some device emulation and
254 in the ``specific_ss`` sourceset.
256 Each emulator also includes sources for files in the ``hw/`` and ``target/``
257 subdirectories. The subdirectory used for each emulator comes
258 from the target's definition of ``TARGET_BASE_ARCH`` or (if missing)
261 Each subdirectory in ``hw/`` adds one sourceset to the ``hw_arch`` dictionary,
269 The sourceset is only used for system emulators.
272 of the ``target_arch`` and ``target_system_arch``, which are used respectively
284 target-dependent modules. When modules are disabled the ``module``
285 source sets are added to ``system_ss`` and the ``target_modules``
290 the toplevel dictionaries. For example::
301 code, and the only two of general interest are ``util_ss`` and ``stub_ss``.
303 The separation between these two is purely for documentation purposes.
308 into the binary if the real implementation is not present. In a way,
309 the stubs can be thought of as a portable implementation of the weak
313 The following files concur in the definition of which files are linked
317 The files under ``configs/devices/`` control the boards and devices
327 describe the dependencies between various features, subsystems and
331 These files mostly define symbols that appear in the ``*-config-target.h``
332 file for each emulator\ [#cfgtarget]_. However, the ``TARGET_ARCH``
333 and ``TARGET_BASE_ARCH`` will also be used to select the ``hw/`` and
337 compiling files from the target-specific sourcesets.
347 Compiler checks can be as simple as the following::
352 comprises the following tasks:
356 - Add code to perform the actual feature check.
358 - Add code to include the feature status in ``config-host.h``
360 - Add code to print out the feature status in the configure summary
363 Taking the probe for SDL2_Image as an example, we have the following
369 Unless the option was given a non-``auto`` value (on the configure
370 command line), the detection code must be performed only if the
380 Most of the libraries used by system-mode emulators are not available for
383 The other supporting code is generally simple::
391 For the configure script to parse the new option, the
400 first line is ``#! /usr/bin/env python3`` and the file is *not* executable,
401 find_program() arranges to invoke the script under the same Python
402 interpreter that was used to invoke Meson. This is the most common
404 because it automatically uses the value of configure's --python= option.
406 In case the script is not written in Python, use a ``#! /usr/bin/env ...``
407 line and make the script executable.
409 Scripts written in Python, where it is desirable to make the script
411 invoke from the command line, such as tests/qapi-schema/test-qapi.py),
412 should be invoked through the ``python`` variable in meson.build. For
419 This is needed to obey the --python= option passed to the configure
420 script, which may point to something other than the first python3
421 binary on the path.
423 By the time Meson runs, Python dependencies are available in the virtual
424 environment and should be invoked through the scripts that ``configure``
436 The next step in building QEMU is to invoke make. GNU Make is required
439 The output of Meson is a ``build.ninja`` file, which is used with the
441 just the emulators (namely firmware and the tests in ``tests/tcg``) which
442 need different cross compilers. The QEMU Makefile wraps both Ninja and
443 the smaller build systems for firmware and tests; it also takes care of
444 running ``configure`` again when the script changes. Apart from invoking
445 these sub-Makefiles, the resulting build is largely non-recursive.
447 Tests, whether defined in ``meson.build`` or not, are also ran by the
448 Makefile with the traditional ``make check`` phony target, while benchmarks
455 The main difference is that ``make`` needs the ``-jN`` flag in order to
462 Print a help message for the most common build targets.
465 Print the value of the variable VAR. Useful for debugging the build
469 Important files for the build system
475 The following key files are statically defined in the source tree, with
476 the rules needed to build QEMU. Their behaviour is influenced by a
480 The main entry point used when invoking make to build all the components
481 of QEMU. The default 'all' target will naturally result in the build of
485 The meson.build file in the root directory is the main entry point for the
486 Meson build system, and it coordinates the configuration and build of all
488 other meson.build files spread throughout the QEMU source tree.
491 A wrapper for the Python ``venv`` and ``distlib.scripts`` packages.
492 It handles creating the virtual environment, creating scripts in
496 Rules for external test harnesses like the TCG tests.
500 included directly by the top level Makefile, anything defined in this
501 file will influence the entire build system.
505 included directly by the top level Makefile, anything defined in this
506 file will influence the entire build system.
511 The following files are generated at run-time in order to control the
512 behaviour of the Makefiles. This avoids the need for QEMU makefiles to
519 When configure has determined the characteristics of the build host it
520 will write the paths to various tools to this file, for use in ``Makefile``
524 sees that the modification timestamp on configure is newer than that on
529 A Meson "cross file" (or native file) used to communicate the paths to
530 the toolchain and other configuration options.
534 A small shell script that will invoke configure again with the same
535 environment variables that were set during the first run. It's used to
536 rerun configure after changes to the source code, but it can also be
537 inspected manually to check the contents of the environment.
541 A set of Makefile dependencies that order the build and execution of
542 firmware and tests after the container images and emulators that they
547 Configuration variables used to build the firmware and TCG tests,
553 during the build. Using a virtual environment ensures that even code
554 that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter
560 Used by C code to determine the properties of the build environment
561 and the set of enabled features for the entire build.
564 TARGET-NAME is the name of a system emulator. The file is
568 TARGET-NAME is the name of a system or usermode emulator. The file is
572 Used by C code to determine the properties and enabled
574 the contents of the corresponding ``*.mak`` files using Meson's
576 the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
579 The build rules.
585 A Makefile include that bridges to ninja for the actual build. The
589 The Makefile definitions that let "make check" run tests defined in
590 meson.build. The rules are produced from Meson's JSON description of
591 tests (obtained with "meson introspect --tests") through the script