Lines Matching full:to
8 the way to full blown functional tests. To get an overview of the
28 2. The call to the code that should be tested;
42 The usual way to run these tests is:
51 Before running tests, it is best to build QEMU programs first. Some tests
52 expect the executables to exist and will fail with obscure messages if they
64 structures. A test case of unit tests should be designed to uncover errors
65 due to erroneous computations, incorrect comparisons, or improper control
69 simple C tests that typically link to individual QEMU object files and
73 for utility modules that are relatively stateless or have few dependencies. To
83 3. Add the test to ``tests/unit/meson.build``. The unit tests are listed in a
85 dependencies to be linked with the test. For a simple test whose source
86 is in ``tests/unit/foo-test.c``, it is enough to add an entry like::
94 Since unit tests don't require environment variables, the simplest way to debug
97 invocations and your manual run, due to ``$MALLOC_PERTURB_`` environment
111 QTest is a device emulation testing framework. It can be very useful to test
113 clock stepping), with a special purpose "qtest" protocol. Refer to
134 "/dev/fd/2" and "/dev/null" on Linux, remember to use "2" and "nul" on
137 * If your test cases uses the blkdebug feature, use relative path to pass
142 passing the command line to QEMU.
145 test cases opens a file to write some data and later wants to compare the
146 written data with the original one, be sure to pass the letter 'b' as
147 part of the mode string to fopen(), or O_BINARY flag for the open() call.
158 predefined input to the parser and comparing the result with the reference
171 parser (either fixing a bug or extending/modifying the syntax). To do this:
194 framework widely used to test block layer related features. It is higher level
199 To run iotests, make sure QEMU is built successfully, then switch to the
214 It's also possible to list test numbers explicitly:
222 that are specific to certain cache mode.
230 Consider writing a tests case when you are making any changes to the block
233 and save the boilerplate to create one. (Unfortunately, there isn't a 100%
234 reliable way to find a related one out of hundreds of tests. One approach is
238 produces output to stdout and stderr, the other is the expected reference
247 There isn't a hard rule about how to write a test script, but a new test is
249 commonly used ways to create a test:
252 to the testing procedure, and could source a group of ``common.*`` libraries
258 harder to debug.
267 you opt for Python, it is strongly recommended to write Python 3 compatible
270 Both Python and Bash frameworks in iotests provide helpers to manage test
271 images. They can be used to create and clean up images under the test
273 more convenient to use the pseudo block driver, ``null-co://``, as the test
276 Otherwise, image locking implications have to be considered. For example,
277 another application on the host may have locked the file, possibly leading to a
279 ``locking=off`` option to disable image locking.
284 The following options to the ``check`` script can be useful when debugging
288 connection from a gdb client. The options given to ``gdbserver`` (e.g. the
289 address on which to listen for connections) are taken from the ``$GDB_OPTIONS``
292 It is possible to connect to it for example with
298 * ``-valgrind`` attaches a valgrind instance to QEMU. If it detects
307 * ``-p`` (print) redirects QEMU’s stdout and stderr to the test output,
314 "Tests may belong to one or more test groups, which are defined in the form
334 # ci - tests to run on build
355 much memory and disk space (since CI pipelines tend to fail otherwise).
367 The container testing framework in QEMU utilizes public images to
369 environments. This makes it possible to expand the test coverage
376 The container images are also used to augment the generation of tests
383 on your development machine, then make sure you have the privilege to run
394 The last command should print an empty table, to verify the system is ready.
396 An alternative method to set up permissions is by adding the current user to
398 ``/var/run/docker.sock``) accessible to the group:
406 Note that any one of above configurations makes it possible for the user to
420 The last command should print an empty table, to verify the system is ready.
425 From source tree, type ``make docker-help`` to see the help. Testing
443 used to pull in pre-built layers. This avoids unnecessary strain on
461 mainly used to do necessary host side setup. One such setup is ``binfmt_misc``,
462 for example, to make qemu-user powered cross build containers work.
466 This has led to an inconsistent set of packages being present across the
469 Thus going forward, QEMU is aiming to automatically generate the Dockerfiles
475 mappings to distribution package names for a wide variety of third
476 party projects. ``lcitool`` applies the mappings to a list of build
478 list of native packages to install on each distribution, and uses them
479 to generate build environments (dockerfiles and Cirrus CI variable files)
487 pre-requisite to QEMU, the prerequisites should to be added to
488 ``tests/lcitool/projects/qemu.yml`` in order to make the dependency
491 In the simple case where the pre-requisite is already known to ``libvirt-ci``
496 * Run ``make lcitool-refresh`` to re-generate all relevant build environment
501 first to contribute the mapping to the ``libvirt-ci`` project:
505 * Add an entry for the new build prerequisite to
511 files, and submit a merge request to the ``libvirt-ci`` project.
515 * CI pipeline will run to validate that the changes to ``mappings.yml``
516 are correct, by attempting to install the newly listed package on
519 * Once the merge request is accepted, go back to QEMU and update
520 the ``tests/lcitool/libvirt-ci`` submodule to point to a commit that
525 for as many OS distros as practical to make sure that there are no
528 page on how to trigger gitlab CI pipelines on your change.
530 For enterprise distros that default to old, end-of-life versions of the
541 desired to be tested. Before adding a new OS distro, discuss the proposed
544 * Send a mail to qemu-devel, copying people listed in the
547 There are limited CI compute resources available to QEMU, so the
548 cost/benefit tradeoff of adding new OS distros needs to be considered.
551 pointing to the qemu-devel mail thread in the archives.
554 to avoid duplication, as well as to get feedback from libvirt-ci
555 maintainers on any tips to ease the addition
557 Assuming there is agreement to add a new OS distro then
566 * Edit the ``lcitool/facts/mappings.yml`` change to add entries for
571 * Commit the changes to ``lcitool/facts`` and the regenerated test
572 files, and submit a merge request to the ``libvirt-ci`` project.
576 * CI pipeline will run to validate that the changes to ``mappings.yml``
577 are correct, by attempting to install the newly listed package on
580 * Once the merge request is accepted, go back to QEMU and update
581 the ``libvirt-ci`` submodule to point to a commit that contains
588 Different tests are added to cover various configurations to build and test
591 library, ``tests/docker/common.rc``, which provides helpers to find the QEMU
600 below steps to debug it:
604 2. Add "V=1" to the command line, try again, to see the verbose output.
605 3. Further add "DEBUG=1" to the command line. This will pause in a shell prompt
607 build QEMU and run tests from there, or press Ctrl-D to let the Docker
610 will hopefully run into the error again. After that, you will be dropped to
616 Various options can be used to affect how Docker tests are done. The full
619 * ``V=1``: the same as in top level ``make``. It will be propagated to the
622 similar to the ``-j $N`` option in top level ``make``. (The ``-j`` option in
649 We recommend using DEBUG=1 to allow launching the test from inside the docker,
650 and to allow review of the warnings generated by TSan.
655 It is possible to build and test with TSan, with a few additional steps.
662 instance) will not be recognized, and will lead to false positives.
664 To build a tsan version of glib:
673 To configure the build for TSan:
680 When executing qemu, don't forget to point to tsan glib:
684 $ glib_dir=/path/to/glib
701 export TSAN_OPTIONS=suppressions=<path to qemu>/tests/tsan/suppressions.tsan \
707 If you want TSan to stop and exit with error on warnings, use exitcode=66.
716 to fix the code if possible to eliminate the data race rather than suppress
721 tests/tsan/suppressions.tsan - Has TSan warnings we wish to suppress at runtime.
727 tests/tsan/ignore.tsan - Has TSan warnings we wish to disable
729 Add flags to configure to enable:
740 of the annotations themselves. Annotations can be used to suppress
748 Good files to start with are: annotate_happens_before.cpp and ignore_race.cpp
757 It is possible to combine Debian's bootstrap scripts with a configured
758 ``binfmt_misc`` to bootstrap a number of Debian's distros including
760 simplify setting up a rootfs by using docker to contain the foreign
766 You can use the script ``qemu-binfmt-conf.sh`` to configure a QEMU
767 user binary to automatically run binaries for the foreign
768 architecture. While the scripts will try their best to work with
772 executable on setup and avoids the need to find and re-open in the
778 For example to setup the HPPA ports builds of Debian::
787 ``debian-bootstrap.pre`` which is called to do the initial debootstrap
796 necessary packages to build QEMU. The basic usage is documented in ``Makefile``
802 Run ``make vm-help`` to list available make targets. Invoke a specific make
803 command to run build test in an image. For example, ``make vm-build-freebsd``
810 access, so they SHOULD NOT be exposed to external interfaces if you are
812 exploiting a QEMU security bug to compromise the host.
817 By default, ``qemu-system-x86_64`` is searched in $PATH to run the guest. If
819 provide the QEMU binary in env var: ``QEMU=/path/to/qemu-2.10+``.
821 Likewise the path to ``qemu-img`` can be set in QEMU_IMG environment variable.
827 specify ``J=$X`` to control the make jobs in the guest.
832 Add ``DEBUG=1`` and/or ``V=1`` to the make command to allow interactive
840 For example to work with the netbsd guest, use ``$QEMU_SRC/tests/vm/netbsd``:
846 # To bootstrap the image
850 # To run an arbitrary command in guest (the output will not be echoed unless
854 # To build QEMU in guest
857 # To get to an interactive shell
863 Please look at existing guest scripts for how to add new guests.
876 - Root password set to ``BaseVM.ROOT_PASS``
877 - User ``BaseVM.GUEST_USER`` is created, and password set to
880 ``$QEMU_SRC/tests/keys/id_rsa.pub`` is added to ssh's ``authorized_keys``
885 - Necessary packages are installed to untar the source tarball and build
896 An image fuzzer was added to exercise format drivers. Currently only qcow2 is
897 supported. To start the fuzzer, run
910 attempting to find errors like incorrect functions, interface errors,
928 linux-user and softmmu TCG functionality. However to build test
929 programs for guest targets you need to have cross compilers available.
943 compiler flags are needed to build for a given target.
945 If you have the ability to run containers as the user the build system
948 use the same container to build tests. However there are a number of
968 Adding ``V=1`` to the invocation will show the details of how to
990 compiler to work with additional libraries can be challenging.
1001 The KVM unit tests are designed to run as a Guest OS under KVM but
1012 kernel. It checks that syscalls behave as documented and strives to
1014 to run to exercise QEMU's linux-user code::
1021 ``gcov`` is a GCC tool to analyze the testing coverage by
1022 instrumenting the tested code. To use it, configure QEMU with
1025 If you want to gather coverage information on a single test the ``make
1026 clean-gcda`` target can be used to delete any existing coverage
1045 In QEMU, tests that are identified to be flaky are normally disabled by
1047 the tests to enable them.