devel/testing/functional.rst

1 .. _checkfunctional-ref:
3 Functional testing with Python
6 The ``tests/functional`` directory hosts functional tests written in
11 using stdio for the TAP protocol. The folder ``tests/functional/qemu_test``
19 that there is no TAP-incompatible output written to stdio, e.g. either by
40 -------------
42 You can run the functional tests simply by executing:
46   make check-functional
53   make check-functional-x86_64
57 the PYTHONPATH that has to include the python folder and the tests/functional
62   $ export PYTHONPATH=../python:../tests/functional
63   $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
64   $ pyvenv/bin/python3 ../tests/functional/test_file.py
73 -------
78  <builddir>/tests/functional/<arch>/<fileid>.<classid>.<testname>/
91 -----------------------------
93 The ``tests/functional/qemu_test`` directory provides the ``qemu_test``
107           res = self.vm.cmd('human-monitor-command',
116 can also be run stand-alone, without a test runner. OTOH when run via a test
154               'human-monitor-command',
158               'human-monitor-command',
162               'human-monitor-command',
171 -----------
174 device-crash-test and other Python scripts.  It's a wrapper around the
202 to a UNIX socket name. The ``qmp-shell`` command can then be
206 -------------------
239 qemu-system-* binaries.
263 --------------
265 Many functional tests download assets (e.g. Linux kernels, initrds,
274 fine for functional tests that can be run without downloading files.
275 ``make check`` then only runs the quick functional tests along with
277 run only ``make check-functional``, the "thorough" tests will be
278 executed, too. And to run all functional tests along with the others,
281   make -j$(nproc) check SPEED=thorough
288 the tests are run. This pre-caching is done with the qemu_test.Asset
304 The pre-caching will be done automatically when running
305 ``make check-functional`` (but not when running e.g.
306 ``make check-functional-<target>``). In case you just want to download
309     make precache-functional
316 --------------
324   https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures
363   # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn