devel/testing/fuzzing.rst

5 This document describes the virtual-device fuzzing infrastructure in QEMU and
9 ------
15 To fuzz QEMU, we rely on libfuzzer. Unlike other fuzzers such as AFL, libfuzzer
16 is an *in-process* fuzzer. For the developer, this means that it is their
17 responsibility to ensure that state is reset between fuzzing-runs.
20 --------------------
24 Here, enable-asan and enable-ubsan are optional but they allow us to reliably
25 detect bugs such as out-of-bounds accesses, uses-after-free, double-frees
28     CC=clang-8 CXX=clang++-8 /path/to/configure \
29         --enable-fuzzing --enable-asan --enable-ubsan
31 Fuzz targets are built similarly to system targets::
33     make qemu-fuzz-i386
35 This builds ``./qemu-fuzz-i386``
37 The first option to this command is: ``--fuzz-target=FUZZ_NAME``
38 To list all of the available fuzzers run ``qemu-fuzz-i386`` with no arguments.
42     ./qemu-fuzz-i386 --fuzz-target=virtio-scsi-fuzz
44 Internally, libfuzzer parses all arguments that do not begin with ``"--"``.
45 Information about these is available by passing ``-help=1``
51 ----------------------
53 As mentioned above, libFuzzer accepts some arguments. Passing ``-help=1`` will
63 * ``-max_len=4096`` : specify the maximum byte-length of the inputs libFuzzer
66 * ``-close_fd_mask={1,2,3}`` : close, stderr, or both. Useful for targets that
69 * ``-jobs=4 -workers=4`` : These arguments configure libFuzzer to run 4 fuzzers in
71   ``-jobs=N``, libFuzzer automatically spawns a number of workers less than or equal
76 * ``-use_value_profile=1`` : For each comparison operation, libFuzzer computes
83 * ``-shrink=1`` : Tries to make elements of the corpus "smaller". Might lead to
90 ---------------------------
94 unique blocks/edges covered. To examine coverage on a line-by-line basis we
101     --enable-fuzzing \
102     --extra-cflags="-fprofile-instr-generate -fcoverage-mapping"
104  3. Re-run the fuzzer. Specify $CORPUS_DIR/* as an argument, telling libfuzzer
107  4. Execute these commands to generate a detailed HTML coverage-report::
109       llvm-profdata merge -output=default.profdata default.profraw
110       llvm-cov show ./path/to/qemu-fuzz-i386 -instr-profile=default.profdata \
111       --format html -output-dir=/path/to/output/report
114 -------------------
117 Fuzzers are kept in ``tests/qtest/fuzz/`` and should be added to
118 ``tests/qtest/fuzz/meson.build``
122 1. Create a new source file. For example ``tests/qtest/fuzz/foo-device-fuzz.c``.
127 3. Add the fuzzer to ``tests/qtest/fuzz/meson.build``.
129 Fuzzers can be more-or-less thought of as special qtest programs which can
132 fuzzer loops over the byte-array interpreting it as a list of qtest commands,
136 ------------------
138 Writing a fuzz target can be a lot of effort (especially if a device driver has
139 not be built-out within libqos). Many devices can be fuzzed to some degree,
140 without any device-specific code, using the generic-fuzz target.
142 The generic-fuzz target is capable of fuzzing devices over their PIO, MMIO,
143 and DMA input-spaces. To apply the generic-fuzz to a device, we need to define
144 two env-variables, at minimum:
147   the device attached. For example, if we want to fuzz the virtio-net device
148   attached to a pc-i440fx machine, we can specify::
150     QEMU_FUZZ_ARGS="-M pc -nodefaults -netdev user,id=user0 \
151     -device virtio-net,netdev=user0"
153 * ``QEMU_FUZZ_OBJECTS=`` is a set of space-delimited strings used to identify
157   virtio-net example, we could use one of ::
159     QEMU_FUZZ_OBJECTS='virtio-net'
161     QEMU_FUZZ_OBJECTS='virtio* pcspk' # Fuzz the virtio devices and the speaker
162     QEMU_FUZZ_OBJECTS='*' # Fuzz the whole machine``
164 The ``"info mtree"`` and ``"info qom-tree"`` monitor commands can be especially
168 As a generic rule-of-thumb, the more ``MemoryRegions``/Devices we match, the
169 greater the input-space, and the smaller the probability of finding crashing
175     ./qemu-fuzz-i386 --fuzz-target=generic-fuzz -runs=0
179 OSS-Fuzz  chapter
180 --------
181 QEMU is continuously fuzzed on `OSS-Fuzz
182 <https://github.com/google/oss-fuzz>`_.  By default, the OSS-Fuzz build
183 will try to fuzz every fuzz-target. Since the generic-fuzz target
185 pre-define some generic-fuzz configs in
186 ``tests/qtest/fuzz/generic_fuzz_configs.h``. Each config must specify:
188 - ``.name``: To identify the fuzzer config
190 - ``.args`` OR ``.argfunc``: A string or pointer to a function returning a
195 - ``.objects``: A string that specifies the ``QEMU_FUZZ_OBJECTS`` environment
198 To fuzz additional devices/device configuration on OSS-Fuzz, send patches for
199 either a new device-specific fuzzer or a new generic-fuzz config.
203 - The Dockerfile that sets up the environment for building QEMU's
204   fuzzers on OSS-Fuzz can be fund in the OSS-Fuzz repository
205   __(https://github.com/google/oss-fuzz/blob/master/projects/qemu/Dockerfile)
207 - The script responsible for building the fuzzers can be found in the
208   QEMU source tree at ``scripts/oss-fuzz/build.sh``
211 -----------------------------------------
213 can be used on a non-fuzzer build of QEMU. This filters out any potential
214 false-positives, and improves the debugging experience for developers.
216 generic-fuzz target.
218 - Ensure the crash reproduces::
220     qemu-fuzz-i386 --fuzz-target... ./crash-...
222 - Gather the QTest output for the crash::
225     qemu-fuzz-i386 --fuzz-target... ./crash-... &> /tmp/trace
227 - Reorder and clean-up the resulting trace::
229     scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py /tmp/trace > /tmp/reproducer
231 - Get the arguments needed to start qemu, and provide a path to qemu::
234     export QEMU_ARGS="-machine ..."
235     export QEMU_PATH="path/to/qemu-system"
237 - Ensure the crash reproduces in qemu-system::
239     $QEMU_PATH $QEMU_ARGS -qtest stdio < /tmp/reproducer
241 - From the crash output, obtain some string that identifies the crash. This
242   can be a line in the stack-trace, for example::
244     export CRASH_TOKEN="hw/usb/hcd-xhci.c:1865"
246 - Minimize the reproducer::
248     scripts/oss-fuzz/minimize_qtest_trace.py -M1 -M2 \
249       /tmp/reproducer /tmp/reproducer-minimized
251 - Confirm that the minimized reproducer still crashes::
253     $QEMU_PATH $QEMU_ARGS -qtest stdio < /tmp/reproducer-minimized
255 - Create a one-liner reproducer that can be sent over email::
257     ./scripts/oss-fuzz/output_reproducer.py -bash /tmp/reproducer-minimized
259 - Output the C source code for a test case that will reproduce the bug::
261     ./scripts/oss-fuzz/output_reproducer.py -owner "John Smith <john@smith.com>"\
262       -name "test_function_name" /tmp/reproducer-minimized
264 - Report the bug and send a patch with the C reproducer upstream
267 -----------------------------------------
282 select the fuzz target. Then, the qtest client is initialized. If the target
287 target-specific hooks that can be called before and after main, for
290 ``LLVMFuzzerTestOneInput``: Uses qtest/qos functions to act based on the fuzz
298   - *Pros*: Straightforward and fast for simple fuzz targets.
300   - *Cons*: Depending on the device, does not reset all device state. If the
302     for QOS-based targets), this initialization needs to be done after each
305   - *Example target*: ``i440fx-qtest-reboot-fuzz``