docs/devel/replay.rst

12 Record/replay functions are used for the deterministic replay of qemu
15 number of times. Execution replaying reads the log and replays all
19 Several parts of QEMU include function calls to make event log recording
28 these moments by counting the number of instructions executed between
29 every pair of consecutive events.
31 Academic papers with description of deterministic replay implementation:
33 * `Deterministic Replay of System's Execution with Multi-target QEMU Simulator for Dynamic Analysis…
34 * `Don't panic: reverse debugging of kernel drivers <https://dl.acm.org/citation.cfm?id=2786805.280…
36 Modifications of qemu include:
40  * synchronization of the bottom halves execution
41  * synchronization of the threads from thread pool
47  * recording of random numbers obtained from the external sources
53 designed to allow deterministic execution in absence of external inputs
54 of the virtual machine. We also use icount to control the occurrence of the
55 non-deterministic events. The number of instructions elapsed from the last event
62 Previously the synchronisation of the main thread and the vCPU thread
63 was ensured by the holding of the BQL. However the trend has been to
65 system emulation. As it is important that batches of events are kept
74 certainly a bug, and a source of deadlocks, to take the
84 Replaying the execution of virtual machine is bound by sources of
89 Invocations of timers are coupled with clock reads and changing the state
90 of the virtual machine. Reads produce non-deterministic data taken from
92 order in replay mode must replicate the order of callbacks in record mode.
99 of time. That's why we do not process a group of timers until the checkpoint
103 Two other checkpoints govern the "warping" of the virtual clock.
107 timer fires at the next deadline of the virtual clock; the virtual clock
109 soon as the timer fires or the CPUs need to go out of the idle state.
111 virtual machine state and must be deterministic, each of them creates a
115 and it warps the virtual clock by the amount of real time that has passed
129 Timers are used to execute callbacks from different subsystems of QEMU
130 at the specified moments of time. There are several kinds of timers:
139    sources (e.g. real time clock chip). Host clock is the one of the sources
140    of non-determinism. Host clock read operations should be logged to
161 Block devices record/replay module (``blkreplay``) intercepts calls of
162 bdrv coroutine functions at the top of block drivers stack.
181 writing the disk may occur at different moments of time in record and replay
184 Reading and writing requests are created by CPU thread of QEMU. Later these
186 halves consist of callback and its parameters. They are processed when
211 Avoid accessing other devices' state, because the order of saving/restoring
220 of the network connections, that could be broken by the remote timeouts).
221 VM can be stopped at any moment of replay by the user. Restarting the VM
227 Record/replay log consists of the header and the sequence of execution
230 using replay log created by another build of qemu.
232 The sequence of the events describes virtual machine state changes.
233 It includes all non-deterministic inputs of VM, synchronization marks and
243 and corresponding number of bytes with data.
244 Here is the list of events that are written into the log:
248    - 4-byte number of executed instructions.
252  - EVENT_ASYNC. This is a group of events. When such an event is generated,
263        parameters of keyboard and mouse input operations
266         - 9-16 bytes depending of input event.
303  - EVENT_CLOCK + clock_id. Group of events for host clock read operations. Followed by:
307  - EVENT_CHECKPOINT + checkpoint_id. Checkpoint for synchronization of