virt/kvm/vcpu-requests.rst

44 ----------
46 The goal of a VCPU kick is to bring a VCPU thread out of guest mode in
48 a guest mode exit.  However, a VCPU thread may not be in guest mode at the
49 time of the kick.  Therefore, depending on the mode and state of the VCPU
53 1) Send an IPI.  This forces a guest mode exit.
55    mode that wait on waitqueues.  Waking them removes the threads from
58 3) Nothing.  When the VCPU is not in guest mode and the VCPU thread is not
61 VCPU Mode
62 ---------
64 VCPUs have a mode state, ``vcpu->mode``, that is used to track whether the
65 guest is running in guest mode or not, as well as some specific
66 outside guest mode states.  The architecture may use ``vcpu->mode`` to
74   The VCPU thread is outside guest mode.
78   The VCPU thread is in guest mode.
87   The VCPU thread is outside guest mode, but it wants the sender of
94 VCPU requests are simply bit indices of the ``vcpu->requests`` bitmap.
95 This means general bitops, like those documented in [atomic-ops]_ could
98   clear_bit(KVM_REQ_UNHALT & KVM_REQUEST_MASK, &vcpu->requests);
106 ---------------------------------
138 ----------------
146 ------------------
150   This flag is applied to requests that only need immediate attention
151   from VCPUs running in guest mode.  That is, sleeping VCPUs do not need
157   When requests with this flag are made with kvm_make_all_cpus_request(),
159   proceeding.  This flag only applies to VCPUs that would receive IPIs.
161   the requesting thread does not wait.  This means that this flag may be
176 scenario 3, Message and Flag, of [lwn-mb]_ and the kernel documentation
177 [memory-barriers]_.
187 executing in guest mode for an arbitrary long time without handling the
189 thread checks kvm_request_pending() before entering guest mode and that a
190 kick will send an IPI to force an exit from guest mode when necessary.
192 kvm_request_pending() check and before it has entered guest mode, as kick
193 IPIs will only trigger guest mode exits for VCPU threads that are in guest
194 mode or at least have already disabled interrupts in order to prepare to
195 enter guest mode.  This means that an optimized implementation (see "IPI
199 - set ``vcpu->mode`` to IN_GUEST_MODE between disabling the interrupts and
201 - enable interrupts atomically when entering the guest.
209 (scenario 10 of [lwn-mb]_).  As the Dekker pattern requires two variables,
210 this solution pairs ``vcpu->mode`` with ``vcpu->requests``.  Substituting
216   WRITE_ONCE(vcpu->mode, IN_GUEST_MODE);  kvm_make_request(REQ, vcpu);
218   if (kvm_request_pending(vcpu)) {        if (READ_ONCE(vcpu->mode) ==
223 As stated above, the IPI is only useful for VCPU threads in guest mode or
226 ``vcpu->mode`` to IN_GUEST_MODE.  WRITE_ONCE() and READ_ONCE() are used to
228 compiler doesn't interfere with ``vcpu->mode``'s carefully planned
232 -------------
236 sending kick also change the VCPU mode to something !IN_GUEST_MODE.  The
240 ----------------------------
242 Some requests, those with the KVM_REQUEST_WAIT flag set, require IPIs to
245 is when a target VCPU thread is in READING_SHADOW_PAGE_TABLES mode, which
247 KVM_REQUEST_WAIT flag changes the condition for sending an IPI from
251 Request-less VCPU Kicks
252 -----------------------
255 two-variable Dekker memory barrier pattern, then it's clear that
256 request-less VCPU kicks are almost never correct.  Without the assurance
257 that a non-IPI generating kick will still result in an action by the
259 request-accompanying kicks, then the kick may not do anything useful at
260 all.  If, for instance, a request-less kick was made to a VCPU that was
261 just about to set its mode to IN_GUEST_MODE, meaning no IPI is sent, then
266 even the request-less VCPU kick is coupled with the same
269 role of ``vcpu->requests``.  When sending a posted interrupt, PIR.ON is
270 set before reading ``vcpu->mode``; dually, in the VCPU thread,
271 vmx_sync_pir_to_irr() reads PIR after setting ``vcpu->mode`` to
278 --------------
288 -----------------
298 themselves.  A possible side-effect of that call is to make the
305 .. [atomic-ops] Documentation/core-api/atomic_ops.rst
306 .. [memory-barriers] Documentation/memory-barriers.txt
307 .. [lwn-mb] https://lwn.net/Articles/573436/