vmm: Move common accessors and vm_eventinfo into sys/dev/vmm

Now that struct vm and struct vcpu are defined in headers, provide
inline accessors. We could just remove the accessors outright, but th

vmm: Move common accessors and vm_eventinfo into sys/dev/vmm

Now that struct vm and struct vcpu are defined in headers, provide
inline accessors. We could just remove the accessors outright, but they
don't hurt and it would result in unneeded churn.

As a part of this, consolidate definitions related to struct
vm_eventinfo as well. I'm not sure if struct vm_eventinfo is really
needed anymore, now that vmmops_run implementations can directly access
vm and vcpu fields, but this can be resolved later.

No functional change intended.

MFC after: 2 months
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53586

show more ...

vmm: Deduplicate VM and vCPU state management code

Now that the machine-independent fields of struct vm and struct vcpu are
available in a header, we can move lots of duplicated code into
sys/dev/vm

vmm: Deduplicate VM and vCPU state management code

Now that the machine-independent fields of struct vm and struct vcpu are
available in a header, we can move lots of duplicated code into
sys/dev/vmm/vmm_vm.c. This change does exactly that.

No functional change intended.

MFC after: 2 months
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53585

show more ...

vmm: Consolidate vm and vcpu definitions

There is quite a lot of duplication of code between amd64, arm64 and
riscv with respect to VM and vCPU state management. This is a bit
tricky to resolve sin

vmm: Consolidate vm and vcpu definitions

There is quite a lot of duplication of code between amd64, arm64 and
riscv with respect to VM and vCPU state management. This is a bit
tricky to resolve since struct vm and struct vcpu are private to vmm.c
and both structures contain a mix of machine-dependent and
machine-independent fields.

To allow deduplication without also introducing a lot of churn, follow
the approach of struct pcpu and 1) lift the definitions of those
structures into a new header, sys/dev/vmm/vmm_vm.h, and 2) define
machine-dependent macros, VMM_VM_MD_FIELDS and VMM_VCPU_MD_FIELDS which
lay out the machine-dependent fields.

One disadvantage of this approach is that the two structures are no
longer private to vmm.c, but I think this is acceptable.

No functional change intended. A follow-up change will move a good deal
of machine/vmm/vmm.c into sys/dev/vmm/vmm_vm.c.

MFC after: 2 months
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53584

show more ...

vmm: Move vm_maxcpu handling into MI code

No functional change intended.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision

vmm: Move vm_maxcpu handling into MI code

No functional change intended.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53477

show more ...

vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name i

vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name is nul-terminated, the vmmctl ioctl
handler and the legacy sysctl handler ensure this. So, don't bother
with strnlen().

Finally, make sure that the name buffers are the same size on all
platforms. VM_MAX_NAMELEN is supposed to be the maximum, not including
the nul terminator.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53422

show more ...

amd64/vmm: Factor vcpu_notify_event() into two functions

vcpu_notify_event() previously took a boolean parameter which determines
whether the implementation should try to use a posted interrupt. On

amd64/vmm: Factor vcpu_notify_event() into two functions

vcpu_notify_event() previously took a boolean parameter which determines
whether the implementation should try to use a posted interrupt. On
arm64 and riscv, the implementation of vcpu_notify_event() is otherwise
identical to that of amd64.

With the aim of deduplicating vcpu state management code, introduce a
separate amd64-only function which tries to use posted interrupts. This
requires some duplication with vcpu_notify_event_locked(), but only a
little bit.

Then, fix up callers. No functional change intended.

Reviewed by: corvink, jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53419

show more ...

vmm: Fix a deadlock between vm_smp_rendezvous() and vcpu_lock_all()

vm_smp_rendezvous() invokes a callback on all vCPUs, blocking the
initiator until all vCPUs have responded. vcpu_lock_all() block

vmm: Fix a deadlock between vm_smp_rendezvous() and vcpu_lock_all()

vm_smp_rendezvous() invokes a callback on all vCPUs, blocking the
initiator until all vCPUs have responded. vcpu_lock_all() blocks each
vCPU by waiting for it to go idle and setting the vCPU state to frozen.
These two operations can deadlock on each other, particularly when
booting a Windows guest, when vcpu_lock_all() blocks waiting for a
rendezvous initiator, and the initiator is blocked waiting for the vCPU
thread which called vcpu_lock_all() to invoke the rendezvous callback.

Implement vcpu_lock_all() in a way that avoids deadlocks with
vm_smp_rendezvous(). In particular, when traversing vCPUs, invoke the
rendezvous callback on the vCPU's behalf to help the initiator finish.
We can only safely do so when the vCPU is IDLE or we have already locked
it, otherwise we may be racing with the target vCPU thread. Thus:
- Use an exclusive lock to serialize vcpu_lock_all() callers, which lets
us lock vCPUs out of order without fear of deadlock with parallel
vcpu_lock_all() callers.
- If a rendezvous is pending, lock all idle vCPUs and invoke the
callback on their behalf. If the vcpu_lock_all() caller is itself a
vCPU thread, this will handle that thread.
- Block waiting for all non-idle vCPUs to idle, or until one of them
initiates a rendezvous, in which case we go back and invoke callbacks
on behalf of already-locked vCPUs.

Note that on !amd64 no changes are needed since there is no rendezvous
mechanism, so there is a separate vcpu_set_state_all() for them based on
the previous vcpu_lock_all(). These will be merged together once vcpu
state handling is consolidated into sys/dev/vmm.

Reviewed by: corvink (previous version)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D52968

show more ...

vmm: Move the guest vmspace into the generic vm_mem structure

This further consolidates handling of guest memory into MI code in
sys/dev/vmm.

No functional change intended.

Reviewed by: corvink
MF

vmm: Move the guest vmspace into the generic vm_mem structure

This further consolidates handling of guest memory into MI code in
sys/dev/vmm.

No functional change intended.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53012

show more ...

vmm: Make vmmops declarations more consistent

- On amd64, make vmmops_* functions globally visible, as some will be
called from machine-independent code in the future.
- On arm64 and riscv, move d

vmm: Make vmmops declarations more consistent

- On amd64, make vmmops_* functions globally visible, as some will be
called from machine-independent code in the future.
- On arm64 and riscv, move declarations to vmm.h, since they're supposed
to be generic across different VMM backends (only amd64 has more than
one backend).
- Make the declaration macros consistent with each other.
- On amd64, make the function typedef names consistent with the
corresponding ifunc names.

No functional change intended.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53011

show more ...

vmm: Suspend the VM before destroying it

Otherwise we don't do anything to kick vcpu threads out of a sleep
state when destroying a VM. For instance, suppose a guest executes hlt
on amd64 or wfi on

vmm: Suspend the VM before destroying it

Otherwise we don't do anything to kick vcpu threads out of a sleep
state when destroying a VM. For instance, suppose a guest executes hlt
on amd64 or wfi on arm64 with interrupts disabled. Then,
bhyvectl --destroy will hang until the vcpu thread somehow comes out of
vm_handle_hlt()/vm_handle_wfi() since destroy_dev() is waiting for vCPU
threads to drain.

Note that on amd64, if hw.vmm.halt_detection is set to 1 (the default),
the guest will automatically exit in this case since it's treated as a
shutdown. But, the above should not hang if halt_detection is set to 0.

Here, vm_suspend() wakes up vcpu threads, and a subsequent attempt to
run the vCPU will result in an error which gets propagated to userspace,
allowing destroy_dev() to proceed.

Add a new suspend code for this purpose. Modify bhyve to exit with
status 4 ("exited due to an error") when it's received, since that's
what'll happen generally when the VM is destroyed asynchronously.

Reported by: def
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D51761

show more ...

vmm: Support INOUT manual decode.

The inout instruction in AMD SVM requires DecodeAssist feature to decode the
segment override prefix. However, without that feature, we are still
able to decode by

vmm: Support INOUT manual decode.

The inout instruction in AMD SVM requires DecodeAssist feature to decode the
segment override prefix. However, without that feature, we are still
able to decode by fetching the instruction directly.

Approved by: markj (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51256

show more ...

vmm: Consolidate code which manages guest memory regions

On all three platforms supported by vmm, we have mostly duplicated code
to manage guest physical memory regions. Deduplicate much of this co

vmm: Consolidate code which manages guest memory regions

On all three platforms supported by vmm, we have mostly duplicated code
to manage guest physical memory regions. Deduplicate much of this code
and move it into sys/dev/vmm/vmm_mem.c.

To avoid exporting struct vm outside of machdep vmm.c, add a new
struct vm_mem to contain the memory segment descriptors, and add a
vm_mem() accessor, akin to vm_vmspace(). This way vmm_mem.c can
implement its routines without needing to see the layout of struct vm.

The handling of the per-VM vmspace is also duplicated but will be moved
to vmm_mem.c in a follow-up patch.

On amd64, move the ppt_is_mmio() check out of vm_mem_allocated() to keep
the code MI, as PPT is only implemented on amd64. There are only a
couple of callers, so this is not unreasonable.

No functional change intended.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D48270

show more ...

amd64/vmm: Rename vm_get_vmspace() to vm_vmspace()

For consistency with other vm accessors.

No functional change intended.

Reviewed by: corvink
Differential Revision: https://reviews.freebsd.org/D

amd64/vmm: Rename vm_get_vmspace() to vm_vmspace()

For consistency with other vm accessors.

No functional change intended.

Reviewed by: corvink
Differential Revision: https://reviews.freebsd.org/D48268

show more ...

vmm: Correctly suspend and resume the vmm driver.

Previously, VMXON would be executed on a resume, contrary to proper
initalization. The contents of MSR_IA32_FEATURE_CONTROL may be lost on
suspensio

vmm: Correctly suspend and resume the vmm driver.

Previously, VMXON would be executed on a resume, contrary to proper
initalization. The contents of MSR_IA32_FEATURE_CONTROL may be lost on
suspension, therefore must be restored. Likewise, the VMX Enable bit may be
cleared upon suspend, requiring it to be re-set.

Concretely disable VMX on suspend, and re-enable it on resume.

Note: any IOMMU context will remain lost for any enabled vmm devices.

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: jhb,imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1419

show more ...

Remove stray whitespaces from sys/amd64/

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1418

amd64/vmm: Make vmm.h more self-contained

CTASSERT is defined in kassert.h, so include that here. No functional
change intended.

MFC after: 1 week

vmm: Fix typo

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/885

vmm: Expose more registers to VM_GET_REGISTER

In a follow-up revision the gdb stub will support sending an XML target
description to gdb, which lets us send additional registers, including
the ones

vmm: Expose more registers to VM_GET_REGISTER

In a follow-up revision the gdb stub will support sending an XML target
description to gdb, which lets us send additional registers, including
the ones added in this patch.

Reviewed by: jhb
MFC after: 1 month
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D43665

show more ...

vmm.h: remove dup declaration

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43139

vmm: implement single-stepping for AMD CPUs

This patch implements single-stepping for AMD CPUs using the RFLAGS.TF
single-stepping mechanism. The GDB stub requests single-stepping
using the VM_CAP_

vmm: implement single-stepping for AMD CPUs

This patch implements single-stepping for AMD CPUs using the RFLAGS.TF
single-stepping mechanism. The GDB stub requests single-stepping
using the VM_CAP_RFLAGS_TF capability. Setting this capability will
set the RFLAGS.TF bit on the selected vCPU, activate DB exception
intercepts, and activate POPF/PUSH instruction intercepts. The
resulting DB exception is then caught by the IDT_DB vmexit handler and
bounced to userland where it is processed by the GDB stub. This patch
also makes sure that the value of the TF bit is correctly updated and
that it is not erroneously propagated into memory. Stepping over PUSHF
will cause the vm_handle_db function to correct the pushed RFLAGS
value and stepping over POPF will update the shadowed TF bit copy.

Reviewed by: jhb
Sponsored by: Google, Inc. (GSoC 2022)
Differential Revision: https://reviews.freebsd.org/D42296

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise have some leeway to change the cpuset_t for the whole
system, but we want to keep the vmm ioctl ABI stable.

Rework IPI reporting to avoid this problem. Along the way, make VM_RUN
a bit more efficient:
- Split vmexit metadata out of the main VM_RUN structure. This data is
only written by the kernel.
- Have userspace pass a cpuset_t pointer and cpusetsize in the VM_RUN
structure, as is done for cpuset syscalls.
- Have the destination CPU mask for VM_EXITCODE_IPIs live outside the
vmexit info structure, and make VM_RUN copy it out separately. Zero
out any extra bytes in the CPU mask, like cpuset syscalls do.
- Modify the vmexit handler prototype to take a full VM_RUN structure.

PR: 271330
Reviewed by: corvink, jhb (previous versions)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40113

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

bhyve: fix vCPU single-stepping on VMX

This patch fixes virtual machine single stepping on VMX hosts.

Currently, when using bhyve's gdb stub, each attempt at single-stepping
a vCPU lands in a timer

bhyve: fix vCPU single-stepping on VMX

This patch fixes virtual machine single stepping on VMX hosts.

Currently, when using bhyve's gdb stub, each attempt at single-stepping
a vCPU lands in a timer interrupt. The current single-stepping mechanism
uses the Monitor Trap Flag feature to cause VMEXIT after a single
instruction is executed. Unfortunately, the SDM states that MTF causes
VMEXITs for the next instruction that gets executed, which is often not
what the person using the debugger expects. [1]

This patch adds a new VM capability that masks interrupts on a vCPU by
blocking interrupt injection and modifies the gdb stub to use the newly
added capability while single-stepping a vCPU.

[1] Intel SDM 26.5.2 Vol. 3C

Reviewed by: corvink, jbh
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39949

show more ...

libvmmapi: Add a struct vcpu and use it in most APIs.

This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.

s

libvmmapi: Add a struct vcpu and use it in most APIs.

This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.

struct vcpu is an opaque type managed by libvmmapi. For now it stores
a pointer to the VM context and an integer id.

As an immediate effect this removes the divergence between the kernel
and userland for the instruction emulation code introduced by the
recent vmm(4) changes.

Since this is a major change to the vmmapi API, bump VMMAPI_VERSION to
0x200 (2.0) and the shared library major version.

While here (and since the major version is bumped), remove unused
vcpu argument from vm_setup_pptdev_msi*().

Add new functions vm_suspend_all_cpus() and vm_resume_all_cpus() for
use by the debug server. The underyling ioctl (which uses a vcpuid of
-1) remains unchanged, but the userlevel API now uses separate
functions for global CPU suspend/resume.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38124

show more ...