vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name i

vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name is nul-terminated, the vmmctl ioctl
handler and the legacy sysctl handler ensure this. So, don't bother
with strnlen().

Finally, make sure that the name buffers are the same size on all
platforms. VM_MAX_NAMELEN is supposed to be the maximum, not including
the nul terminator.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53422

show more ...

vmm: Add support for guest NUMA emulation

This change adds the necessary kernelspace bits required for
supporting NUMA domains in bhyve VMs.

The layout of system memory segments and how they're cre

vmm: Add support for guest NUMA emulation

This change adds the necessary kernelspace bits required for
supporting NUMA domains in bhyve VMs.

The layout of system memory segments and how they're created has
been reworked. Each guest NUMA domain will now have its own memory
segment. Furthermore, this change allows users to tweak the domain's
backing vm_object domainset(9) policy.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D44565

show more ...

Remove stray whitespaces from sys/amd64/

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1418

vmm: Make vmm_dev.h more self-contained

vmm.h is required for VM_MAX_SUFFIXLEN. vmm_snapshot.h is required for
struct vm_snapshot_meta.

This is a prerequisite for including vmm_dev.h in the header

vmm: Make vmm_dev.h more self-contained

vmm.h is required for VM_MAX_SUFFIXLEN. vmm_snapshot.h is required for
struct vm_snapshot_meta.

This is a prerequisite for including vmm_dev.h in the headers parsed by
libsysdecode.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D46485

show more ...

vmm: Move compat ioctl definitions to vmm_dev.c

There is no reason to keep them in vmm_dev.h. No functional change
intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebs

vmm: Move compat ioctl definitions to vmm_dev.c

There is no reason to keep them in vmm_dev.h. No functional change
intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46432

show more ...

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vm

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vmm/vmm_dev.c. This will make it
easier to introduce a cdev-based interface for VM creation, which in
turn makes it possible to implement support for running bhyve as an
unprivileged user.

Machine-dependent ioctls continue to be handled in machine-dependent
code. To make the split a bit easier to handle, introduce a pair of
tables which define MI and MD ioctls. Each table entry can set flags
which determine which locks need to be held in order to execute the
handler. vmmdev_ioctl() now looks up the ioctl in one of the tables,
acquires locks and either handles the ioctl directly or calls
vmmdev_machdep_ioctl() to handle it.

No functional change intended. There is a lot of churn in this change
but the underlying logic in the ioctl handlers is the same. For now,
vmm_dev.h is still mostly separate, even though some parts could be
merged in principle. This would involve changing include paths for
userspace, though.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46431

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise have some leeway to change the cpuset_t for the whole
system, but we want to keep the vmm ioctl ABI stable.

Rework IPI reporting to avoid this problem. Along the way, make VM_RUN
a bit more efficient:
- Split vmexit metadata out of the main VM_RUN structure. This data is
only written by the kernel.
- Have userspace pass a cpuset_t pointer and cpusetsize in the VM_RUN
structure, as is done for cpuset syscalls.
- Have the destination CPU mask for VM_EXITCODE_IPIs live outside the
vmexit info structure, and make VM_RUN copy it out separately. Zero
out any extra bytes in the CPU mask, like cpuset syscalls do.
- Modify the vmexit handler prototype to take a full VM_RUN structure.

PR: 271330
Reviewed by: corvink, jhb (previous versions)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40113

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

vmm: Don't lock a vCPU for VM_PPTDEV_MSI[X].

These are manipulating state in a ppt(4) device none of which is
vCPU-specific. Mark the vcpu fields in the relevant ioctl structures
as unused, but don

vmm: Don't lock a vCPU for VM_PPTDEV_MSI[X].

These are manipulating state in a ppt(4) device none of which is
vCPU-specific. Mark the vcpu fields in the relevant ioctl structures
as unused, but don't remove them for now.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37639

show more ...

Extend the VMM stats interface to support a dynamic count of statistics.

- Add a starting index to 'struct vmstats' and change the
VM_STATS ioctl to fetch the 64 stats starting at that index.
A

Extend the VMM stats interface to support a dynamic count of statistics.

- Add a starting index to 'struct vmstats' and change the
VM_STATS ioctl to fetch the 64 stats starting at that index.
A compat shim for <= 13 continues to fetch only the first 64
stats.

- Extend vm_get_stats() in libvmmapi to use a loop and a static
thread local buffer which grows to hold the stats needed.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D27463

show more ...

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs is properly handled by the PCI emulation
code in general, but a few specific cases need additional support.
fbuf and passthru map additional objects into the guest physical
address space and so need to handle address updates. Here we add a
callback to emulated PCI devices to inform them of a BAR
configuration change. fbuf and passthru then watch for these BAR
changes and relocate the frame buffer memory segment and passthru
device mmio area respectively.

We also add new VM_MUNMAP_MEMSEG and VM_UNMAP_PPTDEV_MMIO ioctls
to vmm(4) to facilitate the unmapping needed for addres updates.

[1]: https://github.com/freebsd/uefi-edk2/pull/9/

Originally by: scottph
MFC After: 1 week
Sponsored by: Intel Corporation
Reviewed by: grehan
Approved by: philip (mentor)
Differential Revision: https://reviews.freebsd.org/D24066

show more ...

Honor the disabled setting for MSI-X interrupts for passthrough devices.

Add a new ioctl to disable all MSI-X interrupts for a PCI passthrough
device and invoke it if a write to the MSI-X capability

Honor the disabled setting for MSI-X interrupts for passthrough devices.

Add a new ioctl to disable all MSI-X interrupts for a PCI passthrough
device and invoke it if a write to the MSI-X capability registers
disables MSI-X. This avoids leaving MSI-X interrupts enabled on the
host if a guest device driver has disabled them (e.g. as part of
detaching a guest device driver).

This was found by Chelsio QA when testing that a Linux guest could
switch from MSI-X to MSI interrupts when using the cxgb4vf driver.

While here, explicitly fail requests to enable MSI on a passthrough
device if MSI-X is enabled and vice versa.

Reported by: Sony Arpita Das @ Chelsio
Reviewed by: grehan, markj
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27212

show more ...

vmm(4): Add 12 user ABI compat after r349948

Reported by: kp
Reviewed by: jhb, kp
Tested by: kp
Differential Revision: https://reviews.freebsd.org/D24929

vmm(4), bhyve(8): Expose kernel-emulated special devices to userspace

Expose the special kernel LAPIC, IOAPIC, and HPET devices to userspace
for use in, e.g., fallback instruction emulation (when us

vmm(4), bhyve(8): Expose kernel-emulated special devices to userspace

Expose the special kernel LAPIC, IOAPIC, and HPET devices to userspace
for use in, e.g., fallback instruction emulation (when userspace has a
newer instruction decode/emulation layer than the kernel vmm(4)).

Plumb the ioctl through libvmmapi and register the memory ranges in
bhyve(8).

Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D24525

show more ...

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implement

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implementation, bhyve(8) creates a UNIX domain socket that is
used by bhyvectl(8) to send a request to save a snapshot (and
optionally exit after the snapshot has been taken). A snapshot
currently consists of two files: the first holds a copy of guest RAM,
and the second file holds other guest state such as vCPU register
values and device model state.

To resume a guest, bhyve(8) must be started with a matching pair of
command line arguments to instantiate the same set of device models as
well as a pointer to the saved snapshot.

While the current implementation is useful for several uses cases, it
has a few limitations. The file format for saving the guest state is
tied to the ABI of internal bhyve structures and is not
self-describing (in that it does not communicate the set of device
models present in the system). In addition, the state saved for some
device models closely matches the internal data structures which might
prove a challenge for compatibility of snapshot files across a range
of bhyve versions. The file format also does not currently support
versioning of individual chunks of state. As a result, the current
file format is not a fixed binary format and future revisions to save
and restore will break binary compatiblity of snapshot files. The
goal is to move to a more flexible format that adds versioning,
etc. and at that point to commit to providing a reasonable level of
compatibility. As a result, the current implementation is not enabled
by default. It can be enabled via the WITH_BHYVE_SNAPSHOT=yes option
for userland builds, and the kernel option BHYVE_SHAPSHOT.

Submitted by: Mihai Tiganus, Flavius Anton, Darius Mihai
Submitted by: Elena Mihailescu, Mihai Carabas, Sergiu Weisz
Relnotes: yes
Sponsored by: University Politehnica of Bucharest
Sponsored by: Matthew Grooms (student scholarships)
Sponsored by: iXsystems
Differential Revision: https://reviews.freebsd.org/D19495

show more ...

MFHead @r350386

Sponsored by: The FreeBSD Foundation

Tie the name limit of a VM to SPECNAMELEN from devfs instead of a
hard-coded value. Don't allocate space for it from the kernel stack.
Account for prefix, suffix, and separator space in the name. Thi

Tie the name limit of a VM to SPECNAMELEN from devfs instead of a
hard-coded value. Don't allocate space for it from the kernel stack.
Account for prefix, suffix, and separator space in the name. This
takes the effective length up to 229 bytes on 13-current, and 37 bytes
on 12-stable. 37 bytes is enough to hold a full GUID string.

PR: 234134
MFC after: 1 week
Differential Revision: http://reviews.freebsd.org/D20924

show more ...

vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name i

vmm: Consolidate VM name length checking

vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().

We can safely assume that the name is nul-terminated, the vmmctl ioctl
handler and the legacy sysctl handler ensure this. So, don't bother
with strnlen().

Finally, make sure that the name buffers are the same size on all
platforms. VM_MAX_NAMELEN is supposed to be the maximum, not including
the nul terminator.

Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53422

show more ...

vmm: Add support for guest NUMA emulation

This change adds the necessary kernelspace bits required for
supporting NUMA domains in bhyve VMs.

The layout of system memory segments and how they're cre

vmm: Add support for guest NUMA emulation

This change adds the necessary kernelspace bits required for
supporting NUMA domains in bhyve VMs.

The layout of system memory segments and how they're created has
been reworked. Each guest NUMA domain will now have its own memory
segment. Furthermore, this change allows users to tweak the domain's
backing vm_object domainset(9) policy.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D44565

show more ...

Remove stray whitespaces from sys/amd64/

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1418

vmm: Make vmm_dev.h more self-contained

vmm.h is required for VM_MAX_SUFFIXLEN. vmm_snapshot.h is required for
struct vm_snapshot_meta.

This is a prerequisite for including vmm_dev.h in the header

vmm: Make vmm_dev.h more self-contained

vmm.h is required for VM_MAX_SUFFIXLEN. vmm_snapshot.h is required for
struct vm_snapshot_meta.

This is a prerequisite for including vmm_dev.h in the headers parsed by
libsysdecode.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D46485

show more ...

vmm: Move compat ioctl definitions to vmm_dev.c

There is no reason to keep them in vmm_dev.h. No functional change
intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebs

vmm: Move compat ioctl definitions to vmm_dev.c

There is no reason to keep them in vmm_dev.h. No functional change
intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46432

show more ...

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vm

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vmm/vmm_dev.c. This will make it
easier to introduce a cdev-based interface for VM creation, which in
turn makes it possible to implement support for running bhyve as an
unprivileged user.

Machine-dependent ioctls continue to be handled in machine-dependent
code. To make the split a bit easier to handle, introduce a pair of
tables which define MI and MD ioctls. Each table entry can set flags
which determine which locks need to be held in order to execute the
handler. vmmdev_ioctl() now looks up the ioctl in one of the tables,
acquires locks and either handles the ioctl directly or calls
vmmdev_machdep_ioctl() to handle it.

No functional change intended. There is a lot of churn in this change
but the underlying logic in the ioctl handlers is the same. For now,
vmm_dev.h is still mostly separate, even though some parts could be
merged in principle. This would involve changing include paths for
userspace, though.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46431

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/