release: Enable installing packages as a non-root user

Provide variables, PKG_REPOS_DIR and PKG_REPO_NAME, to allow the user to
configure the source package repository. Configure pkg to emit added

release: Enable installing packages as a non-root user

Provide variables, PKG_REPOS_DIR and PKG_REPO_NAME, to allow the user to
configure the source package repository. Configure pkg to emit added
files to a metalog which gets concatenated with the main metalog
immediately before we build the root filesystem.

Reviewed by: cperciva, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52453

show more ...

release: Create /firstboot in common VM image creation code

Some services, such as growfs, only run upon the first boot of an image.
The first boot is indicated by the presence of the file /firstboo

release: Create /firstboot in common VM image creation code

Some services, such as growfs, only run upon the first boot of an image.
The first boot is indicated by the presence of the file /firstboot,
which is unlinked after boot.

Individual cloudware types shouldn't be responsible for creating it. Do
so in a centralized place. Aside from simplifying things, this ensures
that we create a metalog entry for the firstboot file.

Reviewed by: cperciva, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52450

show more ...

release: Made it possible to specify more attributes in metalog entries

Verify that the target file or directory exists, and add support for an
optional mode. This is required to enable NO_ROOT bui

release: Made it possible to specify more attributes in metalog entries

Verify that the target file or directory exists, and add support for an
optional mode. This is required to enable NO_ROOT builds of some
cloudware image types. No functional change intended.

Reviewed by: cperciva, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52449

show more ...

Update vmimage.subr

This "noatime" tweak, reduced the OPNsense constant 50kb - 80kb disk writes that wear down the SSD / NVME.

Signed-off-by: Unicorn9x
Reviewed by: imp
Pull Request: https://github

Update vmimage.subr

This "noatime" tweak, reduced the OPNsense constant 50kb - 80kb disk writes that wear down the SSD / NVME.

Signed-off-by: Unicorn9x
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1834

show more ...

release: No newaliases when cross-building VMs

We've been running newaliases(8) when cross-building VMs for a long
time, and it's not entirely clear why -- especially since we don't
do it when we're

release: No newaliases when cross-building VMs

We've been running newaliases(8) when cross-building VMs for a long
time, and it's not entirely clear why -- especially since we don't
do it when we're building a native VM image.

If someone knows why running newaliases is important we can add it
back later, but only after newaliases gets unbroken: As of last week
(a3d4ae7cf351) newaliases fails with EPERM on /etc/dma/dma.conf.

This should unbreak VM (and cloudware) cross-builds.

With hat: re@

show more ...

release: Consistently use variables for invoking certain tools

This change cleans up the inconsistent ways of invoking makefs and mkimg
by making sure that these tools are invoked through their corr

release: Consistently use variables for invoking certain tools

This change cleans up the inconsistent ways of invoking makefs and mkimg
by making sure that these tools are invoked through their corresponding
variables. This was previously only used in a few amd64 and arm64 release
scripts.

Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51039
Reviewed by: emaste, markj

show more ...

release: style: prefer `-n` over `! -z`

This is (nearly) universally the style of other shell scripts.

Reviewed by: cperciva, imp, emaste
Differential Revision: https://reviews.freebsd.org/D50056

release/vm: empty sh functions are disallowed

This happens to work with FreeBSD's shell, but is rejected by dash and
bash.

Reported by: def
Reviewed by: emaste
Sponsored by: DARPA, AFRL
Differentia

release/vm: empty sh functions are disallowed

This happens to work with FreeBSD's shell, but is rejected by dash and
bash.

Reported by: def
Reviewed by: emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D49526

show more ...

release/vm: partially support NO_ROOT

For images that don't require QEMU, support NO_ROOT. This entails:
- Passing NO_ROOT down to mk-vmimage.sh (which sets it for
installworld, etc)
- Handlin

release/vm: partially support NO_ROOT

For images that don't require QEMU, support NO_ROOT. This entails:
- Passing NO_ROOT down to mk-vmimage.sh (which sets it for
installworld, etc)
- Handling etcupdate bootstrapping
- Adding assorted config file METALOG entries
- Running makefs in the right directory and adding -D for dups

The main gap in basic NO_ROOT support is package installation. Each
image type must also be updated to add METALOG entries for any files it
adds.

Reviewed by: emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D49320

show more ...

release/vm: add a WITHOUT_QEMU option

This option disables trying to build a qemu-<target>-static and simply
doesn't do any of the operations that rely on it. This disables package
installation whi

release/vm: add a WITHOUT_QEMU option

This option disables trying to build a qemu-<target>-static and simply
doesn't do any of the operations that rely on it. This disables package
installation which is required by most, but not all image types.

Disable all CLOUDWARE variants that require pkg when WITHOUT_QEMU is set
to avoid broken images.

Reviewed by: emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D49310

show more ...

vmimage.subr: Redirect etcupdate log to stdout

While we're here, send the etcupdate log from generating base.txz
to stdout instead of /dev/null (see e972e408d19a) as well.

Reviewed by: emaste, gord

vmimage.subr: Redirect etcupdate log to stdout

While we're here, send the etcupdate log from generating base.txz
to stdout instead of /dev/null (see e972e408d19a) as well.

Reviewed by: emaste, gordon, jrtc27
MFC after: 1 week
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D48719

show more ...

release: vmimage: setup the default BE properly

All of the BE datasets need to be set canmount=noauto so that creating
a new BE and switching to it can actually work. With the current setup,
the zf

release: vmimage: setup the default BE properly

All of the BE datasets need to be set canmount=noauto so that creating
a new BE and switching to it can actually work. With the current setup,
the zfs rc script will mount the `default` BE over whichever new BE is
activated once it runs.

Reported by: andrew
Reviewed by: andrew, re (cperciva), imp, markj
Differential Revision: https://reviews.freebsd.org/D48017

show more ...

vmimage.subr: Pass $INSTALLOPTS to install*

This makes it possible for a VM build configuration file to pass
options to make installworld/installkernel/distribution, e.g.
WITHOUT_DEBUG_FILES=YES in

vmimage.subr: Pass $INSTALLOPTS to install*

This makes it possible for a VM build configuration file to pass
options to make installworld/installkernel/distribution, e.g.
WITHOUT_DEBUG_FILES=YES in order to produce smaller images.

Note that these options are only applied at install time, not at
build time (since the same build is installed into many different
VM images), so not all src.conf options are usable here.

Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D46506

show more ...

release: Rework vm_extra_pre_umount

The vm_extra_pre_umount function in vmimage.subr served two purposes:
It removed /etc/resolv.conf and /qemu (if cross-building), and it
provided a function for cl

release: Rework vm_extra_pre_umount

The vm_extra_pre_umount function in vmimage.subr served two purposes:
It removed /etc/resolv.conf and /qemu (if cross-building), and it
provided a function for cloudware to override in order to make cloud
specific changes to the filesystem before constructing a disk image.

This resulted in a number of bugs:
1. When cross-building, the emulator binary was left as /qemu in the
Azure, GCE, Openstack and Vagrant images.
2. The build host's resolv.conf was left as /etc/resolv.conf in the
basic-ci and basic-cloudinit images.
3. When building GCE images, a Google-specific resolv.conf file was
constructed, and then deleted before the disk image was created.

Move the bits needed for running code inside a VM staging directory
from vm_install_base into a new vm_emulation_setup routine, and move
the corresponding cleanup bits from vm_extra_pre_umount to a new
vm_emulation_cleanup routine.

Remove the /qemu and /etc/resolv.conf cleanups from the cloudware
configuration files (where they exist) since we will now be running
vm_emulation_cleanup to remove those even when vm_extra_pre_umount
has been overridden.

Override vm_emulation_cleanup in gce.conf since in that one case (and
*only* that one case) we don't want to clean up resolv.conf (since it
was constructed for the VM image rather than copied from the host).

releng/14.1 candidate.

MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva

show more ...

release: Change vmimage EFI GPT label

This matches the default bsdinstall nomenclature.

PR: 278480
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1188

release: add basic cloudinit images

Provide both zfs and ufs images which a 1MB partition reserved for the
config drive wearing a GPT Label "config-drive" to allow consumer to
know where they should

release: add basic cloudinit images

Provide both zfs and ufs images which a 1MB partition reserved for the
config drive wearing a GPT Label "config-drive" to allow consumer to
know where they should push the config drive on the provided image.

2 formats available: qcow2 and raw

This has been tested on OVHCloud baremetal via "bring your own image"
Also tested on openstack

Reviewed by: emaste
Sponsored by: OVHCloud
Differential Revision: https://reviews.freebsd.org/D44369

show more ...

release/tools: Add support for building armv7 vm

Currently there is no support for generating armv7 vm images in the
release artifacts. In fact in terms of release artifacts and
architecture there i

release/tools: Add support for building armv7 vm

Currently there is no support for generating armv7 vm images in the
release artifacts. In fact in terms of release artifacts and
architecture there is no good reason to have a vm release artifact for
armv7 as those are mostly used in SOCs or embedded boards. However
considering that developers actually do need an easy way to test armv7
with a vm running this is really important. As part of pre-commit ci for
developers this can be really helpful for the end developers.

Approved by: cperciva, imp, re
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43952

show more ...

release: Add missing /var/crash ZFS dataset

This matches the layout from bsdinstall.

MFC after: 1 week
Fixes: 89585511cc05 ("release: Add support for creating ZFS-based VM images")
Pull Request: ht

release: Add missing /var/crash ZFS dataset

This matches the layout from bsdinstall.

MFC after: 1 week
Fixes: 89585511cc05 ("release: Add support for creating ZFS-based VM images")
Pull Request: https://github.com/freebsd/freebsd-src/pull/981

show more ...

release: stop generating mergemaster databases

mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.

Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Diff

release: stop generating mergemaster databases

mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.

Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41797

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release/tools/vmimage.subr: switch zfs dataset from /usr/home to /home

Change the vmimage script for zfs to create /home as a dataset
rather than /usr/home, ala change to bsdinstall's zfs script.

R

release/tools/vmimage.subr: switch zfs dataset from /usr/home to /home

Change the vmimage script for zfs to create /home as a dataset
rather than /usr/home, ala change to bsdinstall's zfs script.

Reviewed by: markj
Differential Revision: <https://reviews.freebsd.org/D40111

show more ...

release: Report disk image filename

For someone new to the release bits it's not always clear what files are
being created. Report the disk image name explicitly.

Reviewed by: gjb
Sponsored by: The

release: Report disk image filename

For someone new to the release bits it's not always clear what files are
being created. Report the disk image name explicitly.

Reviewed by: gjb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39953

show more ...

release(7): Enable zpoolupgrade rc script in ZFS based VM images

This will enable VM access to all ZFS feature automatically, only on a
newly installed or provisioned VM or cloud instance.

Reviewed

release(7): Enable zpoolupgrade rc script in ZFS based VM images

This will enable VM access to all ZFS feature automatically, only on a
newly installed or provisioned VM or cloud instance.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37283

show more ...

release: Add support for creating ZFS-based VM images

The change extends vmimage.subr to handle a new parameter, VMFS, which
should be equal to either "ufs" or "zfs". When it is set to ZFS, we use

release: Add support for creating ZFS-based VM images

The change extends vmimage.subr to handle a new parameter, VMFS, which
should be equal to either "ufs" or "zfs". When it is set to ZFS, we use
makefs to create a bootable pool populated using the same dataset layout
as bsdinstall and "poudriere image" use. The pool can be grown using
the growfs rc.d script, just as in UFS images.

This will make it easy to provide VM and cloud images with ZFS as the
root filesystem. So far I did not do extensive testing of cloud images;
I merely verified that creation of ZFS-based AWS AMIs works and allows
me to create amd64 and arm64 EC2 instances with ZFS as the root
filesystem.

Reviewed by: emaste, gjb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34426

show more ...

Update Azure release bits

Imports the changes for building official images on Azure Marketplace,
which fulfill the requirements of Azure and FreeBSD cloud images like
disk layout and UEFI for Gen2 V

Update Azure release bits

Imports the changes for building official images on Azure Marketplace,
which fulfill the requirements of Azure and FreeBSD cloud images like
disk layout and UEFI for Gen2 VM, along with some minor improvements like
configurations to speed up booting.

"CLOUDWARE" list will be updated after some more collaborations with re
completed.

Reviewed by: re (gjb)
Sponsored by: The FreeBSD Foundation
Technical assistance from: Microsoft
Differential Revision: https://reviews.freebsd.org/D23804

show more ...