Makefile.inc1: Force NO_ROOT for distribute* and package*

These targets are used to produce legacy dist sets for install media and
now always use NO_ROOT mode. Extend existing logic that forces NO_

Makefile.inc1: Force NO_ROOT for distribute* and package*

These targets are used to produce legacy dist sets for install media and
now always use NO_ROOT mode. Extend existing logic that forces NO_ROOT
mode to these cases to ensure they do not run in the wrong mode.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D50551

show more ...

make packages: Fix edge case of no kernel

When NO_INSTALLKERNEL and NO_INSTALLEXTRAKERNELS are set, or when
BUILDKERNELS is empty, we can fail to define the create-kernel-packages
make target, which

make packages: Fix edge case of no kernel

When NO_INSTALLKERNEL and NO_INSTALLEXTRAKERNELS are set, or when
BUILDKERNELS is empty, we can fail to define the create-kernel-packages
make target, which we call unconditionally. So if do not have a kernel
staging dir, go ahead and provide an empty create-kernel-packages so
that when it's used elsewhere, the build succeeds.

This lets me build a tree and package it when I don't have a kernel, for
jails and other similar environments. There's no change if any kernels
are built, since that will provide the necessary make target that does
useful things for that case.

Sponsored by: Netflix

show more ...

Makefile.inc1: Fix package-pkg with a non-default LOCALBASE

package-pkg (via make-pkg-package.sh) passes CONFIGURE_ARGS to make when
building ports-mgmt/pkg, which overrides the port's default confi

Makefile.inc1: Fix package-pkg with a non-default LOCALBASE

package-pkg (via make-pkg-package.sh) passes CONFIGURE_ARGS to make when
building ports-mgmt/pkg, which overrides the port's default configure
args that are supposed to set --prefix. This means that pkg is always
built with the default prefix of /usr/local, which then fails when ports
tries to package it from LOCALBASE.

Work around this by explicitly adding --prefix to CONFIGURE_ARGS.

MFC after: 1 day
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D52634

show more ...

Makefile.inc1: Handle pkg development versions

Development versions of pkg have a version like "2.3.0-d8bbde1c3-dirty",
which is not handled by PKG_BIN_VERSION, so PKG_EXT ends up being "tar"
and th

Makefile.inc1: Handle pkg development versions

Development versions of pkg have a version like "2.3.0-d8bbde1c3-dirty",
which is not handled by PKG_BIN_VERSION, so PKG_EXT ends up being "tar"
and this breaks update-packages.

Fix the PKG_BIN_VERSION logic, and while here, remove backwards compat
for pkg versions prior to 2.0, which no one should be using to build
main or stable/15. This means nothing is left to use PKG_BIN_VERSION,
but continue to set it, since we might need it again in the future.

MFC after: 1 day
Reviewed by: manu, kevans
Differential Revision: https://reviews.freebsd.org/D52666

show more ...

pkgbootstrap: Gate building from ports

In release builds we want to build pkg from the ports tree for reasons
mentioned in a previous commit; but non-release pkgbase builds may be
just fine with ins

pkgbootstrap: Gate building from ports

In release builds we want to build pkg from the ports tree for reasons
mentioned in a previous commit; but non-release pkgbase builds may be
just fine with installing pkg from pkg.freebsd.org.

Gate it behind BOOTSTRAP_PKG_FROM_PORTS rather than building form ports
any time we have a ports tree available.

While I'm here, make the pkgbase-repo target in release/Makefile
create the temporary directories we need for building.

Reviewed by: ivy
MFC after: 6 hours
Sponsored by: https://www.patreon.com/cperciva
Fixes: 2c06b824436d ("pkgbootstrap: Build pkg if PORTSDIR exists")

show more ...

pkgbase: Don't include pkg by default

We need to put pkg into the pkgbase repository for release builds, but
not for non-release pkgbase builds. Gate the contents of package-pkg
behind INCLUDE_PKG_

pkgbase: Don't include pkg by default

We need to put pkg into the pkgbase repository for release builds, but
not for non-release pkgbase builds. Gate the contents of package-pkg
behind INCLUDE_PKG_IN_PKGBASE_REPO=YES and set that when building
releases.

Reviewed by: ivy
MFC after: 6 hours
Sponsored by: https://www.patreon.com/cperciva

show more ...

pkgbootstrap: Build pkg if PORTSDIR exists

If we don't already have the pkg tool installed, we need to install it
before we can build packages for pkgbase. (This arises in release
builds which run

pkgbootstrap: Build pkg if PORTSDIR exists

If we don't already have the pkg tool installed, we need to install it
before we can build packages for pkgbase. (This arises in release
builds which run in clean chroots; most users will have pkg installed
already.) Installing it from pkg.freebsd.org via "pkg bootstrap" runs
into two problems: First, it's an external dependency with security
implications; and second, there's a bootstrapping problem on new major
versions where pkg.freebsd.org does not have packages yet.

Instead, if we have a ports tree, use it to build pkg ourselves.

With hat: re
MFC after: 6 hours
Sponsored by: https://www.patreon.com/cperciva

show more ...

Bootstrap pkg before building a pkg package

At sufficiently high levels of build parallelism we can try to create
the pkg before any of the other pkgbase-creation steps have taken
care of the bootst

Bootstrap pkg before building a pkg package

At sufficiently high levels of build parallelism we can try to create
the pkg before any of the other pkgbase-creation steps have taken
care of the bootstrap.

Fixes: 6b5adf33873a ("Include a pkg package in the pkgbase repo")
MFC after: 12 hours
Sponsored by: https://www.patreon.com/cperciva

show more ...

Include a pkg package in the pkgbase repo

It is essential that users be able to install the FreeBSD base system
from release media and have all the bits needed to update the FreeBSD
base system with

Include a pkg package in the pkgbase repo

It is essential that users be able to install the FreeBSD base system
from release media and have all the bits needed to update the FreeBSD
base system without touching the ports tree or pkg.freebsd.org. To
that end, resurrect (and heavily rewrite) the make-pkg-package.sh
script and hook it into the create-packages target; if /usr/ports
exists when building pkgbase packages, we'll also (cross)build pkg
and include it in the repository.

Scripting for actually installing this package as part of the FreeBSD
installation process to come later, but I wanted this in the tree in
time to test it in this week's snapshots.

MFC after: 1 day
Sponsored by: https://www.patreon.com/cperciva

show more ...

packages: Add meta-package sets

Add several metapackages which contain no files themselves, but depend
on other packages. This allows the user to install a system by using
a set like "FreeBSD-set-m

packages: Add meta-package sets

Add several metapackages which contain no files themselves, but depend
on other packages. This allows the user to install a system by using
a set like "FreeBSD-set-minimal" instead of having to know which
specific packages are required, and it means if we add more package
in the future, existing installations will get them automatically as
long as they were installed using a set.

The defined sets are significantly less granular than the actual
packages; the assumption is that users who want a very specific set of
packages installed can do that manually, while sets are intended more
for typical users who just want to install FreeBSD.

The following sets are defined:

* minimal: the set of packages required to boot the system and bring
up a multi-user UNIX system. This includes hardware support,
networking (wireless, DHCP), basic functionality like syslogd, cron
and periodic. It does not include a kernel, because the kernels are
quite large and the user might want to use a custom kernel.

sendmail and DMA are not included in minimal. Since we provide two
MTAs, the user should select which one they want, or install one from
ports. sshd is not included because it's not required for the base
system, and the user might want to install it from ports.

Some other significant subsystems (e.g. NFS, Kerberos) are also not
included.

The minimal set does not include any libraries itself, since we rely
on dependencies to pull these in. This implies it doesn't include
library manpages, which is fine, since users won't want manpages for
libraries unless they've also installed the development packages.

* devel: the compiler toolchain for building software, along with all
"-dev" packages which include header files and static libraries.

* lib32: 32-bit compatibility packages. This includes both runtime
and development packages.

* kernels: the kernel packages. For freebsd.org builds, this will be
GENERIC, GENERIC-DEBUG, etc., for custom builds it will be whatever
was set in $KERNCONF.

* base: everything else; this includes the entire base system.

For each set, also generate a <setname>-dbg set containing the debug
packages for the set.

The sets are built along with the rest of the packages using a new
script called create-sets.sh, which examines the "set" annotation in
each package and puts it in the appropriate set. This is in
anticipation of the later "groups" feature appearing in pkg itself, at
which point we can simply replace the set annotation with the group.

MFC after: 3 days
Reviewed by: imp, bapt
Differential Revision: https://reviews.freebsd.org/D52412

show more ...

pkgbase: fix alpha and beta versioning

MFC After: 1 day

Makefile.inc1: Build openssl as a certctl dependency on old FreeBSD

X509_NAME_hash_ex() is not available in OpenSSL 1.x.

Reviewed by: des
Reported by: syzbot+54c1e57e7184582a8f59@syzkaller.appspotm

Makefile.inc1: Build openssl as a certctl dependency on old FreeBSD

X509_NAME_hash_ex() is not available in OpenSSL 1.x.

Reviewed by: des
Reported by: syzbot+54c1e57e7184582a8f59@syzkaller.appspotmail.com
Fixes: c340ef28fd38 ("certctl: Reimplement in C")
Differential Revision: https://reviews.freebsd.org/D52291

show more ...

Makefile.inc1: Avoid including cwd in path

Buildworld failed when objcopy tried to overwrite itself, with `objcopy:
open objcopy failed: Text file busy`. The PATH ended up with `::`,
effectively in

Makefile.inc1: Avoid including cwd in path

Buildworld failed when objcopy tried to overwrite itself, with `objcopy:
open objcopy failed: Text file busy`. The PATH ended up with `::`,
effectively including the current directory, and we found the wrong
objcopy.

PR: 261215
Sponsored by: The FreeBSD Foundation

show more ...

build: fix list-old-dirs / check-old-dirs

Fix list-old-dirs to not generate empty line at the end of the
output if OLD_DIRS happens to have a space at the end of the
last word of the output before t

build: fix list-old-dirs / check-old-dirs

Fix list-old-dirs to not generate empty line at the end of the
output if OLD_DIRS happens to have a space at the end of the
last word of the output before the final \n. Then that space
is turned into \n and we end up with a blank line.

Futhermore this gets converted into a "/" i.e. root fs when
calling check-old-dirs.

This is the regression since a8267ecc3df0a.

Reviewed by: emaste
Approved by: emaste
Sponsored by: Sippy Software, Inc.
Differential Revision: https://reviews.freebsd.org/D52153
MFC After: 3 days

show more ...

sys/conf: Remap paths for reproducibility

As in bsd.lib.mk and bsd.prog.mk, we need to remap paths in the kernel
and in kernel modules when reproducible builds are configured. Add
directives to ker

sys/conf: Remap paths for reproducibility

As in bsd.lib.mk and bsd.prog.mk, we need to remap paths in the kernel
and in kernel modules when reproducible builds are configured. Add
directives to kern.post.mk and kmod.mk to remap the src, obj and sysroot
dirs.

For the sysroot, we need to pass the value of the SYSROOT variable to
the kernel makefiles, which involves changing Makefile.inc1. In so
doing I reverted a portion of commit 35fb812adfc21; I can't see any harm
in passing the SYSROOT variable down.

MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D50953

show more ...

Remove MK_GSSAPI

For MIT Kerberos, MK_GSSAPI has no meaning: GSSAPI is a required part of
Kerberos and is always built if MK_KERBEROS is enabled. Backport this
behaviour to Heimdal so it works the

Remove MK_GSSAPI

For MIT Kerberos, MK_GSSAPI has no meaning: GSSAPI is a required part of
Kerberos and is always built if MK_KERBEROS is enabled. Backport this
behaviour to Heimdal so it works the same way.

While here, change Heimdal's libcom_err and compile_et to be selected by
MK_KERBEROS, not MK_KERBEROS_SUPPORT, since these are part of Kerberos
and third-party users might need it even if Kerberos support is disabled
in the base system. This means MK_KERBEROS_SUPPORT installs the same
files with both MIT and Heimdal.

Reviewed by: cy
Differential Revision: https://reviews.freebsd.org/D51859

show more ...

build: remove certctl requirement for host OpenSSL libs on macOS

Some platforms, like macOS, do not expose headers for the system's
libcrypto for public consumption. libcrypto is relatively heavy a

build: remove certctl requirement for host OpenSSL libs on macOS

Some platforms, like macOS, do not expose headers for the system's
libcrypto for public consumption. libcrypto is relatively heavy and
needs to know, e.g., the host system's endianness, so we scope the build
down to macOS where OpenSSL headers are known to not be present and we
can be reasonably certain that most of the systems today that would be
cross-building are little endian.

We still don't bother if building WITHOUT_OPENSSL since the end result
is expected to be used by OpenSSL, but perhaps we could revisit that
independently in case one, e.g., brings their own implementation.

Reported by: jrtc27
Reviewed by: jrtc27, ngie
Fixes: c340ef28fd38 ("certctl: Reimplement in C")
Differential Revision: https://reviews.freebsd.org/D51935

show more ...

Makefile.inc1: garbage collect unused variable: install-info

The variable's definition was removed with texinfo in
2d2813618c3818d7d41a7ced1fca4a1a01d3591d, but the reference to the
variable was lef

Makefile.inc1: garbage collect unused variable: install-info

The variable's definition was removed with texinfo in
2d2813618c3818d7d41a7ced1fca4a1a01d3591d, but the reference to the
variable was left behind.

This is effectively a no-op cleanup.

MFC after: 3 days

show more ...

certctl: Reimplement in C

Notable changes include:

* We no longer forget manually untrusted certificates when rehashing.

* Rehash will now scan the existing directory and progressively replace
i

certctl: Reimplement in C

Notable changes include:

* We no longer forget manually untrusted certificates when rehashing.

* Rehash will now scan the existing directory and progressively replace
its contents with those of the new trust store. The trust store as a
whole is not replaced atomically, but each file within it is.

* We no longer attempt to link to the original files, but we don't copy
them either. Instead, we write each certificate out in its minimal
form.

* We now generate a trust bundle in addition to the hashed diretory.
This also contains only the minimal DER form of each certificate.
This allows e.g. Unbound to preload the bundle before chrooting.

* The C version is approximately two orders of magnitude faster than the
sh version, with rehash taking ~100 ms vs ~5-25 s depending on whether
ca_root_nss is installed.

* We now also have tests.

Reviewed by: kevans, markj
Differential Revision: https://reviews.freebsd.org/D42320
Differential Revision: https://reviews.freebsd.org/D51896

show more ...

Revert certctl reimplementation and follow-ups

The reimplementation of certctl, while much needed, broke the release
build and 72 hours later corrections are still under review (D51896).

This rever

Revert certctl reimplementation and follow-ups

The reimplementation of certctl, while much needed, broke the release
build and 72 hours later corrections are still under review (D51896).

This revert should be reverted once that is ready to land; I just need
this out of the tree temporarily because breakage is interfering with
release engineering for the upcoming 15.0-RELEASE.

Unsquashed reversions:
Revert "etc: add missing mtree entry for certctl tests"
This reverts commit f751757259158a8d3b81d4fb7576b3ebe226dece.
Revert "certctl: Fix bootstrap build"
This reverts commit c989e3cc3da1bfd8ac3ec5a05d1e86ab8ff719f7.
Revert "certctl: Reimplement in C"
This reverts commit 81d8827ad8752e35411204541f1f09df1481e417.

With hat: re@

show more ...

build: introduce the notion of a build epoch

Idea and file format shamelessly stolen from CheriBSD, but reimplemented
in terms of the standard FreeBSD build system. We'll use this in some
events th

build: introduce the notion of a build epoch

Idea and file format shamelessly stolen from CheriBSD, but reimplemented
in terms of the standard FreeBSD build system. We'll use this in some
events that call for a deeper cleansing, typically reserved for
situations where the dependencies are too complicated to unwind. This
notably does not preclude us from doing separate cleansing of world for
specific src.conf(5) knob changes that would require a rebuild.

In the FreeBSD version, we either stamp the OBJTOP we're cleaning
(bootstrap or the full OBJDIR) with the current epoch and bail out for
unstamped objdirs, or we compare and either `rm -rf` or `cleandir` as
necessary.

Reviewed by: brooks, des
Differential Revision: https://reviews.freebsd.org/D51848

show more ...

certctl: Reimplement in C

Notable changes include:

* We no longer forget manually untrusted certificates when rehashing.

* Rehash will now scan the existing directory and progressively replace
i

certctl: Reimplement in C

Notable changes include:

* We no longer forget manually untrusted certificates when rehashing.

* Rehash will now scan the existing directory and progressively replace
its contents with those of the new trust store. The trust store as a
whole is not replaced atomically, but each file within it is.

* We no longer attempt to link to the original files, but we don't copy
them either. Instead, we write each certificate out in its minimal
form.

* We now generate a trust bundle in addition to the hashed diretory.
This also contains only the minimal DER form of each certificate.

* The C version is approximately two orders of magnitude faster than the
sh version, with rehash taking ~100 ms vs ~5-25 s depending on whether
ca_root_nss is installed.

* The DISTBASE concept has been dropped; the same effect can be achieved
by adjusting DESTDIR.

* We now also have rudimentary tests.

Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42320

show more ...

gssapi,krb5: Replace libgssapi with the MIT version

lib/libgssapi is based on Heimdal. As on Linux systems, the MIT
libgssapi_krb5 replaces it. With both gssapi libraries and header files
installed

gssapi,krb5: Replace libgssapi with the MIT version

lib/libgssapi is based on Heimdal. As on Linux systems, the MIT
libgssapi_krb5 replaces it. With both gssapi libraries and header files
installed results in broken buildworld (gssd) and ports that will not
build without modifications to support the MIT gssapi in an alternate
location.

73ed0c7992fd removed the MIT GSSAPI headers from /usr/include. Apps using
MIT KRB5 gssapi functions and structures will fail to build without this
patch.

This patch includes a temporary patch to usr.sbin/gssd to allow it
to build with this patch. rmacklem@ has a patch for this and for
kgssapi that uses this patch to resolve kgssapi issues for NFS with
Kerberos.

This patch is an updated version of D51661 to allow it to build following
additional patchs to the tree.

This should have been implmented with 7e35117eb07f.

Fixes: 7e35117eb07f, 73ed0c7992fd
Differential Revision: https://reviews.freebsd.org/D51661

show more ...

ObsoleteFiles: Stop looking for catpages

We stopped installing catpages in 2017, before stable/12.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D51715

krb5: remove libedit from the bootstrap tools

libedit breaks the bootstrap on MacOS and Linux.

Activate libedit only for the regular build not for the bootstrap
tools

While here fix the definition

krb5: remove libedit from the bootstrap tools

libedit breaks the bootstrap on MacOS and Linux.

Activate libedit only for the regular build not for the bootstrap
tools

While here fix the definition of the dependency chain between
libkrb5ss and libedit (and libtinfow) via src.libnames.mk

Remove a local patch to find the readline compatible header and
find them via proper CFLAGS.

show more ...