makefs: Honor -T timestamps when creating images from mtree manifests

makefs backends rely on the fsnode structure to derive most
of the information about the underlying filesystem objects.
Dependin

makefs: Honor -T timestamps when creating images from mtree manifests

makefs backends rely on the fsnode structure to derive most
of the information about the underlying filesystem objects.
Depending on how the image is built, the fsnode structures
are initialized in the walk_dir or read_mtree functions.
However, read_mtree fails to take timestamps passed by -T into account,
leading to nonreproducible images in backends that do not check for -T.
Fix this and make -T backend-agnostic by adding an appropriate check
in read_mtree_keywords while making sure that mtree entries can
still override -T timestamps.

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, emaste
Differential Revision: https://reviews.freebsd.org/D49531

show more ...

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement memb

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement member to struct fsinode to store it on Linux.

Note that we add a fallback definition of HAVE_STRUCT_STAT_ST_FLAGS for
makefs.h since etdump reaches into makefs and won't have it defined in
its CFLAGS like makefs itself does, and on FreeBSD we don't provide an
nbtool_config.h.

Note also that strtofflags is currently a stub on Linux so this doesn't
yet properly mirror the mtree, but that will change in a future commit.

Reviewed by: emaste, markj
Differential Revision: https://reviews.freebsd.org/D50078

show more ...

makefs: Make it possible to silence warnings about duplicate paths

When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in

makefs: Make it possible to silence warnings about duplicate paths

When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in the
manifest. These are harmless and have been around for a long time (see
the phabricator revision for some more details), so let's at least have
a way to make makefs quieter.

Reviewed by: brooks, imp, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D43513

show more ...

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

makefs: Handle missing link keyword in mtree spec

PR: 241178
Reviewed by: emaste
MFC after: 3 days
Pull Request: https://github.com/freebsd/freebsd-src/pull/850

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

makefs: Fix typos

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

makefs: Ignore the "tags" keyword in mtree manifests

An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emittin

makefs: Ignore the "tags" keyword in mtree manifests

An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emitting a warning for each entry.

Reviewed by: brooks, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29384

show more ...

usr.sbin/makefs: fix use-after-free in read_mtree_keywords()

The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing

usr.sbin/makefs: fix use-after-free in read_mtree_keywords()

The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing st.

Reported by: AddressSanitizer
Reviewed By: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28570

show more ...

MFHead @345353

makefs: Fix "time" mtree attribute handling

When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsec

makefs: Fix "time" mtree attribute handling

When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsecond component (e.g. time=1551620152.987220000).

Update the handling logic to properly assign the subsecond component if
built with nanosecond support, or silently discard it otherwise.

Also, re-enable the time attribute for the kyua tests.

PR: 194703
Submitted by: Mitchell Horne <mhorne063@gmail.com>
Differential Revision: https://reviews.freebsd.org/D19627

show more ...

makefs: Honor -T timestamps when creating images from mtree manifests

makefs backends rely on the fsnode structure to derive most
of the information about the underlying filesystem objects.
Dependin

makefs: Honor -T timestamps when creating images from mtree manifests

makefs backends rely on the fsnode structure to derive most
of the information about the underlying filesystem objects.
Depending on how the image is built, the fsnode structures
are initialized in the walk_dir or read_mtree functions.
However, read_mtree fails to take timestamps passed by -T into account,
leading to nonreproducible images in backends that do not check for -T.
Fix this and make -T backend-agnostic by adding an appropriate check
in read_mtree_keywords while making sure that mtree entries can
still override -T timestamps.

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, emaste
Differential Revision: https://reviews.freebsd.org/D49531

show more ...

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement memb

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement member to struct fsinode to store it on Linux.

Note that we add a fallback definition of HAVE_STRUCT_STAT_ST_FLAGS for
makefs.h since etdump reaches into makefs and won't have it defined in
its CFLAGS like makefs itself does, and on FreeBSD we don't provide an
nbtool_config.h.

Note also that strtofflags is currently a stub on Linux so this doesn't
yet properly mirror the mtree, but that will change in a future commit.

Reviewed by: emaste, markj
Differential Revision: https://reviews.freebsd.org/D50078

show more ...

makefs: Make it possible to silence warnings about duplicate paths

When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in

makefs: Make it possible to silence warnings about duplicate paths

When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in the
manifest. These are harmless and have been around for a long time (see
the phabricator revision for some more details), so let's at least have
a way to make makefs quieter.

Reviewed by: brooks, imp, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D43513

show more ...

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

makefs: Handle missing link keyword in mtree spec

PR: 241178
Reviewed by: emaste
MFC after: 3 days
Pull Request: https://github.com/freebsd/freebsd-src/pull/850

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

makefs: Fix typos

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

makefs: Ignore the "tags" keyword in mtree manifests

An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emittin

makefs: Ignore the "tags" keyword in mtree manifests

An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emitting a warning for each entry.

Reviewed by: brooks, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29384

show more ...

usr.sbin/makefs: fix use-after-free in read_mtree_keywords()

The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing

usr.sbin/makefs: fix use-after-free in read_mtree_keywords()

The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing st.

Reported by: AddressSanitizer
Reviewed By: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28570

show more ...

MFHead @345353

makefs: Fix "time" mtree attribute handling

When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsec

makefs: Fix "time" mtree attribute handling

When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsecond component (e.g. time=1551620152.987220000).

Update the handling logic to properly assign the subsecond component if
built with nanosecond support, or silently discard it otherwise.

Also, re-enable the time attribute for the kyua tests.

PR: 194703
Submitted by: Mitchell Horne <mhorne063@gmail.com>
Differential Revision: https://reviews.freebsd.org/D19627

show more ...

makefs: Use ENODATA instead of ENOMSG as a translation for missing ENOATTR.

This is consistent with what some linux filesystems do and has been
adopted in our linuxulator.

MFC after: 3 days