makefs: Calculate indirect block count properly for large files on ffs

When building an ffs image containing large file(s), space requirements
were calculated incorrectly yielding a bigger image tha

makefs: Calculate indirect block count properly for large files on ffs

When building an ffs image containing large file(s), space requirements
were calculated incorrectly yielding a bigger image than necessary.
The reason is that amount of indirect blocks estimation was done wrong:

- single indirect block was treated as it can hold just 12 data blocks
- nested indirect blocks were not taken into account at all

Add support for indirect blocks and fix another tiny bug with
underestimated space requirement for files with size between
(UFS_NDADDR-1)*blksz+fragsz ... (UFS_NDADDR)*blksz requesting N>1
fragments instead of a whole block.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D52120

show more ...

makefs: Fix undefined behaviour in ffs.c

Fix a UBSAN-reported error in `ffs_make_dirbuf` where a zero offset
gets applied to a NULL pointer.

Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD

makefs: Fix undefined behaviour in ffs.c

Fix a UBSAN-reported error in `ffs_make_dirbuf` where a zero offset
gets applied to a NULL pointer.

Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51037
Reviewed by: emaste, mckusick

show more ...

makefs: Remove redundant 'stampst' checks in several backends

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.

makefs: Remove redundant 'stampst' checks in several backends

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D49601

show more ...

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement memb

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement member to struct fsinode to store it on Linux.

Note that we add a fallback definition of HAVE_STRUCT_STAT_ST_FLAGS for
makefs.h since etdump reaches into makefs and won't have it defined in
its CFLAGS like makefs itself does, and on FreeBSD we don't provide an
nbtool_config.h.

Note also that strtofflags is currently a stub on Linux so this doesn't
yet properly mirror the mtree, but that will change in a future commit.

Reviewed by: emaste, markj
Differential Revision: https://reviews.freebsd.org/D50078

show more ...

Standardize the definition of a UFS dinode.

Each program that operates on UFS on-disk inodes defines its own
version of a dinode. They all (of necessity) define the same
layout but use different nam

Standardize the definition of a UFS dinode.

Each program that operates on UFS on-disk inodes defines its own
version of a dinode. They all (of necessity) define the same
layout but use different names. This change adds a definition of
a dinode (a union of a UFS1 on-disk inode and a UFS2 on-disk inode)
as well as a dinodep (a union of a pointer to a UFS1 on-disk inode
and a pointer to a UFS2 on-disk inode) in sys/ufs/ufs/dinode.h.
It then deletes the definitions of dinode and dinodep in all the
programs that operate on them and instead uses these standard
definitions.

No functional change intended.

MFC-after: 1 week

show more ...

makefs: warn that ffs sectorsize other than 512 may not work

newfs always sets sectorsize to DEV_BSIZE (512) and derives some other
values based on the number of 512-byte sectors per real sector. S

makefs: warn that ffs sectorsize other than 512 may not work

newfs always sets sectorsize to DEV_BSIZE (512) and derives some other
values based on the number of 512-byte sectors per real sector. Similar
logic is required in makefs. Until that happens, emit a warning that
the image may be incorrect.

PR: 276571

show more ...

usr.sbin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a p

usr.sbin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

makefs: call brelse from bread

This matches NetBSD and rationalizes makefs with the kernel API.

This reverts commit 370e009188ba90c3290b1479aa06ec98b66e140a.

Reviewed by: mckusick
Sponsored by: Th

makefs: call brelse from bread

This matches NetBSD and rationalizes makefs with the kernel API.

This reverts commit 370e009188ba90c3290b1479aa06ec98b66e140a.

Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
Obtained from: NetBSD 0a62dad69f62, 0c4125e1a19f, cb6a5a3575fd
Differential Revision: https://reviews.freebsd.org/D39070

show more ...

makefs: do not pass mode to open() call lacking O_CREAT

Obtained from: OpenBSD ffs.c 1.35

makefs: whitespace cleanup (remove space before tab)

MFC after: 1 week

makefs: fix calculation of file sizes

When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:

1) Count the space needed for ind

makefs: fix calculation of file sizes

When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:

1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.

However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.

Adjust size calculations to match what is done in ffs_mkfs routine:

* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.

Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.

PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132

show more ...

makefs: Fix warnings and reset WARNS to the default

Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
poin

makefs: Fix warnings and reset WARNS to the default

Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
pointers to various filesystem structures. Fixing this properly would
be too much work for too little gain.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

ufs: Rework shortlink handling to avoid subobject overflows

Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer

ufs: Rework shortlink handling to avoid subobject overflows

Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.

Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].

Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650

show more ...

Allocate extra inodes in makefs when leaving free space in UFS images.

By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great fo

Allocate extra inodes in makefs when leaving free space in UFS images.

By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.

Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29492

show more ...

Remove #define _KERNEL hacks from libprocstat

Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more materi

Remove #define _KERNEL hacks from libprocstat

Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.

Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.

Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679

show more ...

makefs: Also set UFS di_birthtime when building on Linux

Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.

Reviewed By: emaste
Di

makefs: Also set UFS di_birthtime when building on Linux

Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D22386

show more ...

MFhead@r344786

Merge ^/head r344549 through r344775.

Improve error handling: bail out if one of the files scheduled
to go to the FS image we are making cannot be read (e.g. EPERM).
Current behaviour when we issue waring but still proceeed and
return su

Improve error handling: bail out if one of the files scheduled
to go to the FS image we are making cannot be read (e.g. EPERM).
Current behaviour when we issue waring but still proceeed and
return success is definitely not correct: masking out error
condition as well as making a slighly inconsistent FS where
attempt to access the file in question ends up in EBADF. See
linked DR for details.

MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D18584

show more ...

makefs: Calculate indirect block count properly for large files on ffs

When building an ffs image containing large file(s), space requirements
were calculated incorrectly yielding a bigger image tha

makefs: Calculate indirect block count properly for large files on ffs

When building an ffs image containing large file(s), space requirements
were calculated incorrectly yielding a bigger image than necessary.
The reason is that amount of indirect blocks estimation was done wrong:

- single indirect block was treated as it can hold just 12 data blocks
- nested indirect blocks were not taken into account at all

Add support for indirect blocks and fix another tiny bug with
underestimated space requirement for files with size between
(UFS_NDADDR-1)*blksz+fragsz ... (UFS_NDADDR)*blksz requesting N>1
fragments instead of a whole block.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D52120

show more ...

makefs: Fix undefined behaviour in ffs.c

Fix a UBSAN-reported error in `ffs_make_dirbuf` where a zero offset
gets applied to a NULL pointer.

Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD

makefs: Fix undefined behaviour in ffs.c

Fix a UBSAN-reported error in `ffs_make_dirbuf` where a zero offset
gets applied to a NULL pointer.

Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51037
Reviewed by: emaste, mckusick

show more ...

makefs: Remove redundant 'stampst' checks in several backends

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.

makefs: Remove redundant 'stampst' checks in several backends

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D49601

show more ...

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement memb

makefs: Support st_flags even on Linux

Linux's struct stat does not have an st_flags member, but we still want
to be able to create FreeBSD disk images with non-zero st_flags. Add a
replacement member to struct fsinode to store it on Linux.

Note that we add a fallback definition of HAVE_STRUCT_STAT_ST_FLAGS for
makefs.h since etdump reaches into makefs and won't have it defined in
its CFLAGS like makefs itself does, and on FreeBSD we don't provide an
nbtool_config.h.

Note also that strtofflags is currently a stub on Linux so this doesn't
yet properly mirror the mtree, but that will change in a future commit.

Reviewed by: emaste, markj
Differential Revision: https://reviews.freebsd.org/D50078

show more ...

Standardize the definition of a UFS dinode.

Each program that operates on UFS on-disk inodes defines its own
version of a dinode. They all (of necessity) define the same
layout but use different nam

Standardize the definition of a UFS dinode.

Each program that operates on UFS on-disk inodes defines its own
version of a dinode. They all (of necessity) define the same
layout but use different names. This change adds a definition of
a dinode (a union of a UFS1 on-disk inode and a UFS2 on-disk inode)
as well as a dinodep (a union of a pointer to a UFS1 on-disk inode
and a pointer to a UFS2 on-disk inode) in sys/ufs/ufs/dinode.h.
It then deletes the definitions of dinode and dinodep in all the
programs that operate on them and instead uses these standard
definitions.

No functional change intended.

MFC-after: 1 week

show more ...