ffs: fix build with GEOM_LABEL and without FFS, e.g. MINIMAL

The root of vfs.ffs sysctl tree was declared in ffs_alloc.c. The
1111a44301da started to use the root in ffs_subr.c. However, ffs_subr.

ffs: fix build with GEOM_LABEL and without FFS, e.g. MINIMAL

The root of vfs.ffs sysctl tree was declared in ffs_alloc.c. The
1111a44301da started to use the root in ffs_subr.c. However, ffs_subr.c
may be included in kernels that do not have FFS in their config. Such
kernel won't link after 1111a44301da.

Fixes: 1111a44301da39d7b7459c784230e1405e8980f8

show more ...

ffs_reallocblks(): ensure that pref cg is valid

ffs_blkpref_ufsX() must return in-range pref frag number, otherwise
calculated cg index is out of range for fs, causing out of range
accesses to the s

ffs_reallocblks(): ensure that pref cg is valid

ffs_blkpref_ufsX() must return in-range pref frag number, otherwise
calculated cg index is out of range for fs, causing out of range
accesses to the structures sized by the number of cg, e.g. the
fs_maxcluster[] array in ffs_clusteralloc().

The easiest way to trigger it is to overflow the volume.

In collaboration with: pho
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC afer: 1 week
Differential revision: https://reviews.freebsd.org/D48378

show more ...

ffs_blkpref_ufsX(): do not iterate over startcg twice

Tested by: pho
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D

ffs_blkpref_ufsX(): do not iterate over startcg twice

Tested by: pho
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D48378

show more ...

Increase UFS/FFS maximum link count from 32767 to 65530.

The link count for a UFS/FFS inode is stored in a signed 16-bit
integer. Thus the maximum link count has been 32767.

This limit has been rec

Increase UFS/FFS maximum link count from 32767 to 65530.

The link count for a UFS/FFS inode is stored in a signed 16-bit
integer. Thus the maximum link count has been 32767.

This limit has been recently hit by the poudriere build system when
doing a ports build as it needs one directory per port and the
number of ports recently passed 32767.

A long-term solution would be to use one of the spare 32-bit fields
in the inode to store the link count. However, the UFS1 format does
not have a spare and adding the spare in UFS2 would make it hard
to make it compatible when running on older kernels that use the
original link count field. So this patch uses the much simpler
approach of changing the existing link count field from a signed
16-bit value to an unsigned 16-bit value. It has the fewest lines
of code changes. The only thing that changes is the type in the
dinode and inode structures and the definition of UFS_LINK_MAX. It
has the added benefit that it works with both UFS1 and UFS2.

It allows easy backward compatibility. Indeed it is backward
compatibility that is the primary reason to go with this approach.
If a filesystem with the new organization is mounted on an older
kernel, it still needs to work. Thus if we move the new link count
to a new field, we still need to maintain the old link count as
best as possible even when running on a kernel that knows about the
larger link counts. And we would have to carry this overhead for
the indefinite future.

If we have a new link-count field, we will have to add a new
filesystem flag to indicate that we are running with larger link
counts. We will also need to add of one of the new-feature flags
to say that we have larger link counts. Older kernels clear the
new-feature flags that they do not know about, so when a filesystem
is used on an older kernel and then moved back to a newer one, the
newer one will know that the new link counts have not been maintained
and that it will be necessary to run a full fsck on the filesystem
to correct the link counts before it can be mounted.

With this change, older kernels will generally work with the bigger
counts. While it will not itself allow the link count to exceed
32767, it will have no problem working with inodes that have a link
count greater than 32767. Since it tests that i_nlink <= UFS_LINK_MAX,
counts that are bigger than 32767 will appear negative, so will
still pass the test. Of course, if they ever drop below 32767, they
will no longer be able to exceed 32767. The one issue is if the
link count ever exceeds 65535 then it will wrap to zero and the
older kernel will be none the wiser. But this corner case is likely
to be very rare since these kernels and the applications running
on them do not expect to be able to get link counts over 32767. And
over time, the use of new filesystems on older kernels will become
rarer and rarer.

Reported-by: Mark Millard running poudriere on the ports tree
Reviewed-by: kib, olce.freebsd_certner.fr
Tested-by: Peter Holm, Mark Millard
MFC-after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D42767

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Set UFS/FFS file type to snapshot before changing its block pointers.

A UFS/FFS snapshot file is identified with the SF_SNAPSHOT
flag to identify it as a snapshot. This flag needs to be
set before s

Set UFS/FFS file type to snapshot before changing its block pointers.

A UFS/FFS snapshot file is identified with the SF_SNAPSHOT
flag to identify it as a snapshot. This flag needs to be
set before setting some of its block pointers to the special
values BLK_SNAP and BLK_NOCOPY. If the snapshot creation fails
and we call VOP_REMOVE(), the SF_SNAPSHOT flag will let the
remove routine know that the special block pointer values need
to be rolled back before attempting deletion of the file.

Also ensure that an fsck is required after setting superblock
values in the ffs_checkcgintegrity() routine.

Reported-by: Peter Holm
Tested-by: Peter Holm
MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Optimize operations on UFS/FFS filesystems with bad cylinder group(s).

If a UFS/FFS filesystem develops a broken cylinder group (which is
usually detected when its check hash fails), that cylinder g

Optimize operations on UFS/FFS filesystems with bad cylinder group(s).

If a UFS/FFS filesystem develops a broken cylinder group (which is
usually detected when its check hash fails), that cylinder group
will not be usable until the filesystem has been unmounted and fsck
has been run to repair it. On the first attempt to to allocate
resources from the broken cylinder group, its available resources
are set to zero in the superblock summary information. Since it
will appear to have no resources available, no further calls will
be made to allocate resources from it. When resources are freed to
the broken cylinder group, the resource free routines will find the
cylinder group unusable so the resource will simply be discarded
and thus will not show up in the superblock summary information
until they are recovered by fsck.

Reported-by: Peter Holm
Tested-by: Peter Holm
MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Cleanups to UFS/FFS ffs_checkblk().

Rename to ffs_checkfreeblk() to better describe that it is checking
to find out if a block or fragment is free. Clarify its implementation.

No functional change

Cleanups to UFS/FFS ffs_checkblk().

Rename to ffs_checkfreeblk() to better describe that it is checking
to find out if a block or fragment is free. Clarify its implementation.

No functional change intended.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Rate limit kernel UFS/FFS cylinder group check-hash error messages.

When a large file is deleted or a large number of files are deleted,
even a single cylinder group with a bad check hash can genera

Rate limit kernel UFS/FFS cylinder group check-hash error messages.

When a large file is deleted or a large number of files are deleted,
even a single cylinder group with a bad check hash can generate
thousands of check-hash warnings. As with other filesystem messages
such as out-of-space, print a maximum of one check-hash error per
second. Note the limit is per filesystem. If two filesystems have
cylinder group(s) with a bad check hash, each will print a maximum
of one check-hash error message per second.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Handle UFS/FFS file deletion from cylinder groups with check-hash failure.

When a file is deleted, its blocks need to be put back in the free
block list and its inode needs to be put back in the ino

Handle UFS/FFS file deletion from cylinder groups with check-hash failure.

When a file is deleted, its blocks need to be put back in the free
block list and its inode needs to be put back in the inode free list.
These lists reside in cylinder-group maps. If either some of its blocks
or its inode reside in a cylinder-group map with a bad check hash
it is not possible to free the associated resource. Since the cylinder
group cannot be repaired until the filesystem is unmounted these
resources cannot be freed. They simply accumulate in memory. And
any attempt to unmount the filesystem loops forever trying to flush them.

With this change, the resource update claims to succeed so that the
file deletion can successfully complete. The filesystem is marked as
requiring an fsck so that before the next time that the filesystem is
mounted, the offending cylinder groups are reconstructed causing the
lost resources to be reclaimed.

A better solution would be to downgrade the filesystem to read-only,
but that capability is not currently implemented.

Reported-by: Peter Holm
Tested-by: Peter Holm
MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_t.
Where other kernel interfaces allow, migrate u_long to uint64_t.

No functional changes intended.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Comment cleanup.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

vfs: use __enum_uint8 for vtype and vstate

This whacks hackery around only reading v_type once.

Bump __FreeBSD_version to 1400093

Add the ability to adjust directory depths to background fsck_ffs(8).

Commit fe5e6e2 improved FFS directory placement when creating new
directories. It is done by keeping track of the depth of direc

Add the ability to adjust directory depths to background fsck_ffs(8).

Commit fe5e6e2 improved FFS directory placement when creating new
directories. It is done by keeping track of the depth of directories
in the filesystem and placing those lower in the tree closer together
while spreading out those higher in the tree.

Fsck_ffs(8) checks these depths and if incorrect adjusts them to
their correct value. When running in background fsck_ffs(8) needs
to be able to make an adjustment to the depth. This commit adds
the sysctl to make such an adjustment and adds the code to fsck_ffs(8)
to use the new sysctl.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In thos

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In those days
large disks had at most 100 cylinder groups and often as few as 10-20.
Modern multi-terrabyte disks have thousands of cylinder groups. The
original algorithm does not handle these large sizes well. This change
attempts to expand the scope of the original algorithm to work well
with these much larger disks while still retaining the properties
of the original algorithm for small disks.

The filesystem implementation is divided into policy routines and
implementation routines. The policy routines can be changed in any
way desired without risk of corrupting the filesystem. The policy
requests are handled by the implementation layer. If the policy
asks for an available resource, it is granted. But if it asks for
an already in-use resource, then the implementation will provide
an available one nearby the request. Thus it is impossible for a
policy to double allocate. This change is limited to the policy
implementation.

This change updates the ffs_dirpref() routine which is responsible
for selecting the cylinder group into which a new directory should
be placed. If we are near the root of the filesystem we aim to
spread them out as much as possible. As we descend deeper from the
root we cluster them closer together around their parent as we
expect them to be more closely interactive. Higher-level directories
like usr/src/sys and usr/src/bin should be separated while the
directories in these areas are more likely to be accessed together
so should be closer. And directories within commands or kernel
subsystems should be closer still.

We pick a range of cylinder groups around the cylinder group of the
directory in which we are being created. The size of the range for
our search is based on our depth from the root of our filesystem.
We then probe that range based on how many directories are already
present. The first new directory is at 1/2 (middle) of the range;
the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8,
5/8, 7/8, 1/16, 3/16, 5/16, etc.

It is desirable to store the depth of a directory in its on-disk
inode so that it is available when we need it. We add a new field
di_dirdepth to track the depth of each directory. Because there are
few spare fields left in the inode, we choose to share an existing
field in the inode rather than having one of our own. Specifically
we create a union with the di_freelink field. The di_freelink field
is used to track inodes that have been unlinked but remain referenced.
It is not needed until a rmdir(2) operation has been done on a
directory. At that point, the directory has no contents and even
if it is kept active as a current directory is no longer able to
have any new directories or files created in it. Thus the use of
di_dirdepth and di_freelink will never coincide.

Reported by: Timo Voelker
Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39246

show more ...

Explicitly initialize rather than reading newly allocated UFS inodes.

The function ffs_vgetf() is used to find or load UFS inodes into a
vnode. It first looks up the inode and if found in the cache

Explicitly initialize rather than reading newly allocated UFS inodes.

The function ffs_vgetf() is used to find or load UFS inodes into a
vnode. It first looks up the inode and if found in the cache its
vnode is returned. If it is not already in the cache, a new vnode
is allocated and its associated inode read in from the disk. The
read is done even for inodes that are being initially created.
The contents for the inode on the disk are assumed to be empty. If
the on-disk contents had been corrupted either due to a hardware
glitch or an agent deliberately trying to exploit the system, the
UFS code could panic from the unexpected partially-allocated inode.

Rather then having fsck_ffs(8) verify that all unallocated inodes
are properly empty, it is easier and quicker to add a flag to
ffs_vgetf() to indicate that the request is for a newly allocated
inode. When set, the disk read is skipped and the inode is set to
its expected empty (zero'ed out) initial state. As a side benefit,
an unneeded disk I/O is avoided.

Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation

show more ...

Rewrite function definitions in the UFS/FFS code base with identifier lists.

The K&R style in UFS and other places in the tree's days are numbered
as this syntax is removed in C2x proposal N2432:

Rewrite function definitions in the UFS/FFS code base with identifier lists.

The K&R style in UFS and other places in the tree's days are numbered
as this syntax is removed in C2x proposal N2432:

https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2432.pdf

Though running to nearly 6000 lines of diffs this update should cause
no functional change to the code.

Requested by: Warner Losh
MFC after: 2 weeks

show more ...

ffs(3): Fix a common typo in source code comments

- s/quadradically/quadratically/

Obtained from: NetBSD
MFC after: 3 days

ffs: lock buffers after snaplk with LK_NOWITNESS

Reviewed by: mckusick
Discussed with: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://revi

ffs: lock buffers after snaplk with LK_NOWITNESS

Reviewed by: mckusick
Discussed with: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34073

show more ...

ffs_alloc.c: order includes alphabetically

Reviewed by: mckusick
Discussed with: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.fr

ffs_alloc.c: order includes alphabetically

Reviewed by: mckusick
Discussed with: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34073

show more ...

Revert "ffs(3): Fix a typo in a sysctl description"

It should be

- s/contigous/contiguous/ not continuous

Reported by: tuexen@

This reverts commit 42efe994ece337929b6f9937829f7970e620db9b.

ffs(3): Fix a typo in a sysctl description

- s/contigous/continuous/

MFC after: 3 days

ffs: Remove assertions about locked um_devvp in several places

Namely, ffs_blkfree_cg(), and ffs_flushfiles().

Reported and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MF

ffs: Remove assertions about locked um_devvp in several places

Namely, ffs_blkfree_cg(), and ffs_flushfiles().

Reported and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32761

show more ...