packages: Use LIB_PACKAGE for zfs and ufs packages

Commit f9513c334ff7 moved the shared libraries for ZFS and UFS into
separate packages (libzfs and libufs), which resulted in a rather
large number

packages: Use LIB_PACKAGE for zfs and ufs packages

Commit f9513c334ff7 moved the shared libraries for ZFS and UFS into
separate packages (libzfs and libufs), which resulted in a rather
large number of packages being created, e.g. for ZFS:

FreeBSD-libzfs
FreeBSD-libzfs-dbg
FreeBSD-libzfs-dbg-lib32
FreeBSD-libzfs-dev
FreeBSD-libzfs-dev-lib32
FreeBSD-libzfs-lib32
FreeBSD-zfs
FreeBSD-zfs-dbg
FreeBSD-zfs-dbg-lib32
FreeBSD-zfs-dev
FreeBSD-zfs-dev-lib32
FreeBSD-zfs-lib32
FreeBSD-zfs-man

Use LIB_PACKAGE instead, which significantly reduces the number of
packages:

FreeBSD-zfs
FreeBSD-zfs-dbg
FreeBSD-zfs-dbg-lib32
FreeBSD-zfs-dev
FreeBSD-zfs-dev-lib32
FreeBSD-zfs-lib
FreeBSD-zfs-lib32
FreeBSD-zfs-man

MFC after: 3 days
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D52416

show more ...

add packages for libufs, libzfs

currently FreeBSD-utilities depends on both FreeBSD-ufs and FreeBSD-zfs.
this is not desirable, because those are both relatively large packages
and the user may want

add packages for libufs, libzfs

currently FreeBSD-utilities depends on both FreeBSD-ufs and FreeBSD-zfs.
this is not desirable, because those are both relatively large packages
and the user may want to remove one or the other – or perhaps both, e.g.
in a jail or embedded system.

the reason for this dependency is that fstyp(8), which is in
FreeBSD-utilities, links both libufs and libzfs. FreeBSD-utilities is
the correct place for fstyp, so we don't want to move that.

instead, add two new packages: libufs contains libufs, and libzfs
contains libzfs plus the ZFS libraries it depends on: libavl, libnvpair,
libspl, libtpool, libumem, libuutil, libzfs_core and libzutil.

with this change, it is possible to remove FreeBSD-ufs and/or
FreeBSD-zfs while leaving FreeBSD-libufs, FreeBSD-libzfs and
FreeBSD-utilities installed.

Reviewed by: manu, des
Approved by: des (mentor)
Differential Revision: https://reviews.freebsd.org/D50148

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

Ensure I/O buffers in libufs(3) are 128-byte aligned.

Various disk controllers require their buffers to be aligned to a
cache-line size (128 bytes). For buffers allocated in structures,
ensure that

Ensure I/O buffers in libufs(3) are 128-byte aligned.

Various disk controllers require their buffers to be aligned to a
cache-line size (128 bytes). For buffers allocated in structures,
ensure that they are 128-byte aligned. Use aligned_malloc to allocate
memory to ensure that the returned memory is 128-byte aligned.

While we are here, we replace the dynamically allocated inode buffer
with a buffer allocated in the uufsd structure just as the superblock
and cylinder group buffers do.

This can be removed if/when the kernel is fixed. Because this problem
has existed on one I/O subsystem or another since the 1990's, we
are probably stuck with dealing with it forever.

The problem most recent showed up in Azure, see:
https://reviews.freebsd.org/D41728
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267654
Before these fixes were applied, it was confirmed that the changes
in this commit also fixed the issue in Azure.

Reviewed-by: Warner Losh, kib
Tested-by: Souradeep Chakrabarti of Microsoft (earlier version)
PR: 267654
Differential Revision: https://reviews.freebsd.org/D41724

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

pkgbase: Put ufs related tools and lib in their own package

It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by: mckusick
Differential Rev

pkgbase: Put ufs related tools and lib in their own package

It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by: mckusick
Differential Revision: https://reviews.freebsd.org/D36227

show more ...

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, ha

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation

show more ...

Merge ^/head r351732 through r352104.

pkgbase: Put a lot of binaries and lib in FreeBSD-runtime

All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
Fre

pkgbase: Put a lot of binaries and lib in FreeBSD-runtime

All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21503

show more ...

MFHead @349234

Sponsored by: The FreeBSD Foundation

Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR: 229763
Submitted by: Yosh

Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR: 229763
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision: https://reviews.freebsd.org/D20193

show more ...

Merge ^/head r340368 through r340426.

In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getin

In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.

Convert the utilities that had been using the undocumented
interface to use the new documented interface.

No functional change (as for now the libufs library does not
do inode check-hashes).

Reviewed by: kib
Tested by: Peter Holm
Sponsored by: Netflix

show more ...

packages: Use LIB_PACKAGE for zfs and ufs packages

Commit f9513c334ff7 moved the shared libraries for ZFS and UFS into
separate packages (libzfs and libufs), which resulted in a rather
large number

packages: Use LIB_PACKAGE for zfs and ufs packages

Commit f9513c334ff7 moved the shared libraries for ZFS and UFS into
separate packages (libzfs and libufs), which resulted in a rather
large number of packages being created, e.g. for ZFS:

FreeBSD-libzfs
FreeBSD-libzfs-dbg
FreeBSD-libzfs-dbg-lib32
FreeBSD-libzfs-dev
FreeBSD-libzfs-dev-lib32
FreeBSD-libzfs-lib32
FreeBSD-zfs
FreeBSD-zfs-dbg
FreeBSD-zfs-dbg-lib32
FreeBSD-zfs-dev
FreeBSD-zfs-dev-lib32
FreeBSD-zfs-lib32
FreeBSD-zfs-man

Use LIB_PACKAGE instead, which significantly reduces the number of
packages:

FreeBSD-zfs
FreeBSD-zfs-dbg
FreeBSD-zfs-dbg-lib32
FreeBSD-zfs-dev
FreeBSD-zfs-dev-lib32
FreeBSD-zfs-lib
FreeBSD-zfs-lib32
FreeBSD-zfs-man

MFC after: 3 days
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D52416

show more ...

add packages for libufs, libzfs

currently FreeBSD-utilities depends on both FreeBSD-ufs and FreeBSD-zfs.
this is not desirable, because those are both relatively large packages
and the user may want

add packages for libufs, libzfs

currently FreeBSD-utilities depends on both FreeBSD-ufs and FreeBSD-zfs.
this is not desirable, because those are both relatively large packages
and the user may want to remove one or the other – or perhaps both, e.g.
in a jail or embedded system.

the reason for this dependency is that fstyp(8), which is in
FreeBSD-utilities, links both libufs and libzfs. FreeBSD-utilities is
the correct place for fstyp, so we don't want to move that.

instead, add two new packages: libufs contains libufs, and libzfs
contains libzfs plus the ZFS libraries it depends on: libavl, libnvpair,
libspl, libtpool, libumem, libuutil, libzfs_core and libzutil.

with this change, it is possible to remove FreeBSD-ufs and/or
FreeBSD-zfs while leaving FreeBSD-libufs, FreeBSD-libzfs and
FreeBSD-utilities installed.

Reviewed by: manu, des
Approved by: des (mentor)
Differential Revision: https://reviews.freebsd.org/D50148

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

Ensure I/O buffers in libufs(3) are 128-byte aligned.

Various disk controllers require their buffers to be aligned to a
cache-line size (128 bytes). For buffers allocated in structures,
ensure that

Ensure I/O buffers in libufs(3) are 128-byte aligned.

Various disk controllers require their buffers to be aligned to a
cache-line size (128 bytes). For buffers allocated in structures,
ensure that they are 128-byte aligned. Use aligned_malloc to allocate
memory to ensure that the returned memory is 128-byte aligned.

While we are here, we replace the dynamically allocated inode buffer
with a buffer allocated in the uufsd structure just as the superblock
and cylinder group buffers do.

This can be removed if/when the kernel is fixed. Because this problem
has existed on one I/O subsystem or another since the 1990's, we
are probably stuck with dealing with it forever.

The problem most recent showed up in Azure, see:
https://reviews.freebsd.org/D41728
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267654
Before these fixes were applied, it was confirmed that the changes
in this commit also fixed the issue in Azure.

Reviewed-by: Warner Losh, kib
Tested-by: Souradeep Chakrabarti of Microsoft (earlier version)
PR: 267654
Differential Revision: https://reviews.freebsd.org/D41724

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

pkgbase: Put ufs related tools and lib in their own package

It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by: mckusick
Differential Rev

pkgbase: Put ufs related tools and lib in their own package

It's not really useful in a jail or in a mdroot or even if a users
wants to do a full zfs machine.

Reviewed by: mckusick
Differential Revision: https://reviews.freebsd.org/D36227

show more ...

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, ha

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation

show more ...

Merge ^/head r351732 through r352104.

pkgbase: Put a lot of binaries and lib in FreeBSD-runtime

All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
Fre

pkgbase: Put a lot of binaries and lib in FreeBSD-runtime

All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21503

show more ...

MFHead @349234

Sponsored by: The FreeBSD Foundation

Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR: 229763
Submitted by: Yosh

Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR: 229763
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision: https://reviews.freebsd.org/D20193

show more ...

Merge ^/head r340368 through r340426.