random: Allow pure entropy sources to provide a min-entropy estimate

The current implementation of the NIST health tests assumes a
min-entropy estimate of one bit per sample, which is quite conserva

random: Allow pure entropy sources to provide a min-entropy estimate

The current implementation of the NIST health tests assumes a
min-entropy estimate of one bit per sample, which is quite conservative.
For so-called "pure" sources (e.g., virtio-random, TPM) it might be nice
to support larger estimates so that the tests catch failed devices more
quickly.

Thus:
- let each pure random source provide an estimate, so that downstreams
or driver implementors can override defaults if they want to;
- increase the default estimate for pure sources;
- for pure sources initialize the state machine at source registration
time.

Reviewed by: cem
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52232

show more ...

random: Make the entropy source registration interface more uniform

Most pure sources work under a "pull" model wherein a dedicated thread
polls the source at regular intervals (every 100ms). A cou

random: Make the entropy source registration interface more uniform

Most pure sources work under a "pull" model wherein a dedicated thread
polls the source at regular intervals (every 100ms). A couple of
sources, however, will instead call random_harvest_direct() to provide
entropy samples. Such sources were not calling random_source_register()
and thus weren't in the global random source list.

Modify "push" sources to use random_source_register() like other sources
do. Such sources omit an implementation of rs_read and are thus skipped
by the above-mentioned thread. This makes it easier to allow pure
sources to provide a min-entropy estimate in a uniform way.

Reviewed by: cem
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52229

show more ...

random: Make random_source definitions const

We can do so trivially, so make these tables read-only. No functional
change intended.

Reviewed by: cem, emaste
MFC after: 2 weeks
Sponsored by: Storms

random: Make random_source definitions const

We can do so trivially, so make these tables read-only. No functional
change intended.

Reviewed by: cem, emaste
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52003

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

random(4): Simplify RANDOM_LOADABLE

Simplify RANDOM_LOADABLE by removing the ability to unload a LOADABLE
random(4) implementation. This allows one-time random module selection
at boot, by loader(8

random(4): Simplify RANDOM_LOADABLE

Simplify RANDOM_LOADABLE by removing the ability to unload a LOADABLE
random(4) implementation. This allows one-time random module selection
at boot, by loader(8). Swapping modules on the fly doesn't seem
especially useful.

This removes the need to hold a lock over the sleepable module calls
read_random and read_random_uio.

init/deinit have been pulled out of random_algorithm entirely. Algorithms
can run their own sysinits to initialize; deinit is removed entirely, as
algorithms can not be unloaded. Algorithms should initialize at
SI_SUB_RANDOM:SI_ORDER_SECOND. In LOADABLE systems, algorithms install
a pointer to their local random_algorithm context in p_random_alg_context at
that time.

Go ahead and const'ify random_algorithm objects; there is no need to mutate
them at runtime.

LOADABLE kernel NULL checks are removed from random_harvestq by ordering
random_harvestq initialization at SI_SUB_RANDOM:SI_ORDER_THIRD, after
algorithm init. Prior to random_harvestq init, hc_harvest_mask is zero and
no events are forwarded to algorithms; after random_harvestq init, the
relevant pointers will already have been installed.

Remove the bulk of random_infra shim wrappers and instead expose the bare
function pointers in sys/random.h. In LOADABLE systems, read_random(9) et
al are just thin shim macros around invoking the associated function
pointer. We do not provide a registration system but instead expect
LOADABLE modules to register themselves at SI_SUB_RANDOM:SI_ORDER_SECOND.
An example is provided in randomdev.c, as used in the random_fortuna.ko
module.

Approved by: csprng(markm)
Discussed with: gordon
Differential Revision: https://reviews.freebsd.org/D22512

show more ...

random(4): De-export random_sources list

The internal datastructures do not need to be visible outside of
random_harvestq, and this helps ensure they are not misused.

No functional change.

Approve

random(4): De-export random_sources list

The internal datastructures do not need to be visible outside of
random_harvestq, and this helps ensure they are not misused.

No functional change.

Approved by: csprng(delphij, markm)
Differential Revision: https://reviews.freebsd.org/D22485

show more ...

random(4): Remove unused definitions

Approved by: csprng(gordon, markm)
Differential Revision: https://reviews.freebsd.org/D22481

MFHead @349234

Sponsored by: The FreeBSD Foundation

random(4): Generalize algorithm-independent APIs

At a basic level, remove assumptions about the underlying algorithm (such as
output block size and reseeding requirements) from the algorithm-indepen

random(4): Generalize algorithm-independent APIs

At a basic level, remove assumptions about the underlying algorithm (such as
output block size and reseeding requirements) from the algorithm-independent
logic in randomdev.c. Chacha20 does not have many of the restrictions that
AES-ICM does as a PRF (Pseudo-Random Function), because it has a cipher
block size of 512 bits. The motivation is that by generalizing the API,
Chacha is not penalized by the limitations of AES.

In READ_RANDOM_UIO, first attempt to NOWAIT allocate a large enough buffer
for the entire user request, or the maximal input we'll accept between
signal checking, whichever is smaller. The idea is that the implementation
of any randomdev algorithm is then free to divide up large requests in
whatever fashion it sees fit.

As part of this, two responsibilities from the "algorithm-generic" randomdev
code are pushed down into the Fortuna ra_read implementation (and any other
future or out-of-tree ra_read implementations):

1. If an algorithm needs to rekey every N bytes, it is responsible for
handling that in ra_read(). (I.e., Fortuna's 1MB rekey interval for AES
block generation.)

2. If an algorithm uses a block cipher that doesn't tolerate partial-block
requests (again, e.g., AES), it is also responsible for handling that in
ra_read().

Several APIs are changed from u_int buffer length to the more canonical
size_t. Several APIs are changed from taking a blockcount to a bytecount,
to permit PRFs like Chacha20 to directly generate quantities of output that
are not multiples of RANDOM_BLOCKSIZE (AES block size).

The Fortuna algorithm is changed to NOT rekey every 1MiB when in Chacha20
mode (kern.random.use_chacha20_cipher="1"). This is explicitly supported by
the math in FS&K §9.4 (Ferguson, Schneier, and Kohno; "Cryptography
Engineering"), as well as by their conclusion: "If we had a block cipher
with a 256-bit [or greater] block size, then the collisions would not
have been an issue at all."

For now, continue to break up reads into PAGE_SIZE chunks, as they were
before. So, no functional change, mostly.

Reviewed by: markm
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D20312

show more ...

MFHead @347527

Sponsored by: The FreeBSD Foundation

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for now,
restore the prior default.

Some of this is self-induced pain we should/could do better about; for
example, programmatic CI systems and VM managers should introduce entropy
from the host for individual VM instances. This is considered a future work
item.

On modern x86 and Power9 systems, this may be wholly unnecessary after
D19928 lands (even in the non-ideal case where early /boot/entropy is
unavailable), because they have fast hardware random sources available early
in boot. But D19928 is not yet landed and we have a host of architectures
which do not provide fast random sources.

This change adds several tunables and diagnostic sysctls, documented
thoroughly in UPDATING and sys/dev/random/random_infra.c.

PR: 230875 (reopens)
Reported by: adrian, jhb, imp, and probably others
Reviewed by: delphij, imp (earlier version), markm (earlier version)
Discussed with: adrian
Approved by: secteam(delphij)
Relnotes: yeah
Security: related
Differential Revision: https://reviews.freebsd.org/D19944

show more ...

random(4): Add is_random_seeded(9) KPI

The imagined use is for early boot consumers of random to be able to make
decisions based on whether random is available yet or not. One such
consumer seems t

random(4): Add is_random_seeded(9) KPI

The imagined use is for early boot consumers of random to be able to make
decisions based on whether random is available yet or not. One such
consumer seems to be __stack_chk_init(), which runs immediately after random
is initialized. A follow-up patch will attempt to address that.

Reported by: many
Reviewed by: delphij (except man page)
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D19926

show more ...

random(4): Block read_random(9) on initial seeding

read_random() is/was used, mostly without error checking, in a lot of
very sensitive places in the kernel -- including seeding the widely used
arc4

random(4): Block read_random(9) on initial seeding

read_random() is/was used, mostly without error checking, in a lot of
very sensitive places in the kernel -- including seeding the widely used
arc4random(9).

Most uses, especially arc4random(9), should block until the device is seeded
rather than proceeding with a bogus or empty seed. I did not spy any
obvious kernel consumers where blocking would be inappropriate (in the
sense that lack of entropy would be ok -- I did not investigate locking
angle thoroughly). In many instances, arc4random_buf(9) or that family
of APIs would be more appropriate anyway; that work was done in r345865.

A minor cleanup was made to the implementation of the READ_RANDOM function:
instead of using a variable-length array on the stack to temporarily store
all full random blocks sufficient to satisfy the requested 'len', only store
a single block on the stack. This has some benefit in terms of reducing
stack usage, reducing memcpy overhead and reducing devrandom output leakage
via the stack. Additionally, the stack block is now safely zeroed if it was
used.

One caveat of this change is that the kern.arandom sysctl no longer returns
zero bytes immediately if the random device is not seeded. This means that
FreeBSD-specific userspace applications which attempted to handle an
unseeded random device may be broken by this change. If such behavior is
needed, it can be replaced by the more portable getrandom(2) GRND_NONBLOCK
option.

On any typical FreeBSD system, entropy is persisted on read/write media and
used to seed the random device very early in boot, and blocking is never a
problem.

This change primarily impacts the behavior of /dev/random on embedded
systems with read-only media that do not configure "nodevice random". We
toggle the default from 'charge on blindly with no entropy' to 'block
indefinitely.' This default is safer, but may cause frustration. Embedded
system designers using FreeBSD have several options. The most obvious is to
plan to have a small writable NVRAM or NAND to persist entropy, like larger
systems. Early entropy can be fed from any loader, or by writing directly
to /dev/random during boot. Some embedded SoCs now provide a fast hardware
entropy source; this would also work for quickly seeding Fortuna. A 3rd
option would be creating an embedded-specific, more simplistic random
module, like that designed by DJB in [1] (this design still requires a small
rewritable media for forward secrecy). Finally, the least preferred option
might be "nodevice random", although I plan to remove this in a subsequent
revision.

To help developers emulate the behavior of these embedded systems on
ordinary workstations, the tunable kern.random.block_seeded_status was
added. When set to 1, it blocks the random device.

I attempted to document this change in random.4 and random.9 and ran into a
bunch of out-of-date or irrelevant or inaccurate content and ended up
rototilling those documents more than I intended to. Sorry. I think
they're in a better state now.

PR: 230875
Reviewed by: delphij, markm (earlier version)
Approved by: secteam(delphij), devrandom(markm)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19744

show more ...

random: Allow pure entropy sources to provide a min-entropy estimate

The current implementation of the NIST health tests assumes a
min-entropy estimate of one bit per sample, which is quite conserva

random: Allow pure entropy sources to provide a min-entropy estimate

The current implementation of the NIST health tests assumes a
min-entropy estimate of one bit per sample, which is quite conservative.
For so-called "pure" sources (e.g., virtio-random, TPM) it might be nice
to support larger estimates so that the tests catch failed devices more
quickly.

Thus:
- let each pure random source provide an estimate, so that downstreams
or driver implementors can override defaults if they want to;
- increase the default estimate for pure sources;
- for pure sources initialize the state machine at source registration
time.

Reviewed by: cem
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52232

show more ...

random: Make the entropy source registration interface more uniform

Most pure sources work under a "pull" model wherein a dedicated thread
polls the source at regular intervals (every 100ms). A cou

random: Make the entropy source registration interface more uniform

Most pure sources work under a "pull" model wherein a dedicated thread
polls the source at regular intervals (every 100ms). A couple of
sources, however, will instead call random_harvest_direct() to provide
entropy samples. Such sources were not calling random_source_register()
and thus weren't in the global random source list.

Modify "push" sources to use random_source_register() like other sources
do. Such sources omit an implementation of rs_read and are thus skipped
by the above-mentioned thread. This makes it easier to allow pure
sources to provide a min-entropy estimate in a uniform way.

Reviewed by: cem
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52229

show more ...

random: Make random_source definitions const

We can do so trivially, so make these tables read-only. No functional
change intended.

Reviewed by: cem, emaste
MFC after: 2 weeks
Sponsored by: Storms

random: Make random_source definitions const

We can do so trivially, so make these tables read-only. No functional
change intended.

Reviewed by: cem, emaste
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52003

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

random(4): Simplify RANDOM_LOADABLE

Simplify RANDOM_LOADABLE by removing the ability to unload a LOADABLE
random(4) implementation. This allows one-time random module selection
at boot, by loader(8

random(4): Simplify RANDOM_LOADABLE

Simplify RANDOM_LOADABLE by removing the ability to unload a LOADABLE
random(4) implementation. This allows one-time random module selection
at boot, by loader(8). Swapping modules on the fly doesn't seem
especially useful.

This removes the need to hold a lock over the sleepable module calls
read_random and read_random_uio.

init/deinit have been pulled out of random_algorithm entirely. Algorithms
can run their own sysinits to initialize; deinit is removed entirely, as
algorithms can not be unloaded. Algorithms should initialize at
SI_SUB_RANDOM:SI_ORDER_SECOND. In LOADABLE systems, algorithms install
a pointer to their local random_algorithm context in p_random_alg_context at
that time.

Go ahead and const'ify random_algorithm objects; there is no need to mutate
them at runtime.

LOADABLE kernel NULL checks are removed from random_harvestq by ordering
random_harvestq initialization at SI_SUB_RANDOM:SI_ORDER_THIRD, after
algorithm init. Prior to random_harvestq init, hc_harvest_mask is zero and
no events are forwarded to algorithms; after random_harvestq init, the
relevant pointers will already have been installed.

Remove the bulk of random_infra shim wrappers and instead expose the bare
function pointers in sys/random.h. In LOADABLE systems, read_random(9) et
al are just thin shim macros around invoking the associated function
pointer. We do not provide a registration system but instead expect
LOADABLE modules to register themselves at SI_SUB_RANDOM:SI_ORDER_SECOND.
An example is provided in randomdev.c, as used in the random_fortuna.ko
module.

Approved by: csprng(markm)
Discussed with: gordon
Differential Revision: https://reviews.freebsd.org/D22512

show more ...

random(4): De-export random_sources list

The internal datastructures do not need to be visible outside of
random_harvestq, and this helps ensure they are not misused.

No functional change.

Approve

random(4): De-export random_sources list

The internal datastructures do not need to be visible outside of
random_harvestq, and this helps ensure they are not misused.

No functional change.

Approved by: csprng(delphij, markm)
Differential Revision: https://reviews.freebsd.org/D22485

show more ...

random(4): Remove unused definitions

Approved by: csprng(gordon, markm)
Differential Revision: https://reviews.freebsd.org/D22481

MFHead @349234

Sponsored by: The FreeBSD Foundation

random(4): Generalize algorithm-independent APIs

At a basic level, remove assumptions about the underlying algorithm (such as
output block size and reseeding requirements) from the algorithm-indepen

random(4): Generalize algorithm-independent APIs

At a basic level, remove assumptions about the underlying algorithm (such as
output block size and reseeding requirements) from the algorithm-independent
logic in randomdev.c. Chacha20 does not have many of the restrictions that
AES-ICM does as a PRF (Pseudo-Random Function), because it has a cipher
block size of 512 bits. The motivation is that by generalizing the API,
Chacha is not penalized by the limitations of AES.

In READ_RANDOM_UIO, first attempt to NOWAIT allocate a large enough buffer
for the entire user request, or the maximal input we'll accept between
signal checking, whichever is smaller. The idea is that the implementation
of any randomdev algorithm is then free to divide up large requests in
whatever fashion it sees fit.

As part of this, two responsibilities from the "algorithm-generic" randomdev
code are pushed down into the Fortuna ra_read implementation (and any other
future or out-of-tree ra_read implementations):

1. If an algorithm needs to rekey every N bytes, it is responsible for
handling that in ra_read(). (I.e., Fortuna's 1MB rekey interval for AES
block generation.)

2. If an algorithm uses a block cipher that doesn't tolerate partial-block
requests (again, e.g., AES), it is also responsible for handling that in
ra_read().

Several APIs are changed from u_int buffer length to the more canonical
size_t. Several APIs are changed from taking a blockcount to a bytecount,
to permit PRFs like Chacha20 to directly generate quantities of output that
are not multiples of RANDOM_BLOCKSIZE (AES block size).

The Fortuna algorithm is changed to NOT rekey every 1MiB when in Chacha20
mode (kern.random.use_chacha20_cipher="1"). This is explicitly supported by
the math in FS&K §9.4 (Ferguson, Schneier, and Kohno; "Cryptography
Engineering"), as well as by their conclusion: "If we had a block cipher
with a 256-bit [or greater] block size, then the collisions would not
have been an issue at all."

For now, continue to break up reads into PAGE_SIZE chunks, as they were
before. So, no functional change, mostly.

Reviewed by: markm
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D20312

show more ...

MFHead @347527

Sponsored by: The FreeBSD Foundation

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for now,
restore the prior default.

Some of this is self-induced pain we should/could do better about; for
example, programmatic CI systems and VM managers should introduce entropy
from the host for individual VM instances. This is considered a future work
item.

On modern x86 and Power9 systems, this may be wholly unnecessary after
D19928 lands (even in the non-ideal case where early /boot/entropy is
unavailable), because they have fast hardware random sources available early
in boot. But D19928 is not yet landed and we have a host of architectures
which do not provide fast random sources.

This change adds several tunables and diagnostic sysctls, documented
thoroughly in UPDATING and sys/dev/random/random_infra.c.

PR: 230875 (reopens)
Reported by: adrian, jhb, imp, and probably others
Reviewed by: delphij, imp (earlier version), markm (earlier version)
Discussed with: adrian
Approved by: secteam(delphij)
Relnotes: yeah
Security: related
Differential Revision: https://reviews.freebsd.org/D19944

show more ...

random(4): Add is_random_seeded(9) KPI

The imagined use is for early boot consumers of random to be able to make
decisions based on whether random is available yet or not. One such
consumer seems t

random(4): Add is_random_seeded(9) KPI

The imagined use is for early boot consumers of random to be able to make
decisions based on whether random is available yet or not. One such
consumer seems to be __stack_chk_init(), which runs immediately after random
is initialized. A follow-up patch will attempt to address that.

Reported by: many
Reviewed by: delphij (except man page)
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D19926

show more ...