Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align thin

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.

In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.

Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947

show more ...

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https:/

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48486

show more ...

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: im

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48485

show more ...

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version st

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version standardizes on not using explicit casts and instead
preserving the original type. A couple of tweaks were required to
make this work.

Reviewed by: brooks, kib, markj
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D48450

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

i386: clean up empty lines in .c and .h files

Merge ^/head r357408 through r357661.

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it bre

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.

PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512

show more ...

Merge ^/head r343320 through r343570.

i386: Merge PAE and non-PAE pmaps into same kernel.

Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at c

i386: Merge PAE and non-PAE pmaps into same kernel.

Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at cold time depending on the CPU features. The vm_paddr_t is
always 64bit now. As result, nx bit can be used on all capable CPUs.

Option PAE only affects the bus_addr_t: it is still 32bit for non-PAE
configs, for drivers compatibility. Kernel layout, esp. max kernel
address, low memory PDEs and max user address (same as trampoline
start) are now same for PAE and for non-PAE regardless of the type of
page tables used.

Non-PAE kernel (when using PAE pagetables) can handle physical memory
up to 24G now, larger memory requires re-tuning the KVA consumers and
instead the code caps the maximum at 24G. Unfortunately, a lot of
drivers do not use busdma(9) properly so by default even 4G barrier is
not easy. There are two tunables added: hw.above4g_allow and
hw.above24g_allow, the first one is kept enabled for now to evaluate
the status on HEAD, second is only for dev use.

i386 now creates three freelists if there is any memory above 4G, to
allow proper bounce pages allocation. Also, VM_KMEM_SIZE_SCALE changed
from 3 to 1.

The PAE_TABLES kernel config option is retired.

In collaboarion with: pho
Discussed with: emaste
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18894

show more ...

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align thin

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.

In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.

Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947

show more ...

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https:/

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48486

show more ...

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: im

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48485

show more ...

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version st

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version standardizes on not using explicit casts and instead
preserving the original type. A couple of tweaks were required to
make this work.

Reviewed by: brooks, kib, markj
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D48450

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

i386: clean up empty lines in .c and .h files

Merge ^/head r357408 through r357661.

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it bre

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.

PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512

show more ...

Merge ^/head r343320 through r343570.

i386: Merge PAE and non-PAE pmaps into same kernel.

Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at c

i386: Merge PAE and non-PAE pmaps into same kernel.

Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at cold time depending on the CPU features. The vm_paddr_t is
always 64bit now. As result, nx bit can be used on all capable CPUs.

Option PAE only affects the bus_addr_t: it is still 32bit for non-PAE
configs, for drivers compatibility. Kernel layout, esp. max kernel
address, low memory PDEs and max user address (same as trampoline
start) are now same for PAE and for non-PAE regardless of the type of
page tables used.

Non-PAE kernel (when using PAE pagetables) can handle physical memory
up to 24G now, larger memory requires re-tuning the KVA consumers and
instead the code caps the maximum at 24G. Unfortunately, a lot of
drivers do not use busdma(9) properly so by default even 4G barrier is
not easy. There are two tunables added: hw.above4g_allow and
hw.above24g_allow, the first one is kept enabled for now to evaluate
the status on HEAD, second is only for dev use.

i386 now creates three freelists if there is any memory above 4G, to
allow proper bounce pages allocation. Also, VM_KMEM_SIZE_SCALE changed
from 3 to 1.

The PAE_TABLES kernel config option is retired.

In collaboarion with: pho
Discussed with: emaste
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18894

show more ...

i386 4/4G split.

The change makes the user and kernel address spaces on i386
independent, giving each almost the full 4G of usable virtual addresses
except for one PDE at top used for trampoline and

i386 4/4G split.

The change makes the user and kernel address spaces on i386
independent, giving each almost the full 4G of usable virtual addresses
except for one PDE at top used for trampoline and per-CPU trampoline
stacks, and system structures that must be always mapped, namely IDT,
GDT, common TSS and LDT, and process-private TSS and LDT if allocated.

By using 1:1 mapping for the kernel text and data, it appeared
possible to eliminate assembler part of the locore.S which bootstraps
initial page table and KPTmap. The code is rewritten in C and moved
into the pmap_cold(). The comment in vmparam.h explains the KVA
layout.

There is no PCID mechanism available in protected mode, so each
kernel/user switch forth and back completely flushes the TLB, except
for the trampoline PTD region. The TLB invalidations for userspace
becomes trivial, because IPI handlers switch page tables. On the other
hand, context switches no longer need to reload %cr3.

copyout(9) was rewritten to use vm_fault_quick_hold(). An issue for
new copyout(9) is compatibility with wiring user buffers around sysctl
handlers. This explains two kind of locks for copyout ptes and
accounting of the vslock() calls. The vm_fault_quick_hold() AKA slow
path, is only tried after the 'fast path' failed, which temporary
changes mapping to the userspace and copies the data to/from small
per-cpu buffer in the trampoline. If a page fault occurs during the
copy, it is short-circuit by exception.s to not even reach C code.

The change was motivated by the need to implement the Meltdown
mitigation, but instead of KPTI the full split is done. The i386
architecture already shows the sizing problems, in particular, it is
impossible to link clang and lld with debugging. I expect that the
issues due to the virtual address space limits would only exaggerate
and the split gives more liveness to the platform.

Tested by: pho
Discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D14633

show more ...

i386: Bump KSTACK_PAGES default to match amd64

Logically, extend r286288 to cover all threads, by default.

The world has largely moved on from i386. Most FreeBSD users and developers
test on amd64

i386: Bump KSTACK_PAGES default to match amd64

Logically, extend r286288 to cover all threads, by default.

The world has largely moved on from i386. Most FreeBSD users and developers
test on amd64 hardware. For better or worse, we have written a non-trivial
amount of kernel code that relies on stacks larger than 8 kB, and it "just
works" on amd64, so there has been little incentive to shrink it.

amd64 had its KSTACK_PAGES bumped to 4 back in Peter's initial AMD64 commit,
r114349, in 2003. Since that time, i386 has limped along on a stack half
the size. We've even observed the stack overflows years ago, but neglected
to fix the issue; see the 20121223 and 20150728 entries in UPDATING.

If anyone is concerned with this change, I suggest they configure their
AMD64 kernels with KSTACK_PAGES 2 and fix the fallout there first. Eugene
has identified a list of high stack usage functions in the first PR below.

PR: 219476, 224218
Reported by: eugen@, Shreesh Holla <hshreesh AT yahoo.com>
Relnotes: maybe
Sponsored by: Dell EMC Isilon

show more ...

Merge ^/head r325999 through r326131.