amd64: define and use STACKALIGN and REDZONE_SZ

Reviewed by: brooks, emaste, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D55151

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align thin

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.

In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.

Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947

show more ...

amd64: bump sleepq hash size to 2048

This is the most contended lock type during the first hour of -j 104 poudriere.

Drops significantly with the change.

Note there are suspicous acquires which mo

amd64: bump sleepq hash size to 2048

This is the most contended lock type during the first hour of -j 104 poudriere.

Drops significantly with the change.

Note there are suspicous acquires which most likely don't need to
happen, artificially exacerbating tehe problem..

show more ...

amd64: Remove support for "nooptions SMP"

It does not appear to get much, if any, testing, and doesn't seem to be
worth the maintenance overhead. Virtually all amd64 hardware has
multiple cores. T

amd64: Remove support for "nooptions SMP"

It does not appear to get much, if any, testing, and doesn't seem to be
worth the maintenance overhead. Virtually all amd64 hardware has
multiple cores. The CPU and memory usage overhead of the SMP option in
single-vCPU VMs is quite marginal and not worth maintaining.

Reviewed by: alc (pmap.c), kib
Differential Revision: https://reviews.freebsd.org/D51403
Differential Revision: https://reviews.freebsd.org/D51345

show more ...

amd64: add kva_layout

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D51085

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https:/

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48486

show more ...

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: im

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48485

show more ...

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version st

Centralize and simpify implemention of some VM macros

These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.

This version standardizes on not using explicit casts and instead
preserving the original type. A couple of tweaks were required to
make this work.

Reviewed by: brooks, kib, markj
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D48450

show more ...

Remove stray whitespaces from sys/amd64/

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1418

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

amd64: Bump MAXCPU to 1024 (from 256)

Hardware with more than 256 CPU cores is currently available and will
become increasingly common over FreeBSD 14's lifetime. Increase MAXCPU
in the amd64 GENER

amd64: Bump MAXCPU to 1024 (from 256)

Hardware with more than 256 CPU cores is currently available and will
become increasingly common over FreeBSD 14's lifetime. Increase MAXCPU
in the amd64 GENERIC kernel configuration to 1024.

Earlier commits increased some related limits. These prerequisite
commits include at least:

- d7ed40243769 Increase MAX_APIC_ID safeguard to 0x800
- d1639e43c589 cpuset: increase userland maximum size to 1024

Global and allocated arrays sized by MAXCPU result in excessive bloat
on systems with lower core counts. In addition, some code used u_char
(8 bits) to hold a CPU index, which is not valid if MAXCPU is greater
than 256.

A number of recent commits addressed these sorts of issues, including
at least:

- 133935d26f20 pf: atomically increment state ids
- 74ac712f72cf vmm: Dynamically allocate a couple of per-CPU state save areas
- 78cfa762ebf2 callout: Move per-CPU callout state into the dpcpu region
- 42f722e721cd amd64: store pcids pmap data in pcpu zone
- 9801e7c275f6 smp_topo: dynamically allocate group array
- 9fb6718d1b18 smp: Dynamically allocate the stoppcbs array
- 2bb16c635249 x86: retire use of intr_bind

There are some additional allocations still to be converted and
more scalability work is required to make effective use of very high
core count systems, but this change allows us to boot on these systems
and provides a Kernel Binary Interface (KBI) for the FreeBSD 14 release
that supports these configurations.

Special thanks to AMD for providing hardware to test these changes.

PR: 269572
Reviewed by: des
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36838

show more ...

amd64: Populate the KMSAN shadow maps and integrate with the VM

- During boot, allocate PDP pages for the shadow maps. The region above
KERNBASE is currently not shadowed.
- Create a dummy shadow

amd64: Populate the KMSAN shadow maps and integrate with the VM

- During boot, allocate PDP pages for the shadow maps. The region above
KERNBASE is currently not shadowed.
- Create a dummy shadow for the vm page array. For now, this array is
not protected by the shadow map to help reduce kernel memory usage.
- Grow shadows when growing the kernel map.
- Increase the default kernel stack size when KMSAN is enabled. As with
KASAN, sanitizer instrumentation appears to create stack frames large
enough that the default value is not sufficient.
- Disable UMA's use of the direct map when KMSAN is configured. KMSAN
cannot validate the direct map.
- Disable unmapped I/O when KMSAN configured.
- Lower the limit on paging buffers when KMSAN is configured. Each
buffer has a static MAXPHYS-sized allocation of KVA, which in turn
eats 2*MAXPHYS of space in the shadow map.

Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31295

show more ...

amd64: Add MD bits for KASAN

- Initialize KASAN before executing SYSINITs.
- Add a GENERIC-KASAN kernel config, akin to GENERIC-KCSAN.
- Increase the kernel stack size if KASAN is enabled. Some of

amd64: Add MD bits for KASAN

- Initialize KASAN before executing SYSINITs.
- Add a GENERIC-KASAN kernel config, akin to GENERIC-KCSAN.
- Increase the kernel stack size if KASAN is enabled. Some of the
ASAN instrumentation increases stack usage and it's enough to
trigger stack overflows in ZFS.
- Mark the trapframe as valid in interrupt handlers if it is
assigned to td_intr_frame. Otherwise, an interrupt in a function
which creates a poisoned alloca region can trigger false positives.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29455

show more ...

amd64: clean up empty lines in .c and .h files

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

amd64 pmap: LA57 AKA 5-level paging

Since LA57 was moved to the main SDM document with revision 072, it
seems that we should have a support for it, and silicons are coming.

This patch makes pmap su

amd64 pmap: LA57 AKA 5-level paging

Since LA57 was moved to the main SDM document with revision 072, it
seems that we should have a support for it, and silicons are coming.

This patch makes pmap support both LA48 and LA57 hardware. The
selection of page table level is done at startup, kernel always
receives control from loader with 4-level paging. It is not clear how
UEFI spec would adapt LA57, for instance it could hand out control in
LA57 mode sometimes.

To switch from LA48 to LA57 requires turning off long mode, requesting
LA57 in CR4, then re-entering long mode. This is somewhat delicate
and done in pmap_bootstrap_la57(). AP startup in LA57 mode is much
easier, we only need to toggle a bit in CR4 and load right value in CR3.

I decided to not change kernel map for now. Single PML5 entry is
created that points to the existing kernel_pml4 (KML4Phys) page, and a
pml5 entry to create our recursive mapping for vtopte()/vtopde().
This decision is motivated by the fact that we cannot overcommit for
KVA, so large space there is unusable until machines start providing
wider physical memory addressing. Another reason is that I do not
want to break our fragile autotuning, so the KVA expansion is not
included into this first step. Nice side effect is that minidumps are
compatible.

On the other hand, (very) large address space is definitely
immediately useful for some userspace applications.

For userspace, numbering of pte entries (or page table pages) is
always done for 5-level structures even if we operate in 4-level mode.
The pmap_is_la57() function is added to report the mode of the
specified pmap, this is done not to allow simultaneous 4-/5-levels
(which is not allowed by hw), but to accomodate for EPT which has
separate level control and in principle might not allow 5-leve EPT
despite x86 paging supports it. Anyway, it does not seems critical to
have 5-level EPT support now.

Tested by: pho (LA48 hardware)
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D25273

show more ...

Merge ^/head r357408 through r357661.

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it bre

Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.

PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512

show more ...

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align thin

Add sys/_align.h replacing machine/_align.h

Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.

In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.

Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947

show more ...

amd64: bump sleepq hash size to 2048

This is the most contended lock type during the first hour of -j 104 poudriere.

Drops significantly with the change.

Note there are suspicous acquires which mo

amd64: bump sleepq hash size to 2048

This is the most contended lock type during the first hour of -j 104 poudriere.

Drops significantly with the change.

Note there are suspicous acquires which most likely don't need to
happen, artificially exacerbating tehe problem..

show more ...

amd64: Remove support for "nooptions SMP"

It does not appear to get much, if any, testing, and doesn't seem to be
worth the maintenance overhead. Virtually all amd64 hardware has
multiple cores. T

amd64: Remove support for "nooptions SMP"

It does not appear to get much, if any, testing, and doesn't seem to be
worth the maintenance overhead. Virtually all amd64 hardware has
multiple cores. The CPU and memory usage overhead of the SMP option in
single-vCPU VMs is quite marginal and not worth maintaining.

Reviewed by: alc (pmap.c), kib
Differential Revision: https://reviews.freebsd.org/D51403
Differential Revision: https://reviews.freebsd.org/D51345

show more ...

amd64: add kva_layout

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D51085

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https:/

machine/param.h: drop __HAVE_ACPI define

It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48486

show more ...

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: im

pci: cleanup __PCI_REROUTE_INTERRUPTS

This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.

Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48485

show more ...