kernel: Fix defining of .init_array and .fini_array sections

These input sections can have decimal numbers as the priority suffix.
Clang emits the '%u' form, while SORT is an alias for SORT_BY_NAME,

kernel: Fix defining of .init_array and .fini_array sections

These input sections can have decimal numbers as the priority suffix.
Clang emits the '%u' form, while SORT is an alias for SORT_BY_NAME,
hence will result in wrong order of constructors / destructors in
output sections. Fix by using the correct sorting command
SORT_BY_INIT_PRIORITY instead [1].

The functions referenced by section .fini_array is in the normal order,
but been executed in the reverse order. The order is same with
.init_array section.

Currently these sections are not used, there should be no functional
change.

Note: As for the .ctors and .dtors sections, both Clang and GCC emit
the priority suffix in the form of '%05u', so there is no semantic
difference between SORT_BY_NAME and SORT_BY_INIT_PRIORITY for those
sections [2].

This fix is extracted from a bigger patch [3] of hselasky, with
additional fix for .fini_array section.

1. https://sourceware.org/binutils/docs/ld/Input-Section-Wildcards.html
2. https://reviews.llvm.org/D91187
3. https://reviews.freebsd.org/D40467

Reviewed by: imp (previous version)
Obtained from: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45194

show more ...

kernel: Move the debug stuff into a common script

Move a copy of amd64's debug code into debug.ldscript. Make all the
kernels use this. This has the effect of modernizing the STABS for
powerpc as th

kernel: Move the debug stuff into a common script

Move a copy of amd64's debug code into debug.ldscript. Make all the
kernels use this. This has the effect of modernizing the STABS for
powerpc as the others were almost already in sync. For the ones that
weren't this adds the DWARF 3 debug symbols from i386/amd64.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D44071

show more ...

kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete

kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete comment that
was removed from i386 in 2010.

Sponsored by: Netflix
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44069

show more ...

sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assu

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assumption kernel makes about loader->kernel handoff with regard to
the MMU programming are explicitly listed at the beginning of hammer_time(),
where kernphys is calculated. Now kernphys is the variable instead of
symbol designating the physical address.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31121

show more ...

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in the linker script, which albeit
documented via comment but isn't programmatically accessible at a compile time.

Use KERNLOAD to eliminate another (matching) magic constant 100 lines down
inside unremarkable TU "copy.c" 3 levels deep in the EFI loader tree.

Reviewed by: markj
Approved by: markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D27355

show more ...

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldat

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldata and related sections in the kernel linker
script. They come from ld.bfd's default linker script, but are not
used, and we now use ld.lld to link the amd64 kernel. lld does not
contain a default linker script.
- Pad the .bss to a 2MB as we do between .text and .data. This
forces the loader to load additional files starting in the following
2MB page, preserving the use of superpage mappings for kernel data.
- Map memory above the kernel image with NX. The kernel linker now
upgrades protections as needed, and other preloaded file types
(e.g., entropy, microcode) need not be mapped with execute permissions
in the first place.

Reviewed by: kib
MFC after: 1 month
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21859

show more ...

MFHead @348740

Sponsored by: The FreeBSD Foundation

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow th

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow the user to identify their running kernel.

Submitted by: Ali Mashtizadeh <ali_mashtizadeh.com>
MFC after: 2 weeks
Relnotes: Yes
Event: Waterloo Hackathon 2019
Differential Revision: https://reviews.freebsd.org/D20326

show more ...

Merge ^/head r340235 through r340367.

Add comment to explain kernel ldscript 0x200000 constant

Reported by: linimon

kernel: Fix defining of .init_array and .fini_array sections

These input sections can have decimal numbers as the priority suffix.
Clang emits the '%u' form, while SORT is an alias for SORT_BY_NAME,

kernel: Fix defining of .init_array and .fini_array sections

These input sections can have decimal numbers as the priority suffix.
Clang emits the '%u' form, while SORT is an alias for SORT_BY_NAME,
hence will result in wrong order of constructors / destructors in
output sections. Fix by using the correct sorting command
SORT_BY_INIT_PRIORITY instead [1].

The functions referenced by section .fini_array is in the normal order,
but been executed in the reverse order. The order is same with
.init_array section.

Currently these sections are not used, there should be no functional
change.

Note: As for the .ctors and .dtors sections, both Clang and GCC emit
the priority suffix in the form of '%05u', so there is no semantic
difference between SORT_BY_NAME and SORT_BY_INIT_PRIORITY for those
sections [2].

This fix is extracted from a bigger patch [3] of hselasky, with
additional fix for .fini_array section.

1. https://sourceware.org/binutils/docs/ld/Input-Section-Wildcards.html
2. https://reviews.llvm.org/D91187
3. https://reviews.freebsd.org/D40467

Reviewed by: imp (previous version)
Obtained from: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45194

show more ...

kernel: Move the debug stuff into a common script

Move a copy of amd64's debug code into debug.ldscript. Make all the
kernels use this. This has the effect of modernizing the STABS for
powerpc as th

kernel: Move the debug stuff into a common script

Move a copy of amd64's debug code into debug.ldscript. Make all the
kernels use this. This has the effect of modernizing the STABS for
powerpc as the others were almost already in sync. For the ones that
weren't this adds the DWARF 3 debug symbols from i386/amd64.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D44071

show more ...

kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete

kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete comment that
was removed from i386 in 2010.

Sponsored by: Netflix
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44069

show more ...

sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assu

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assumption kernel makes about loader->kernel handoff with regard to
the MMU programming are explicitly listed at the beginning of hammer_time(),
where kernphys is calculated. Now kernphys is the variable instead of
symbol designating the physical address.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31121

show more ...

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in the linker script, which albeit
documented via comment but isn't programmatically accessible at a compile time.

Use KERNLOAD to eliminate another (matching) magic constant 100 lines down
inside unremarkable TU "copy.c" 3 levels deep in the EFI loader tree.

Reviewed by: markj
Approved by: markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D27355

show more ...

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldat

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldata and related sections in the kernel linker
script. They come from ld.bfd's default linker script, but are not
used, and we now use ld.lld to link the amd64 kernel. lld does not
contain a default linker script.
- Pad the .bss to a 2MB as we do between .text and .data. This
forces the loader to load additional files starting in the following
2MB page, preserving the use of superpage mappings for kernel data.
- Map memory above the kernel image with NX. The kernel linker now
upgrades protections as needed, and other preloaded file types
(e.g., entropy, microcode) need not be mapped with execute permissions
in the first place.

Reviewed by: kib
MFC after: 1 month
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21859

show more ...

MFHead @348740

Sponsored by: The FreeBSD Foundation

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow th

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow the user to identify their running kernel.

Submitted by: Ali Mashtizadeh <ali_mashtizadeh.com>
MFC after: 2 weeks
Relnotes: Yes
Event: Waterloo Hackathon 2019
Differential Revision: https://reviews.freebsd.org/D20326

show more ...

Merge ^/head r340235 through r340367.

Add comment to explain kernel ldscript 0x200000 constant

Reported by: linimon

amd64: tweak the read_frequently section

1. align to 128 bytes to avoid possible waste from the preceeding section
2. sort entries by alignment SORT_BY_ALIGNMENT, plugging the holes (most
entries ar

amd64: tweak the read_frequently section

1. align to 128 bytes to avoid possible waste from the preceeding section
2. sort entries by alignment SORT_BY_ALIGNMENT, plugging the holes (most
entries are one byte in size, but they got interleaved with bigger ones)

Interestingly I was looking for a feature of the sort earlier and failed
to find it. It turns out the script was already utilizing sorting in other
places, so shame on me.

Thanks for Travis Geiselbrecht for pointing me at the feature.

show more ...

amd64: align the .data.exclusive_cache_line section to 128

This aligns the section itself compared to other sections, does not change
internal alignment of fields stored inside. This may or may not

amd64: align the .data.exclusive_cache_line section to 128

This aligns the section itself compared to other sections, does not change
internal alignment of fields stored inside. This may or may not come later.

The motivation is partially combating adverse effects of the adjacent cache
line prefetcher. Without the annotation part of read_mostly section was on
the line of fire.

show more ...

amd64: Protect the kernel text, data, and BSS by setting the RW/NX bits
correctly for the data contained on each memory page.

There are several components to this change:
* Add a variable to indica

amd64: Protect the kernel text, data, and BSS by setting the RW/NX bits
correctly for the data contained on each memory page.

There are several components to this change:
* Add a variable to indicate the start of the R/W portion of the
initial memory.
* Stop detecting NX bit support for each AP. Instead, use the value
from the BSP and, if supported, activate the feature on the other
APs just before loading the correct page table. (Functionally, we
already assume that the BSP and all APs had the same support or
lack of support for the NX bit.)
* Set the RW and NX bits correctly for the kernel text, data, and
BSS (subject to some caveats below).
* Ensure DDB can write to memory when necessary (such as to set a
breakpoint).
* Ensure GDB can write to memory when necessary (such as to set a
breakpoint). For this purpose, add new MD functions gdb_begin_write()
and gdb_end_write() which the GDB support code can call before and
after writing to memory.

This change is not comprehensive:
* It doesn't do anything to protect modules.
* It doesn't do anything for kernel memory allocated after the kernel
starts running.
* In order to avoid excessive memory inefficiency, it may let multiple
types of data share a 2M page, and assigns the most permissions
needed for data on that page.

Reviewed by: jhb, kib
Discussed with: emaste
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14282

show more ...