rtld: Use uintptr_t instead of Elf_Addr for init/fini function pointers

This is a no-op on non-CHERI architectures, but is required for CHERI
where Elf_Addr is only an address and not a complete poi

rtld: Use uintptr_t instead of Elf_Addr for init/fini function pointers

This is a no-op on non-CHERI architectures, but is required for CHERI
where Elf_Addr is only an address and not a complete pointer.

While here, consistently use `uintptr_t *` for arrays of init/fini
function pointers.

Reviewed by: imp, kib
Effort: CHERI upstreaming
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D54711

show more ...

rtld: Simplify walking program headers

Store phnum in Obj_Entry instead of phsize and use that to simplify
the terminate expressions when iterating over program headers.

Reviewed by: kib
Obtained f

rtld: Simplify walking program headers

Store phnum in Obj_Entry instead of phsize and use that to simplify
the terminate expressions when iterating over program headers.

Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D54710

show more ...

rtld-elf: Don't include duplicate semicolon in Obj_Entry

MD_OBJ_ENTRY is a list of members, possibly empty, to include in
Obj_Entry. By including the semicolon here, in the case that it's empty,
we

rtld-elf: Don't include duplicate semicolon in Obj_Entry

MD_OBJ_ENTRY is a list of members, possibly empty, to include in
Obj_Entry. By including the semicolon here, in the case that it's empty,
we end up with a duplicate semicolon. In the case that it's not empty,
whether there's a duplicate depends on each architecture's definition,
but they all in fact put a semicolon after every member, so there is
also a duplicate semicolon there. This is invalid C syntax, although
both GCC and Clang accept it, treating it only as a pedantic warning,
but there is no need for us to rely on that, and downstream it masked a
missing semicolon for an added field, but only on architectures where
MD_OBJ_ENTRY is empty, leading to conditional compilation failure for
something that should have been detected as an unconditional error.

Note that PCPU_MD_FIELDS, which this is based on, follows a different
style. There, every architecture defines at least one member, and there
is a semicolon after PCPU_MD_FIELDS in sys/sys/pcpu.h, but every
architecture makes sure to not put a semicolon after the final member in
its definition of the macro. This is not a pattern we can adhere to here
though given not all architectures add members.

Fixes: 06db20ffeca9 ("rtld: Add MD_OBJ_ENTRY to extend Struct_Obj_Entry")

show more ...

rtld-elf: Pass TCB to allocate_module_tls to avoid re-getting

The only caller already has the current TCB to hand, so just pass it
down rather than get it again. This also makes it clear in the call

rtld-elf: Pass TCB to allocate_module_tls to avoid re-getting

The only caller already has the current TCB to hand, so just pass it
down rather than get it again. This also makes it clear in the caller
that it depends on the (current) TCB, rather than being storage that
could be assigned to any thread (concurrency issues aside).

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50594

show more ...

rtld-elf: Pass struct tcb * around rather than struct dtv **

When this code was first written we didn't have even a struct tcb, so to
make it MI a pointer to the DTV pointer in the TCB was passed ar

rtld-elf: Pass struct tcb * around rather than struct dtv **

When this code was first written we didn't have even a struct tcb, so to
make it MI a pointer to the DTV pointer in the TCB was passed around.
Now that we have a struct tcb we can simplify the code by instead
passing around a pointer to that, and the MI code can access the tcb_dtv
member wherever it happens to be in the layout. This reduces boilerplate
in all the various callers of tls_get_addr_common/slow and makes it
clearer that tls_get_addr_common/slow are operating on the TCB, rather
than obfuscating it slightly through the double pointer.

Whilst here, clarify the comments in aarch64's TLSDESC dynamic resolver,
which were using tp without clarifying what this was for (previously a
pointer to the DTV pointer, now a pointer to the TCB, which happen to be
the same thing for Variant I TLS, and in the case of AArch64 are what
TPIDR_EL0 point to directly, with no offset/bias).

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50591

show more ...

tls: Introduce struct dtv and struct dtv_slot

Rather than treating the DTV as a raw array of uintptr_t, use proper
struct types and gain the benefit of having different types for
different members.

tls: Introduce struct dtv and struct dtv_slot

Rather than treating the DTV as a raw array of uintptr_t, use proper
struct types and gain the benefit of having different types for
different members. In particular, the module slots now have real pointer
types so less casting is generally needed.

Note that, whilst struct dtv_slot may seem a little unnecessary, this
will help downstream in CheriBSD where we wish to be able to easily
alter the layout of a module's slot, which this helps abstract.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50231

show more ...

rtld-elf: Fix executable's TLS module index for direct exec

For direct exec mode we reuse map_object, but tls_max_index is
initialised to 1. As a result, the executable ends up being assigned
module

rtld-elf: Fix executable's TLS module index for direct exec

For direct exec mode we reuse map_object, but tls_max_index is
initialised to 1. As a result, the executable ends up being assigned
module 2 (and the generation is pointlessly incremented, unlike in
digest_phdr for the normal case). For most architectures this is
harmless, since TLS linker relaxation will optimise General Dynamic
accesses to Initial Exec or Local Exec for executables, but on RISC-V
this relaxation does not exist, yet the linker will initialise the
tls_index in the GOT with module 1, and at run time the call to
__tls_get_addr will fail with:

ld-elf.so.1: Can't find module with TLS index 1

Fix this by making map_object use 1 for obj->tlsindex when it's loading
the main executable, and don't bother to increment tls_dtv_generation
either, matching digest_phdr (though that one is harmless).

(Note this also applies to MIPS on stable/13)

Reviewed by: kib
Fixes: 0fc65b0ab82c ("Make ld-elf.so.1 directly executable.")
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D50186

show more ...

rtld: add support for -z initfirst

Internally, initfirst objects and their needed objects are put on the
dedicated initlist, which is prepended to the current regular initlist
at the last moment.

T

rtld: add support for -z initfirst

Internally, initfirst objects and their needed objects are put on the
dedicated initlist, which is prepended to the current regular initlist
at the last moment.

This results in the move of the needed objects into the beginning of the
initlist, which is required for the proper initialization of the
dependencies. It seems that glibc moves only the initfirst object,
which makes its constructors depend on not yet initialized dsos.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D50132

show more ...

rtld: Support multiple PT_GNU_RELRO program headers

Iterate over all the program headers in obj_remap_relro and remove the
relro fields from Obj_Entry.

Skip the call to obj_enforce_relro() in reloc

rtld: Support multiple PT_GNU_RELRO program headers

Iterate over all the program headers in obj_remap_relro and remove the
relro fields from Obj_Entry.

Skip the call to obj_enforce_relro() in relocate_object() for the rtld
object as well as the main program object. obj_enforce_relro() is
called later when it safe to reference globals such as page_size.

Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D47884

show more ...

rtld-elf: Pass parsed aux_info to ifunc_init

Currently we pass the raw pointer to the on-stack auxargs. This can
legitimately have fewer than AT_COUNT entries, so the use of
__min_size(AT_COUNT), i.

rtld-elf: Pass parsed aux_info to ifunc_init

Currently we pass the raw pointer to the on-stack auxargs. This can
legitimately have fewer than AT_COUNT entries, so the use of
__min_size(AT_COUNT), i.e. static AT_COUNT, is inaccurate, and also
needlessly forces the callee to iterate over the elements to find the
entry for a given type. Instead we can just pass aux_info like we use
for everything else.

Note that the argument has been left unused by every callee since its
introduction in 4352999e0e6c ("Pass CPUID[1] %edx (cpu_feature), %ecx
(cpu_feature2) and CPUID[7].%ebx (cpu_stdext_feature), %ecx
(cpu_stdext_feature2) to the ifunc resolvers on x86.")

Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D46276

show more ...

rtld: add LD_NO_DL_ITERATE_PHDR_AFTER_FORK env var

which makes threaded fork ignore the phdr rtld lock, in particular
allowing the dl_iterate_phdr() to block in callback. The cost is that
the image

rtld: add LD_NO_DL_ITERATE_PHDR_AFTER_FORK env var

which makes threaded fork ignore the phdr rtld lock, in particular
allowing the dl_iterate_phdr() to block in callback. The cost is that
the image started in this mode cannot use dl_iterate_phdr() after fork.

PR: 280318
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

rtld: make ld_get_env_var() usable for all rtld source files

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Move powerpc specific code to powerpc files

There are two variables set by dynamic tags in the powerpc runtime
linker. Now we have a way to split out architecture-specific dynamic
tags use it

rtld: Move powerpc specific code to powerpc files

There are two variables set by dynamic tags in the powerpc runtime
linker. Now we have a way to split out architecture-specific dynamic
tags use it to handle these.

Reviewed by: kib, jhibbits
Obtained from: jhibbits (earlier version)
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45182

show more ...

rtld: Add MD_OBJ_ENTRY to extend Struct_Obj_Entry

Add a macro the architectures can use to add per-arch fields to
Struct_Obj_Entry.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: htt

rtld: Add MD_OBJ_ENTRY to extend Struct_Obj_Entry

Add a macro the architectures can use to add per-arch fields to
Struct_Obj_Entry.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45116

show more ...

rtld: ignore load_filtees() calls if we already loading filtees for the obj

in addition to avoiding it for already loaded filtees. Issue is that
during load, rtld needs to resolve some special ABI s

rtld: ignore load_filtees() calls if we already loading filtees for the obj

in addition to avoiding it for already loaded filtees. Issue is that
during load, rtld needs to resolve some special ABI symbols, like
executable stack fixer and static TLS initializer, which might trigger
recursion.

Example is libthr which is filter for libsys, and which exports
__pthread_distribute_static_tls.

Tested by: kevans, krion
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43858

show more ...

RTLD_DEEPBIND: make lookup not just symbolic, but walk all refobj' DAGs

before starting the walk over the global list. Effectively we visit
needed objects first as well, instead of just the object

RTLD_DEEPBIND: make lookup not just symbolic, but walk all refobj' DAGs

before starting the walk over the global list. Effectively we visit
needed objects first as well, instead of just the object itself.
This seems to better match the semantic offered by the glibc flag.

Reported by: kevans
PR: 275393
Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D42841

show more ...

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'. This extra static TLS space is used
for objects loaded with dlopen.

The value specified in the variable must be no less than the default
value and no greater than the maximum allowed value for size_t type.

If an invalid value is specified, rtld will ignore it and just use
the default value.

The rtld(1) man page is updated to document this new option.

Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D42025

show more ...

rtld: switch from malloc_aligned() to __crt_aligned_alloc()

Use regular free(), since it works now.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision:

rtld: switch from malloc_aligned() to __crt_aligned_alloc()

Use regular free(), since it works now.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41150

show more ...

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

rtld: remove dup __crt_malloc prototypes

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41150

rtld: do not allow both dynamic DTV index and static TLS offset

If we are allocating static offset for an object with dynamic index,
return failure. In the opposite case, if dynamic index is reques

rtld: do not allow both dynamic DTV index and static TLS offset

If we are allocating static offset for an object with dynamic index,
return failure. In the opposite case, if dynamic index is requested for
statically allocated TLS area, directly use the offset instead of
setting the index.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

rtld: fix SysV hash function overflow

Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:

The System V Application Binary Interface (generic ABI) specifies the
ELF object file format

rtld: fix SysV hash function overflow

Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:

The System V Application Binary Interface (generic ABI) specifies the
ELF object file format. When producing an output executable or shared
object needing a dynamic symbol table (.dynsym), a linker generates a
.hash section with type SHT_HASH to hold a symbol hash table. A DT_HASH
tag is produced to hold the address of .hash.

The function is supposed to return a value no larger than 0x0fffffff.
Unfortunately, there is a bug. When unsigned long consists of more than
32 bits, the return value may be larger than UINT32_MAX. For instance,
elf_hash((const unsigned char *)"\xff\x0f\x0f\x0f\x0f\x0f\x12") returns
0x100000002, which is clearly unintended, as the function should behave
the same way regardless of whether long represents a 32-bit integer or
a 64-bit integer.

Reviewed by: kib, Fangrui Song
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39517

show more ...

Have rtld query the page size from the kernel

To allow for a dynamic page size on arm64 have the runtime linker
query the kernel for the currentl page size.

Reviewed by: kib
Sponsored by: The FreeB

Have rtld query the page size from the kernel

To allow for a dynamic page size on arm64 have the runtime linker
query the kernel for the currentl page size.

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34765

show more ...