amd64: assume the kernel supports RDFSBASE and RDGSBASE

There is no need to support pre-12.0 (and thus pre-ino64) kernels in a
15.x libc.

Continue to check if the CPU supports these features before

amd64: assume the kernel supports RDFSBASE and RDGSBASE

There is no need to support pre-12.0 (and thus pre-ino64) kernels in a
15.x libc.

Continue to check if the CPU supports these features before using them
and fall back as required.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50732

show more ...

rtld-elf: Pass struct tcb * around rather than struct dtv **

When this code was first written we didn't have even a struct tcb, so to
make it MI a pointer to the DTV pointer in the TCB was passed ar

rtld-elf: Pass struct tcb * around rather than struct dtv **

When this code was first written we didn't have even a struct tcb, so to
make it MI a pointer to the DTV pointer in the TCB was passed around.
Now that we have a struct tcb we can simplify the code by instead
passing around a pointer to that, and the MI code can access the tcb_dtv
member wherever it happens to be in the layout. This reduces boilerplate
in all the various callers of tls_get_addr_common/slow and makes it
clearer that tls_get_addr_common/slow are operating on the TCB, rather
than obfuscating it slightly through the double pointer.

Whilst here, clarify the comments in aarch64's TLSDESC dynamic resolver,
which were using tp without clarifying what this was for (previously a
pointer to the DTV pointer, now a pointer to the TCB, which happen to be
the same thing for Variant I TLS, and in the case of AArch64 are what
TPIDR_EL0 point to directly, with no offset/bias).

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50591

show more ...

amd64: switch to amd64_set_tlsbase to set tls base

Reported and tested: Alex S <iwtcex@gmail.com>
Tested by: pho
Reviewed by: olce
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Different

amd64: switch to amd64_set_tlsbase to set tls base

Reported and tested: Alex S <iwtcex@gmail.com>
Tested by: pho
Reviewed by: olce
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D50482

show more ...

tls: Introduce struct dtv and struct dtv_slot

Rather than treating the DTV as a raw array of uintptr_t, use proper
struct types and gain the benefit of having different types for
different members.

tls: Introduce struct dtv and struct dtv_slot

Rather than treating the DTV as a raw array of uintptr_t, use proper
struct types and gain the benefit of having different types for
different members. In particular, the module slots now have real pointer
types so less casting is generally needed.

Note that, whilst struct dtv_slot may seem a little unnecessary, this
will help downstream in CheriBSD where we wish to be able to easily
alter the layout of a module's slot, which this helps abstract.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D50231

show more ...

rtld-elf/amd64/reloc.c: apply clang-format

Discussed with: emaste, imp
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D48509

rtld-elf/{amd64,i386}/reloc.c: remove unneeded #ifdef dbg

Discussed with: emaste, imp
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D48509

rtld-elf: Pass parsed aux_info to ifunc_init

Currently we pass the raw pointer to the on-stack auxargs. This can
legitimately have fewer than AT_COUNT entries, so the use of
__min_size(AT_COUNT), i.

rtld-elf: Pass parsed aux_info to ifunc_init

Currently we pass the raw pointer to the on-stack auxargs. This can
legitimately have fewer than AT_COUNT entries, so the use of
__min_size(AT_COUNT), i.e. static AT_COUNT, is inaccurate, and also
needlessly forces the callee to iterate over the elements to find the
entry for a given type. Instead we can just pass aux_info like we use
for everything else.

Note that the argument has been left unused by every callee since its
introduction in 4352999e0e6c ("Pass CPUID[1] %edx (cpu_feature), %ecx
(cpu_feature2) and CPUID[7].%ebx (cpu_stdext_feature), %ecx
(cpu_stdext_feature2) to the ifunc resolvers on x86.")

Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D46276

show more ...

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'. This extra static TLS space is used
for objects loaded with dlopen.

The value specified in the variable must be no less than the default
value and no greater than the maximum allowed value for size_t type.

If an invalid value is specified, rtld will ignore it and just use
the default value.

The rtld(1) man page is updated to document this new option.

Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D42025

show more ...

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

sys/param.h: Add _WANT_P_OSREL

Use it instead of defining IN_RTLD by base sources that want P_OSREL_
defines in userspace, but are not rtld.
This allows to remove abuse of IN_RTLD from userspace.

R

sys/param.h: Add _WANT_P_OSREL

Use it instead of defining IN_RTLD by base sources that want P_OSREL_
defines in userspace, but are not rtld.
This allows to remove abuse of IN_RTLD from userspace.

Reviewed by: dchagin, markj, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38585

show more ...

TLS: Use <machine/tls.h> for libc and rtld.

- Include <machine/tls.h> in MD rtld_machdep.h headers.

- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the va

TLS: Use <machine/tls.h> for libc and rtld.

- Include <machine/tls.h> in MD rtld_machdep.h headers.

- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the values from <machine/tls.h> instead.

- Use _tcb_set() instead of inlined versions in MD
allocate_initial_tls() routines in rtld. The one exception is amd64
whose _tcb_set() invokes the amd64_set_fsbase ifunc. rtld cannot
use ifuncs, so amd64 inlines the logic to optionally write to fsbase
directly.

- Use _tcb_set() instead of _set_tp() in libc.

- Use '&_tcb_get()->tcb_dtv' instead of _get_tp() in both rtld and libc.
This permits removing _get_tp.c from rtld.

- Use TLS_TCB_SIZE and TLS_TCB_ALIGN with allocate_tls() in MD
allocate_initial_tls() routines in rtld.

Reviewed by: kib, jrtc27 (earlier version)
Differential Revision: https://reviews.freebsd.org/D33353

show more ...

amd64: Allocate TCB with alignment of 16 rather than 8.

This matches the TLS_TCB_ALIGN definition in libc.

Reviewed by: kib, jrtc27
Sponsored by: The University of Cambridge, Google Inc.
Differenti

amd64: Allocate TCB with alignment of 16 rather than 8.

This matches the TLS_TCB_ALIGN definition in libc.

Reviewed by: kib, jrtc27
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33349

show more ...

rtld: Remove calculate_tls_end

Variant I architectures use off and Variant II ones use size + off.
Define TLS_VARIANT_I/TLS_VARIANT_II symbols similarly to how libc
handles it.

Reviewed by: kib
MFC

rtld: Remove calculate_tls_end

Variant I architectures use off and Variant II ones use size + off.
Define TLS_VARIANT_I/TLS_VARIANT_II symbols similarly to how libc
handles it.

Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31539
Differential revision: https://reviews.freebsd.org/D31541

show more ...

rtld: Fix i386/amd64 TP offset when p_vaddr % p_align != 0

For a Variant II architecture, the TP offset of a TLS symbol is st_value -
tlsoffset + r_addend. tlsoffset is computed by either calculate_

rtld: Fix i386/amd64 TP offset when p_vaddr % p_align != 0

For a Variant II architecture, the TP offset of a TLS symbol is st_value -
tlsoffset + r_addend. tlsoffset is computed by either calculate_tls_offset
or calculate_first_tls_offset.

The return value of calculate_first_tls_offset is the smallest integer
satisfying res >= size and (-res) % p_align = p_vaddr % p_align
(= p_offset % p_align). (The formula is a bit contrived. The basic idea
is to subtract the minimum integer from size + align - 1 so that the result
ihas the expected remainder.)

Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31538
Differential revision: https://reviews.freebsd.org/D31541

show more ...

rtld: use _get_tp() in __tls_get_addr()

This eliminates some non-trivial amount of code duplication, where done.
Only x86 and mips are handled right now.

Tested by: bdragon (powerpc), mhorne (

rtld: use _get_tp() in __tls_get_addr()

This eliminates some non-trivial amount of code duplication, where done.
Only x86 and mips are handled right now.

Tested by: bdragon (powerpc), mhorne (riscv)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623

show more ...

rtld/x86/reloc.c: style

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623

rtld-elf(1): remove obsolete pre_init() hook

It's no longer used since 600ee699ed2805894f5972c6ac2c3d17dca7f6ce
and r358358 respectively.

Align initial-exec TLS segments to the p_vaddr % align.

This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed

Align initial-exec TLS segments to the p_vaddr % align.

This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed.

Tested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D24366

show more ...

Merge ^/head r357855 through r357920.

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23652

show more ...

Improve R_AARCH64_TLSDESC relocation.
The original code did not support dynamically loaded libraries and used
suboptimal access to TLS variables.
New implementation removes lazy resolving of TLS relo

Improve R_AARCH64_TLSDESC relocation.
The original code did not support dynamically loaded libraries and used
suboptimal access to TLS variables.
New implementation removes lazy resolving of TLS relocation - due to flaw
in TLSDESC design is impossible to switch resolver function at runtime
without expensive locking.

Due to this, 3 specialized resolvers are implemented:
- load time resolver for TLS relocation from libraries loaded with main
executable (thus with known TLS offset).
- resolver for undefined thread weak symbols.
- slower lazy resolver for dynamically loaded libraries with fast path for
already resolved symbols.

PR: 228892, 232149, 233204, 232311
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18417

show more ...

Merge ^/head r340427 through r340868.

Silence gcc warnings.

Reported by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 3 days