accel/tcg: Remove TARGET_PAGE_DATA_SIZE

This macro is used by only one target, and even then under
unusual conditions -- AArch64 with mmap's PROT_MTE flag.

Since page size for aarch64-linux-user is

accel/tcg: Remove TARGET_PAGE_DATA_SIZE

This macro is used by only one target, and even then under
unusual conditions -- AArch64 with mmap's PROT_MTE flag.

Since page size for aarch64-linux-user is variable, the
per-page data size is also variable.
Since page_reset_target_data via target_munmap does not
have ready access to CPUState, simply pass in the size
from the first allocation and remember that.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

include: Remove 'exec/exec-all.h'

"exec/exec-all.h" is now fully empty, let's remove it.

Mechanical change running:

$ sed -i '/exec\/exec-all.h/d' $(git grep -wl exec/exec-all.h)

Signed-off-by:

include: Remove 'exec/exec-all.h'

"exec/exec-all.h" is now fully empty, let's remove it.

Mechanical change running:

$ sed -i '/exec\/exec-all.h/d' $(git grep -wl exec/exec-all.h)

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250424202412.91612-14-philmd@linaro.org>

show more ...

accel/tcg: Extract probe API out of 'exec/exec-all.h'

Declare probe methods in "accel/tcg/probe.h" to emphasize
they are specific to TCG accelerator.

Suggested-by: Richard Henderson <richard.hender

accel/tcg: Extract probe API out of 'exec/exec-all.h'

Declare probe methods in "accel/tcg/probe.h" to emphasize
they are specific to TCG accelerator.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250424202412.91612-13-philmd@linaro.org>

show more ...

exec: Restrict 'cpu_ldst.h' to accel/tcg/

Mechanical change using:

$ sed -i -e 's,exec/cpu_ldst,accel/tcg/cpu-ldst,' \
$(git grep -l exec/cpu_ldst.h)

Signed-off-by: Philippe Mathieu-Daud

exec: Restrict 'cpu_ldst.h' to accel/tcg/

Mechanical change using:

$ sed -i -e 's,exec/cpu_ldst,accel/tcg/cpu-ldst,' \
$(git grep -l exec/cpu_ldst.h)

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

exec/cpu-all: extract tlb flags defines to exec/tlb-flags.h

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by:

exec/cpu-all: extract tlb flags defines to exec/tlb-flags.h

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250320223002.2915728-3-pierrick.bouvier@linaro.org>

show more ...

include/system: Move exec/ram_addr.h to system/ram_addr.h

Convert the existing includes with sed.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bou

include/system: Move exec/ram_addr.h to system/ram_addr.h

Convert the existing includes with sed.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

accel/tcg: Rename 'hw/core/tcg-cpu-ops.h' -> 'accel/tcg/cpu-ops.h'

TCGCPUOps structure makes more sense in the accelerator context
rather than hardware emulation. Move it under the accel/tcg/ scope.

accel/tcg: Rename 'hw/core/tcg-cpu-ops.h' -> 'accel/tcg/cpu-ops.h'

TCGCPUOps structure makes more sense in the accelerator context
rather than hardware emulation. Move it under the accel/tcg/ scope.

Mechanical change doing:

$ sed -i -e 's,hw/core/tcg-cpu-ops.h,accel/tcg/cpu-ops.h,g' \
$(git grep -l hw/core/tcg-cpu-ops.h)

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20250123234415.59850-11-philmd@linaro.org>

show more ...

user: Move various declarations out of 'exec/exec-all.h'

Move various declarations related to user emulation
from "exec/exec-all.h" to "user/cpu_loop.h".

Signed-off-by: Philippe Mathieu-Daudé <phil

user: Move various declarations out of 'exec/exec-all.h'

Move various declarations related to user emulation
from "exec/exec-all.h" to "user/cpu_loop.h".

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20241212185341.2857-19-philmd@linaro.org>

show more ...

accel/tcg: Move user-related declarations out of 'exec/cpu-all.h' (2/4)

Move declarations related to page protection under user
emulation from "exec/cpu-all.h" to "user/page-protection.h".

Signed-o

accel/tcg: Move user-related declarations out of 'exec/cpu-all.h' (2/4)

Move declarations related to page protection under user
emulation from "exec/cpu-all.h" to "user/page-protection.h".

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20241212185341.2857-13-philmd@linaro.org>

show more ...

target/arm/mte: Restrict 'exec/ram_addr.h' to system emulation

"exec/ram_addr.h" contains system specific declarations.
Restrict its inclusion to sysemu to avoid build errors
when refactoring.

Sign

target/arm/mte: Restrict 'exec/ram_addr.h' to system emulation

"exec/ram_addr.h" contains system specific declarations.
Restrict its inclusion to sysemu to avoid build errors
when refactoring.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20241114011310.3615-10-philmd@linaro.org>

show more ...

target/arm: Make some MTE helpers widely available

Make the MTE helpers allocation_tag_mem_probe, load_tag1, and store_tag1
available to other subsystems.

Signed-off-by: Gustavo Romero <gustavo.rom

target/arm: Make some MTE helpers widely available

Make the MTE helpers allocation_tag_mem_probe, load_tag1, and store_tag1
available to other subsystems.

Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240628050850.536447-6-gustavo.romero@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240705084047.857176-35-alex.bennee@linaro.org>

show more ...

target/arm: Fix exception case in allocation_tag_mem_probe

If page in 'ptr_access' is inaccessible and probe is 'true'
allocation_tag_mem_probe should not throw an exception, but currently it
does,

target/arm: Fix exception case in allocation_tag_mem_probe

If page in 'ptr_access' is inaccessible and probe is 'true'
allocation_tag_mem_probe should not throw an exception, but currently it
does, so fix it.

Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20240628050850.536447-5-gustavo.romero@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240705084047.857176-34-alex.bennee@linaro.org>

show more ...

exec/cpu: Extract page-protection definitions to page-protection.h

Extract page-protection definitions from "exec/cpu-all.h"
to "exec/page-protection.h".

The list of files requiring the new header

exec/cpu: Extract page-protection definitions to page-protection.h

Extract page-protection definitions from "exec/cpu-all.h"
to "exec/page-protection.h".

The list of files requiring the new header was generated
using:

$ git grep -wE \
'PAGE_(READ|WRITE|EXEC|RWX|VALID|ANON|RESERVED|TARGET_.|PASSTHROUGH)'

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Nicholas Piggin <npiggin@gmail.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20240427155714.53669-3-philmd@linaro.org>

show more ...

target/arm: Split out arm_env_mmu_index

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

target/arm: Correct MTE tag checking for reverse-copy MOPS

When we are doing a FEAT_MOPS copy that must be performed backwards,
we call mte_mops_probe_rev(), passing it the address of the last byte

target/arm: Correct MTE tag checking for reverse-copy MOPS

When we are doing a FEAT_MOPS copy that must be performed backwards,
we call mte_mops_probe_rev(), passing it the address of the last byte
in the region we are probing. However, allocation_tag_mem_probe()
wants the address of the first byte to get the tag memory for.
Because we passed it (ptr, size) we could incorrectly trip the
allocation_tag_mem_probe() check for "does this access run across to
the following page", and if that following page happened not to be
valid then we would assert.

We know we will always be only dealing with a single page because the
code that calls mte_mops_probe_rev() ensures that. We could make
mte_mops_probe_rev() pass 'ptr - (size - 1)' to
allocation_tag_mem_probe(), but then we would have to adjust the
returned 'mem' pointer to get back to the tag RAM for the last byte
of the region. It's simpler to just pass in a size of 1 byte,
because we know that allocation_tag_mem_probe() in pure-probe
single-page mode doesn't care about the size.

Fixes: 69c51dc3723b ("target/arm: Implement MTE tag-checking functions for FEAT_MOPS copies")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20231110162546.2192512-1-peter.maydell@linaro.org

show more ...

target/arm: Replace TARGET_PAGE_ENTRY_EXTRA

TARGET_PAGE_ENTRY_EXTRA is a macro that allows guests to specify additional
fields for caching with the full TLB entry. This macro is replaced with
a uni

target/arm: Replace TARGET_PAGE_ENTRY_EXTRA

TARGET_PAGE_ENTRY_EXTRA is a macro that allows guests to specify additional
fields for caching with the full TLB entry. This macro is replaced with
a union in CPUTLBEntryFull, thus making CPUTLB target-agnostic at the
cost of slightly inflated CPUTLBEntryFull for non-arm guests.

Note, this is needed to ensure that fields in CPUTLB don't vary in
offset between various targets.

(arm is the only guest actually making use of this feature.)

Signed-off-by: Anton Johansson <anjo@rev.ng>
Message-Id: <20230912153428.17816-2-anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

target/arm: Implement MTE tag-checking functions for FEAT_MOPS copies

The FEAT_MOPS memory copy operations need an extra helper routine
for checking for MTE tag checking failures beyond the ones we

target/arm: Implement MTE tag-checking functions for FEAT_MOPS copies

The FEAT_MOPS memory copy operations need an extra helper routine
for checking for MTE tag checking failures beyond the ones we
already added for memory set operations:
* mte_mops_probe_rev() does the same job as mte_mops_probe(), but
it checks tags starting at the provided address and working
backwards, rather than forwards

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230912140434.1333369-11-peter.maydell@linaro.org

show more ...

target/arm: Implement the SETG* instructions

The FEAT_MOPS SETG* instructions are very similar to the SET*
instructions, but as well as setting memory contents they also
set the MTE tags. They are a

target/arm: Implement the SETG* instructions

The FEAT_MOPS SETG* instructions are very similar to the SET*
instructions, but as well as setting memory contents they also
set the MTE tags. They are architecturally required to operate
on tag-granule aligned regions only.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230912140434.1333369-10-peter.maydell@linaro.org

show more ...

target/arm: Implement MTE tag-checking functions for FEAT_MOPS

The FEAT_MOPS instructions need a couple of helper routines that
check for MTE tag failures:
* mte_mops_probe() checks whether there i

target/arm: Implement MTE tag-checking functions for FEAT_MOPS

The FEAT_MOPS instructions need a couple of helper routines that
check for MTE tag failures:
* mte_mops_probe() checks whether there is going to be a tag
error in the next up-to-a-page worth of data
* mte_check_fail() is an existing function to record the fact
of a tag failure, which we need to make global so we can
call it from helper-a64.c

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230912140434.1333369-7-peter.maydell@linaro.org

show more ...

target/arm: New function allocation_tag_mem_probe()

For the FEAT_MOPS operations, the existing allocation_tag_mem()
function almost does what we want, but it will take a watchpoint
exception even fo

target/arm: New function allocation_tag_mem_probe()

For the FEAT_MOPS operations, the existing allocation_tag_mem()
function almost does what we want, but it will take a watchpoint
exception even for an ra == 0 probe request, and it requires that the
caller guarantee that the memory is accessible. For FEAT_MOPS we
want a function that will not take any kind of exception, and will
return NULL for the not-accessible case.

Rename allocation_tag_mem() to allocation_tag_mem_probe() and add an
extra 'probe' argument that lets us distinguish these cases;
allocation_tag_mem() is now a wrapper that always passes 'false'.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230912140434.1333369-6-peter.maydell@linaro.org

show more ...

target/arm: Remove unused allocation_tag_mem() argument

The allocation_tag_mem() function takes an argument tag_size,
but it never uses it. Remove the argument. In mte_probe_int()
in particular this

target/arm: Remove unused allocation_tag_mem() argument

The allocation_tag_mem() function takes an argument tag_size,
but it never uses it. Remove the argument. In mte_probe_int()
in particular this also lets us delete the code computing
the value we were passing in.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

show more ...

target/arm: Support more GM blocksizes

Support all of the easy GM block sizes.
Use direct memory operations, since the pointers are aligned.

While BS=2 (16 bytes, 1 tag) is a legal setting, that re

target/arm: Support more GM blocksizes

Support all of the easy GM block sizes.
Use direct memory operations, since the pointers are aligned.

While BS=2 (16 bytes, 1 tag) is a legal setting, that requires
an atomic store of one nibble. This is not difficult, but there
is also no point in supporting it until required.

Note that cortex-a710 sets GM blocksize to match its cacheline
size of 64 bytes. I expect many implementations will also
match the cacheline, which makes 16 bytes very unlikely.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20230811214031.171020-4-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

target/arm: Allow cpu to configure GM blocksize

Previously we hard-coded the blocksize with GMID_EL1_BS.
But the value we choose for -cpu max does not match the
value that cortex-a710 uses.

Mirror

target/arm: Allow cpu to configure GM blocksize

Previously we hard-coded the blocksize with GMID_EL1_BS.
But the value we choose for -cpu max does not match the
value that cortex-a710 uses.

Mirror the way we handle dcz_blocksize.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230811214031.171020-3-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

target/arm: Check alignment in helper_mte_check

Fixes a bug in that with SCTLR.A set, we should raise any
alignment fault before raising any MTE check fault.

Reviewed-by: Peter Maydell <peter.mayde

target/arm: Check alignment in helper_mte_check

Fixes a bug in that with SCTLR.A set, we should raise any
alignment fault before raising any MTE check fault.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230530191438.411344-15-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

softmmu: Restrict cpu_check_watchpoint / address_matches to TCG accel

Both cpu_check_watchpoint() and cpu_watchpoint_address_matches()
are specific to TCG system emulation. Declare them in "tcg-cpu-

softmmu: Restrict cpu_check_watchpoint / address_matches to TCG accel

Both cpu_check_watchpoint() and cpu_watchpoint_address_matches()
are specific to TCG system emulation. Declare them in "tcg-cpu-ops.h"
to be sure accessing them from non-TCG code is a compilation error.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230328173117.15226-2-philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...