pmu: fix macOS build

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Merge tag 'kvm-x86-2025.02.21' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

KVM-Unit-Tests x86 changes:

- Expand the per-CPU data+stack area to 12KiB per CPU to reduce the
probability

Merge tag 'kvm-x86-2025.02.21' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

KVM-Unit-Tests x86 changes:

- Expand the per-CPU data+stack area to 12KiB per CPU to reduce the
probability of tests overflowing their stack and clobbering pre-CPU
data.

- Add testcases for LA57 canonical checks.

- Add testcases for LAM.

- Add a smoke test to make sure KVM doesn't bleed split-lock #AC/#DB into
the guest.

- Fix many warts and bugs in the PMU test, and prepare it for PMU version 5
and beyond.

- Many misc fixes and cleanups.

show more ...

x86: pmu: Optimize emulated instruction validation

For support CPUs supporting PERF_GLOBAL_CTRL MSR, the validation for
emulated instruction can be improved to check against precise counts for
instr

x86: pmu: Optimize emulated instruction validation

For support CPUs supporting PERF_GLOBAL_CTRL MSR, the validation for
emulated instruction can be improved to check against precise counts for
instructions and branches events instead of a rough range.

Move enabling and disabling PERF_GLOBAL_CTRL MSR into kvm_fep_asm blob,
thus instructions and branches events can be verified against precise
counts.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-19-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Adjust lower boundary of branch-misses event

Since the IBPB command is added to force to trigger a branch miss at
least, the lower boundary of branch misses event is increased to 1 by
defa

x86: pmu: Adjust lower boundary of branch-misses event

Since the IBPB command is added to force to trigger a branch miss at
least, the lower boundary of branch misses event is increased to 1 by
default. For these CPUs without IBPB support, adjust dynamically the
lower boundary to 0 to avoid false positive.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-18-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Add IBPB indirect jump asm blob

Currently the lower boundary of branch misses event is set to 0.
Strictly speaking 0 shouldn't be a valid count since it can't tell us if
branch misses even

x86: pmu: Add IBPB indirect jump asm blob

Currently the lower boundary of branch misses event is set to 0.
Strictly speaking 0 shouldn't be a valid count since it can't tell us if
branch misses event counter works correctly or even disabled. Whereas
it's also possible and reasonable that branch misses event count is 0
especailly for such simple loop() program with advanced branch
predictor.

To eliminate such ambiguity and make branch misses event verification
more acccurately, an extra IBPB indirect jump asm blob is appended and
IBPB command is leveraged to clear the branch target buffer and force to
cause a branch miss for the indirect jump.

Suggested-by: Jim Mattson <jmattson@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-17-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Adjust lower boundary of llc-misses event to 0 for legacy CPUs

For these legacy Intel CPUs without clflush/clflushopt support, there is
on way to force to trigger a LLC miss and the measur

x86: pmu: Adjust lower boundary of llc-misses event to 0 for legacy CPUs

For these legacy Intel CPUs without clflush/clflushopt support, there is
on way to force to trigger a LLC miss and the measured llc misses is
possible to be 0. Thus adjust the lower boundary of llc-misses event to
0 to avoid possible false positive.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-16-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Improve LLC misses event verification

When running pmu test on SPR, sometimes the following failure is
reported.

1 <= 0 <= 1000000
FAIL: Intel: llc misses-4

Currently The LLC misses

x86: pmu: Improve LLC misses event verification

When running pmu test on SPR, sometimes the following failure is
reported.

1 <= 0 <= 1000000
FAIL: Intel: llc misses-4

Currently The LLC misses occurring only depends on probability. It's
possible that there is no LLC misses happened in the whole loop(),
especially along with processors have larger and larger cache size just
like what we observed on SPR.

Thus, add clflush instruction into the loop() asm blob and ensure once
LLC miss is triggered at least.

Suggested-by: Jim Mattson <jmattson@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-15-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Improve instruction and branches events verification

If HW supports GLOBAL_CTRL MSR, enabling and disabling PMCs are moved in
__precise_count_loop(). Thus, instructions and branches events

x86: pmu: Improve instruction and branches events verification

If HW supports GLOBAL_CTRL MSR, enabling and disabling PMCs are moved in
__precise_count_loop(). Thus, instructions and branches events can be
verified against a precise count instead of a rough range.

Unfortunately, AMD CPUs count VMRUN as a branch instruction in guest
context, which leads to intermittent failures as the counts will vary
depending on how many asynchronous exits occur while running the measured
code, e.g. if the host takes IRQs, NMIs, etc.

So only enable this precise check for Intel processors.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/all/6d512a14-ace1-41a3-801e-0beb41425734@amd.com
[sean: explain AMD VMRUN behavior, use "INSNS"]
Link: https://lore.kernel.org/r/20250215013636.1214612-14-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Enable and disable PMCs in loop() asm blob

Currently enabling PMCs, executing loop() and disabling PMCs are divided
3 separated functions. So there could be other instructions executed
bet

x86: pmu: Enable and disable PMCs in loop() asm blob

Currently enabling PMCs, executing loop() and disabling PMCs are divided
3 separated functions. So there could be other instructions executed
between enabling PMCS and running loop() or running loop() and disabling
PMCs, e.g. if there are multiple counters enabled in measure_many()
function, the instructions which enabling the 2nd and more counters
would be counted in by the 1st counter.

So current implementation can only verify the correctness of count by an
rough range rather than a precise count even for instructions and
branches events. Strictly speaking, this verification is meaningless as
the test could still pass even though KVM vPMU has something wrong and
reports an incorrect instructions or branches count which is in the rough
range.

Thus, move the PMCs enabling and disabling into the loop() asm blob and
ensure only the loop asm instructions would be counted, then the
instructions or branches events can be verified with an precise count
instead of an rough range.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-13-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Use macro to replace hard-coded instructions event index

Replace hard-coded instruction event index with macro to avoid possible
mismatch issue if new event is added in the future and caus

x86: pmu: Use macro to replace hard-coded instructions event index

Replace hard-coded instruction event index with macro to avoid possible
mismatch issue if new event is added in the future and cause
instructions event index changed, but forget to update the hard-coded
event index.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Use macro to replace hard-coded ref-cycles event index

Replace hard-coded ref-cycles event index with macro to avoid possible
mismatch issue if new event is added in the future and cause r

x86: pmu: Use macro to replace hard-coded ref-cycles event index

Replace hard-coded ref-cycles event index with macro to avoid possible
mismatch issue if new event is added in the future and cause ref-cycles
event index changed, but forget to update the hard-coded ref-cycles
event index.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Use macro to replace hard-coded branches event index

Currently the branches event index is a hard-coded number. User could
add new events and cause the branches event index changes in the

x86: pmu: Use macro to replace hard-coded branches event index

Currently the branches event index is a hard-coded number. User could
add new events and cause the branches event index changes in the future,
but don't notice the hard-coded event index and forget to update the
event index synchronously, then the issue comes.

Thus, replace the hard-coded index to a macro.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Fix cycles event validation failure

When running pmu test on SPR, sometimes the following failure is
reported.

PMU version: 2
GP counters: 8
GP counter width: 48
Mask l

x86: pmu: Fix cycles event validation failure

When running pmu test on SPR, sometimes the following failure is
reported.

PMU version: 2
GP counters: 8
GP counter width: 48
Mask length: 8
Fixed counters: 3
Fixed counter width: 48
1000000 <= 55109398 <= 50000000
FAIL: Intel: core cycles-0
1000000 <= 18279571 <= 50000000
PASS: Intel: core cycles-1
1000000 <= 12238092 <= 50000000
PASS: Intel: core cycles-2
1000000 <= 7981727 <= 50000000
PASS: Intel: core cycles-3
1000000 <= 6984711 <= 50000000
PASS: Intel: core cycles-4
1000000 <= 6773673 <= 50000000
PASS: Intel: core cycles-5
1000000 <= 6697842 <= 50000000
PASS: Intel: core cycles-6
1000000 <= 6747947 <= 50000000
PASS: Intel: core cycles-7

The count of the "core cycles" on first counter would exceed the upper
boundary and leads to a failure, and then the "core cycles" count would
drop gradually and reach a stable state.

That looks reasonable. The "core cycles" event is defined as the 1st
event in xxx_gp_events[] array and it is always verified at first.
when the program loop() is executed at the first time it needs to warm
up the pipeline and cache, such as it has to wait for cache is filled.
All these warm-up work leads to a quite large core cycles count which
may exceeds the verification range.

To avoid the false positive of cycles event caused by warm-up,
explicitly introduce a warm-up state before really starting
verification.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
[sean: use a for loop and an more obviously arbitrary number]
Link: https://lore.kernel.org/r/20250215013636.1214612-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Fix potential out of bound access for fixed events

Current PMU code doesn't check whether PMU fixed counter number is
larger than pre-defined fixed events. If so, it would cause memory
acc

x86: pmu: Fix potential out of bound access for fixed events

Current PMU code doesn't check whether PMU fixed counter number is
larger than pre-defined fixed events. If so, it would cause memory
access out of range.

So limit validated fixed counters number to
MIN(pmu.nr_fixed_counters, ARRAY_SIZE(fixed_events)) and print message
to warn that KUT/pmu tests need to be updated if fixed counters number
exceeds defined fixed events number.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Print measured event count if test fails

Print the measured event count if the test case fails. This helps users
quickly know why the test case fails.

Signed-off-by: Dapeng Mi <dapeng1.mi

x86: pmu: Print measured event count if test fails

Print the measured event count if the test case fails. This helps users
quickly know why the test case fails.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Enlarge cnt[] length to 48 in check_counters_many()

Considering there are already 8 GP counters and 4 fixed counters on
latest Intel processors, like Sapphire Rapids. The original cnt[] ar

x86: pmu: Enlarge cnt[] length to 48 in check_counters_many()

Considering there are already 8 GP counters and 4 fixed counters on
latest Intel processors, like Sapphire Rapids. The original cnt[] array
length 10 is definitely not enough to cover all supported PMU counters on
these new processors even through currently KVM only supports 3 fixed
counters at most. This would cause out of bound memory access and may trigger
false alarm on PMU counter validation

It's probably more and more GP and fixed counters are introduced in the
future and then directly extends the cnt[] array length to 48 once and
for all. Base on the layout of IA32_PERF_GLOBAL_CTRL and
IA32_PERF_GLOBAL_STATUS, 48 looks enough in near feature.

Reviewed-by: Jim Mattson <jmattson@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
[sean: assert() on the size]
Link: https://lore.kernel.org/r/20250215013636.1214612-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Align fields in pmu_counter_t to better pack the struct

Hoist "idx" up in the pmu_counter_t structure so that the structure is
naturally packed for 64-bit builds.

Signed-off-by: Dapeng Mi

x86: pmu: Align fields in pmu_counter_t to better pack the struct

Hoist "idx" up in the pmu_counter_t structure so that the structure is
naturally packed for 64-bit builds.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://lore.kernel.org/r/20240914101728.33148-5-dapeng1.mi@linux.intel.com
[sean: rewrite changelog]
Link: https://lore.kernel.org/r/20250215013636.1214612-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Refine fixed_events[] names

In SDM the fixed counter is numbered from 0 but currently the
fixed_events names are numbered from 1. It would cause confusion for
users. So Change the fixed_ev

x86: pmu: Refine fixed_events[] names

In SDM the fixed counter is numbered from 0 but currently the
fixed_events names are numbered from 1. It would cause confusion for
users. So Change the fixed_events[] names to number from 0 as well and
keep identical with SDM.

Reviewed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: pmu: Remove blank line and redundant space

code style changes.

Reviewed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <j

x86: pmu: Remove blank line and redundant space

code style changes.

Reviewed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://lore.kernel.org/r/20250215013636.1214612-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

Merge tag 'kvm-x86-2024.06.14' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

x86 fixes, cleanups, and new testcases:

- Add a testcase to verify that KVM doesn't inject a triple fault (or

Merge tag 'kvm-x86-2024.06.14' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

x86 fixes, cleanups, and new testcases:

- Add a testcase to verify that KVM doesn't inject a triple fault (or any
other "error") if a nested VM is run with an EP4TA pointing MMIO.

- Play nice with CR4.CET in test_vmxon_bad_cr()

- Force emulation when testing MSR_IA32_FLUSH_CMD to workaround an issue where
Skylake CPUs don't follow the architecturally defined behavior, and so that
the test doesn't break if/when new bits are supported by future CPUs.

- Rework the async #PF test to support IRQ-based page-ready notifications.

- Fix a variety of issues related to adaptive PEBS.

- Add several nested VMX tests for virtual interrupt delivery and posted
interrupts.

- Ensure PAT is loaded with the default value after the nVMX PAT tests
(failure to do so was causing tests to fail due to all memory being UC).

- Misc cleanups.

show more ...

x86/pmu: Clear mask in PMI handler to allow delivering subsequent PMIs

Clear mask in PMI handler to allow subsequent PMI delivered. SDM 11.5.1
Local Vector Table mentions: "When the local APIC handl

x86/pmu: Clear mask in PMI handler to allow delivering subsequent PMIs

Clear mask in PMI handler to allow subsequent PMI delivered. SDM 11.5.1
Local Vector Table mentions: "When the local APIC handles a
performance-monitoring counters interrupt, it automatically sets the mask
flag in the LVT performance counter register. This flag is set to 1 on
reset. It can be cleared only by software."

Previously KVM vPMU does not set the mask when injecting the PMI, so there
is no issue for this test to work correctly. To ensure the test still works
after the KVM fix merges, add the mask clearing behavior to PMI handler.

Signed-off-by: Mingwei Zhang <mizhang@google.com>
Link: https://lore.kernel.org/r/20230927225441.871050-1-mizhang@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

lib: Use vmalloc.h for setup_mmu definition

There is no good reason to put setup_vm in libcflat.h when it's
defined in vmalloc.h.

Acked-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: Nich

lib: Use vmalloc.h for setup_mmu definition

There is no good reason to put setup_vm in libcflat.h when it's
defined in vmalloc.h.

Acked-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20240504122841.1177683-24-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

x86: Fix various typos

Fix typos that have been discovered with the "codespell" utility.

Message-ID: <20240109132902.129377-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

Merge tag 'kvm-x86-2023.09.01' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

x86 fixes, cleanups, and new testcases, and a few generic changes

- Fix a bug in runtime.bash that caused it t

Merge tag 'kvm-x86-2023.09.01' of https://github.com/kvm-x86/kvm-unit-tests into HEAD

x86 fixes, cleanups, and new testcases, and a few generic changes

- Fix a bug in runtime.bash that caused it to mishandle "check" strings with
multiple entries, e.g. a test that depends on multiple module params
- Make the PMU tests depend on vPMU support being enabled in KVM
- Fix PMU's forced emulation test on CPUs with full-width writes
- Add a PMU testcase for measuring TSX transactional cycles
- Nested SVM testcase for virtual NMIs
- Move a pile of code to ASM_TRY() and "safe" helpers
- Set up the guest stack in the LBRV tests so that the tests don't fail if the
compiler decides to generate function calls in guest code
- Ignore the "mispredict" flag in nSVM's LBRV tests to fix false failures
- Clean up usage of helpers that disable interrupts, e.g. stop inserting
unnecessary nops
- Add helpers to dedup code for programming the APIC timer
- Fix a variety of bugs in nVMX testcases related to being a 64-bit host

show more ...

x86: replace irq_{enable|disable}() with sti()/cli()

This removes a layer of indirection which is strictly
speaking not needed since its x86 code anyway.

Signed-off-by: Maxim Levitsky <mlevitsk@red

x86: replace irq_{enable|disable}() with sti()/cli()

This removes a layer of indirection which is strictly
speaking not needed since its x86 code anyway.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Link: https://lore.kernel.org/r/20221122161152.293072-2-mlevitsk@redhat.com
[sean: move and reword IRQ shadow to be above and about safe_halt()]
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...