x86: Provide result of RDMSR from "safe" variant

Provide the result of RDMSR from rdmsr_safe() so that it can be used by
tests that are unsure whether or not RDMSR will fault, but want the value
if

x86: Provide result of RDMSR from "safe" variant

Provide the result of RDMSR from rdmsr_safe() so that it can be used by
tests that are unsure whether or not RDMSR will fault, but want the value
if it doesn't fault.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220608235238.3881916-6-seanjc@google.com

show more ...

x86: Use "safe" helpers to implement unsafe CRs accessors

Use the "safe" helpers to read and write CR0, CR3, and CR4, so that an
unexpected fault results in a detailed message instead of an generic

x86: Use "safe" helpers to implement unsafe CRs accessors

Use the "safe" helpers to read and write CR0, CR3, and CR4, so that an
unexpected fault results in a detailed message instead of an generic
"unexpected fault" explosion.

Do not give RDMSR/WRMSR the same treatment. KUT's exception fixup uses
per-CPU data and thus needs a stable GS.base. Various tests modify
MSR_GS_BASE and routing them through the safe variants will cause
fireworks when trying to clear/read the exception vector with a garbage
GS.base.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220608235238.3881916-5-seanjc@google.com

show more ...

x86: Use "safe" terminology instead of "checking"

Rename all helpers that eat (and return) exceptions to use "safe" instead
of "checking". This aligns KUT with the kernel and KVM selftests.

Signed

x86: Use "safe" terminology instead of "checking"

Rename all helpers that eat (and return) exceptions to use "safe" instead
of "checking". This aligns KUT with the kernel and KVM selftests.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220608235238.3881916-4-seanjc@google.com

show more ...

x86: Replace spaces with tables in processor.h

Fix the myriad instances of using spaces instead of tabs in processor.h.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@go

x86: Replace spaces with tables in processor.h

Fix the myriad instances of using spaces instead of tabs in processor.h.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220608235238.3881916-3-seanjc@google.com

show more ...

x86: Use BIT() to define architectural bits

Use BIT() to define bits in EFLAGS, CR0, and CR4. Intel's SDM and AMD's
APM reference flags/features by the bit number, not by their mask, making
it absu

x86: Use BIT() to define architectural bits

Use BIT() to define bits in EFLAGS, CR0, and CR4. Intel's SDM and AMD's
APM reference flags/features by the bit number, not by their mask, making
it absurdly difficult to audit and/or add definitions.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220608235238.3881916-2-seanjc@google.com

show more ...

x86: Add ap_online() to consolidate final "AP is alive!" code

Add ap_online() to consolidate the last stage of onlining an AP CPU.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https

x86: Add ap_online() to consolidate final "AP is alive!" code

Add ap_online() to consolidate the last stage of onlining an AP CPU.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-14-seanjc@google.com

show more ...

x86: Rename ap_init() to bringup_aps()

Rename the helper that wakes and waits for APs to bringup_aps(), ap_init()
is terribly confusing because it's called from the BSP, not APs.

Signed-off-by: Sea

x86: Rename ap_init() to bringup_aps()

Rename the helper that wakes and waits for APs to bringup_aps(), ap_init()
is terribly confusing because it's called from the BSP, not APs.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-13-seanjc@google.com

show more ...

x86: Provide a common 64-bit AP entrypoint for EFI and non-EFI

ap_start64() currently serves as the 64-bit entrypoint for non-EFI
tests.

Having ap_start64() and save_id() written in asm prevents sh

x86: Provide a common 64-bit AP entrypoint for EFI and non-EFI

ap_start64() currently serves as the 64-bit entrypoint for non-EFI
tests.

Having ap_start64() and save_id() written in asm prevents sharing these
routines between EFI and non-EFI tests.

Rewrite them in C and use ap_start64 as the 64-bit entrypoint in the EFI
boot flow.

With this, EFI tests support -smp > 1. smptest.efi now passes.

Cc: Andrew Jones <drjones@redhat.com>
Cc: Marc Orr <marcorr@google.com>
Cc: Zixuan Wang <zxwang42@gmail.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Thomas.Lendacky@amd.com
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Borislav Petkov <bp@alien8.de>
Signed-off-by: Varad Gautam <varad.gautam@suse.com>
[sean: reset APIC before TSS setup, halt in AP wait loop, add comment]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-12-seanjc@google.com

show more ...

x86: efi, smp: Transition APs from 16-bit to 32-bit mode

Sending INIT/SIPI to APs from ap_init() resets them into 16-bit mode
to loop into sipi_entry().

To drive the APs into 32-bit mode, the SIPI

x86: efi, smp: Transition APs from 16-bit to 32-bit mode

Sending INIT/SIPI to APs from ap_init() resets them into 16-bit mode
to loop into sipi_entry().

To drive the APs into 32-bit mode, the SIPI vector needs:
1. A GDT descriptor reachable from 16-bit code (gdt32_descr).
2. A 32-bit entrypoint reachable from 16-bit code (ap_start32).
3. The locations of GDT and the 32-bit entrypoint.

Setting these up at compile time (like on non-EFI builds) is not
possible since EFI builds with -shared -fPIC and efistart64.S cannot
reference any absolute addresses.

Relative addressing is unavailable on 16-bit mode.

Moreover, EFI may not load the 32-bit entrypoint to be reachable from
16-bit mode.

To overcome these problems,
1. Fill the GDT descriptor at runtime after relocating
[sipi_entry-sipi_end] to lowmem. Since sipi_entry does not know the
address of this descriptor, use the last two bytes of SIPI page to
communicate it.
2. Place a call gate in the GDT to point to ap_start32.
3. Popluate sipi_entry() to lcall to ap_start32.

With this, the APs can transition to 32-bit mode and loop at a known
location.

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-11-seanjc@google.com

show more ...

x86: efi: Provide percpu storage

UEFI tests do not update MSR_GS_BASE during bringup, and continue
using the GS_BASE set up by the UEFI implementation for percpu
storage.

Update this MSR during set

x86: efi: Provide percpu storage

UEFI tests do not update MSR_GS_BASE during bringup, and continue
using the GS_BASE set up by the UEFI implementation for percpu
storage.

Update this MSR during setup_segments64() to allow storing percpu
data at a sane location reserved by the testcase, and ensure that
this happens before any operation that ends up storing to the percpu
space.

Since apic_ops (touched by reset_apic()) is percpu, move reset_apic()
to happen after setup_gdt_tss(); pre_boot_apic_id() works with either
xAPIC or x2APIC, so it's not necessary to put the APIC back into xAPIC]
mode prior to configuring the per-vCPU TSS, GS.base, etc...

With this, ap_init() can now use percpu apic_ops via apic_icr_write().

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
[sean: add comment to call out GS.base dependency]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-9-seanjc@google.com

show more ...

x86: apic: Play nice with x2APIC being enabled when getting "pre-boot" ID

Retrieve the "pre-boot" APIC ID via the x2APIC interface if x2APIC is
enabled instead of assuming that the APIC is always in

x86: apic: Play nice with x2APIC being enabled when getting "pre-boot" ID

Retrieve the "pre-boot" APIC ID via the x2APIC interface if x2APIC is
enabled instead of assuming that the APIC is always in xAPIC mode. EFI
has a catch-22 where it needs the APID ID to initialize the per-vCPU
GS.base, but calling reset_apic() = >disable_apic() needs GS.base to be
correctly initialized in order to set the correct APIC ops. Play nice
with either xAPIC or x2APIC so that EFI can be used for SMP tests, in
particular the SVM INIT-SIPI tests which send APs back through the boot
sequence while x2APIC is enabled.

Alternatively, disabling x2APIC and updating the APIC ops could be split,
but there's no obvious advantage in doing so. Retrieving the pre-boot
APIC ID isn't a hot path, i.e. the cost of the RDMSR is likely negligible,
and letting callers force xAPIC without updating the ops isn't any less
fragile.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220725201336.2158604-2-seanjc@google.com

show more ...

x86: Move load_gdt_tss() to desc.c

Split load_gdt_tss() functionality into:
1. Load gdt/tss
2. Setup segments in 64-bit mode and update %cs via far-return

and move load_gdt_tss() to desc.c to share

x86: Move load_gdt_tss() to desc.c

Split load_gdt_tss() functionality into:
1. Load gdt/tss
2. Setup segments in 64-bit mode and update %cs via far-return

and move load_gdt_tss() to desc.c to share this code between
EFI and non-EFI tests.

Move the segment setup code specific to EFI into
setup.c:setup_segments64().

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-7-seanjc@google.com

show more ...

x86: desc: Split IDT entry setup into a generic helper

EFI bootstrapping code configures a call gate in a later commit to jump
from 16-bit to 32-bit code.

Introduce a set_desc_entry() routine which

x86: desc: Split IDT entry setup into a generic helper

EFI bootstrapping code configures a call gate in a later commit to jump
from 16-bit to 32-bit code.

Introduce a set_desc_entry() routine which can be used to fill both
an interrupt descriptor and a call gate descriptor on x86.

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-6-seanjc@google.com

show more ...

x86: Move load_idt() to desc.c

This allows sharing IDT setup code between EFI (-fPIC) and
non-EFI builds.

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Signed-off-by: Sean Christopherson <sea

x86: Move load_idt() to desc.c

This allows sharing IDT setup code between EFI (-fPIC) and
non-EFI builds.

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-5-seanjc@google.com

show more ...

x86: Move ap_init() to smp.c

ap_init() copies the SIPI vector to lowmem, sends INIT/SIPI to APs
and waits on the APs to come up.

Port this routine to C from asm and move it to smp.c to allow sharin

x86: Move ap_init() to smp.c

ap_init() copies the SIPI vector to lowmem, sends INIT/SIPI to APs
and waits on the APs to come up.

Port this routine to C from asm and move it to smp.c to allow sharing
this functionality between the EFI (-fPIC) and non-EFI builds.

Call ap_init() from the EFI setup path to reset the APs to a known
location.

Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20220615232943.1465490-4-seanjc@google.com

show more ...

x86: Add flags to control behavior of set_mmu_range()

Add flags to control the behavior of setup_mmu_range() so that it can be
used in a future patch to dynamically build SVM's Nested Page Tables,
e

x86: Add flags to control behavior of set_mmu_range()

Add flags to control the behavior of setup_mmu_range() so that it can be
used in a future patch to dynamically build SVM's Nested Page Tables,
e.g. all NPT accesses are treated as user accesses.

Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Manali Shukla <manali.shukla@amd.com>
Link: https://lore.kernel.org/r/20220628113853.392569-5-manali.shukla@amd.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

show more ...

x86: apic: Make xAPIC and I/O APIC pointers static

Make the pointers to the xAPIC and I/O APIC static as there are no users
outside of apic.c. Opportunistically use #defines for the default values

x86: apic: Make xAPIC and I/O APIC pointers static

Make the pointers to the xAPIC and I/O APIC static as there are no users
outside of apic.c. Opportunistically use #defines for the default values
instead of open coding magic numbers.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: apic: Track APIC ops on a per-cpu basis

Track the virtual function table to handle xAPIC vs. x2APIC on a per-cpu
basis. Using a common global is racy as nothing in KUT synchronizes
CPUs when s

x86: apic: Track APIC ops on a per-cpu basis

Track the virtual function table to handle xAPIC vs. x2APIC on a per-cpu
basis. Using a common global is racy as nothing in KUT synchronizes
CPUs when switching to/from x2APIC.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-8-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: apic: Replace spaces with tabs

Replace spaces with tabs in apic.c. No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-7-

x86: apic: Replace spaces with tabs

Replace spaces with tabs in apic.c. No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-7-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: Add proper helpers for per-cpu reads/writes

Add helpers to read/write per-cpu data instead of open coding access
with gs: and magic numbers. Keeping track of what offsets are used for
what and

x86: Add proper helpers for per-cpu reads/writes

Add helpers to read/write per-cpu data instead of open coding access
with gs: and magic numbers. Keeping track of what offsets are used for
what and by whom is a nightmare.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-6-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: desc: Replace spaces with tabs

Replace spaces with tabs in smp.c, and opportunistically clean up a
handful of minor coding style violations.

No functional change intended.

Signed-off-by: Sean

x86: desc: Replace spaces with tabs

Replace spaces with tabs in smp.c, and opportunistically clean up a
handful of minor coding style violations.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: smp: Replace spaces with tabs

Replace spaces with tabs in smp.c, and opportunistically clean up a
handful of minor coding style violations.

No functional change intended.

Signed-off-by: Sean

x86: smp: Replace spaces with tabs

Replace spaces with tabs in smp.c, and opportunistically clean up a
handful of minor coding style violations.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: Always use legacy xAPIC to get APIC ID during TSS setup

Force use of xAPIC to retrieve the APIC ID during TSS setup to fix an
issue where an AP can switch apic_ops to point at x2apic_ops before

x86: Always use legacy xAPIC to get APIC ID during TSS setup

Force use of xAPIC to retrieve the APIC ID during TSS setup to fix an
issue where an AP can switch apic_ops to point at x2apic_ops before
setup_tss() completes, leading to a #GP and triple fault due to trying
to read an x2APIC MSR without x2APIC being enabled.

A future patch will make apic_ops a per-cpu pointer, but that's not of
any help for 32-bit, which uses the APIC ID to determine the GS selector,
i.e. 32-bit KUT has a chicken-and-egg problem. All setup_tss() callers
ensure the local APIC is in xAPIC mode, so just force use of xAPIC in
this case.

Fixes: 7e33895 ("x86: Move 32-bit GDT and TSS to desc.c")
Fixes: dbd3800 ("x86: Move 64-bit GDT and TSS to desc.c")
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220121231852.1439917-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

x86: efi: Fix pagetable creation

setup_page_table() ends up filling invalid page table entries
at ptl2 due to improper typecasting. This sometimes leads to
unhandled pagefaults when writing to APIC

x86: efi: Fix pagetable creation

setup_page_table() ends up filling invalid page table entries
at ptl2 due to improper typecasting. This sometimes leads to
unhandled pagefaults when writing to APIC registers. Fix it.

Fixes: e6f65fa464 ("x86 UEFI: Set up page tables")
Signed-off-by: Varad Gautam <varad.gautam@suse.com>
Message-Id: <20220406123312.12986-1-varad.gautam@suse.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

svm: add test for pause filter and threshold

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20220322205613.250925-10-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@red

svm: add test for pause filter and threshold

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20220322205613.250925-10-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...