kvm tools: Allow piping debug output to file descriptor

This patch makes debug output go to a 'debug_fd' instead of stdout.

Doing so allows us to send the output to a different console when
require

kvm tools: Allow piping debug output to file descriptor

This patch makes debug output go to a 'debug_fd' instead of stdout.

Doing so allows us to send the output to a different console when
required.

This patch also changes the behaviour of 'kvm debug' to show the debug
output in the console that executed the debug command instead of in the
console of the guest.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Enable fast string operations

Recent kernels check for MSR_IA32_MISC_ENABLE_FAST_STRING in the
MSR_IA32_MISC_ENABLE MSR before enabling reps/movs memcpy.

So far we didn't set it, and got

kvm tools: Enable fast string operations

Recent kernels check for MSR_IA32_MISC_ENABLE_FAST_STRING in the
MSR_IA32_MISC_ENABLE MSR before enabling reps/movs memcpy.

So far we didn't set it, and got a slower memcpy and a warning:

[ 0.000000] Disabled fast string operations

This patch enables fast string operations.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Make keyboard termination go through regular termination path

Instead of exiting directly when a user enters 'ctrl x + a', go through
the regular termination path by stopping all VCPUs an

kvm tools: Make keyboard termination go through regular termination path

Instead of exiting directly when a user enters 'ctrl x + a', go through
the regular termination path by stopping all VCPUs and letting the
main thread handle it.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Fix guest single-stepping setup

"K. Watts" writes:

When the singlestep is enabled the ioctl to sent out when the kvm_cpu
is initialized (kvm-cpu.c in the for loop that gets each vcpu

kvm tools: Fix guest single-stepping setup

"K. Watts" writes:

When the singlestep is enabled the ioctl to sent out when the kvm_cpu
is initialized (kvm-cpu.c in the for loop that gets each vcpu built).
When the ioct goes out the CPU is sitting at the initialization vector
of 0xf000:0xfff0 on CPU #0 and 0x000000 on the other SMP CPUs. The
new host kernel code that handles setting the TF was changed in 2.6.32
and again at 2.6.38. 2.6.32 seems just flat broken, but 2.6.38 checks
that the linear address of the RIP matches what it was when the
KVM_GUESTDBG_SINGLESTEP flag was set. Because the kvm-tool doesn't
start the CPU at the initialization vector (0xfff0) (0x7c00 for the
MBR and where ever you guys map the linux kernel to) they don't match
and the host kernel won't set the trap flag.

Basically the debug and singlestep ioclts need to happen after the CPU
has been initialized. I moved kvm_cpu__enable_singlestep to happen in
kvm_cpu__reset_vcpu after the registers are set (EIP points to boot
address) and the TRAP flags get set and all is good with the world.

Singlestepping is disabled when the guest issues a CLI because the
Linux host doesn't support features in new Intel and AMD CPUs. We can
sort of "shadow" the interrupt mask and still get the CPU to trap out
at ever instruction even when the guest has disabled interrupts on the
CPU. I have to get the bios disk handler working completely first,
but that may be my next task so that we can trace all the CPU
instructions. My current hack is to just re-enable the trap flag
every time a VMEXIT occurs. I get enough instructions to get me by.

This patch fixes the problem by moving the kvm_cpu__enable_singlestep() into
kvm_cpu__start().

Suggested-by: K. Watts <traetox@gmail.com>
Cc: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Implement keyboard reset method

Implement the keyboard reset method which allows guest kernel
to reboot the guest using the keyboard controller.

This will allow guest kernel to reboot th

kvm tools: Implement keyboard reset method

Implement the keyboard reset method which allows guest kernel
to reboot the guest using the keyboard controller.

This will allow guest kernel to reboot the guest when it needs to,
for example - kernel panic (when passing "panic=1" as kernel parameter).

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Add MMIO coalescing support

Coalescing MMIO allows us to avoid an exit every time we have a
MMIO write, instead - MMIO writes are coalesced in a ring which
can be flushed once an exit for

kvm tools: Add MMIO coalescing support

Coalescing MMIO allows us to avoid an exit every time we have a
MMIO write, instead - MMIO writes are coalesced in a ring which
can be flushed once an exit for a different reason is needed.
A MMIO exit is also trigged once the ring is full.

Coalesce all MMIO regions registered in the MMIO mapper.
Add a coalescing handler under kvm_cpu.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Add APIs to allow pausing guests

Allow pausing and unpausing guests running on the host.
Pausing a guest means that none of the VCPU threads are running
KVM_RUN until they are unpaused.

kvm tools: Add APIs to allow pausing guests

Allow pausing and unpausing guests running on the host.
Pausing a guest means that none of the VCPU threads are running
KVM_RUN until they are unpaused.

The following API functions are added:
void kvm__pause(void);
void kvm__continue(void);
void kvm__notify_paused(void);

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Exit VCPU thread only when SIGKVMEXIT is received

Currently the VCPU loop would exit when the thread received any signal.

Change behaviour to exit only when SIGKVMEXIT is received. This

kvm tools: Exit VCPU thread only when SIGKVMEXIT is received

Currently the VCPU loop would exit when the thread received any signal.

Change behaviour to exit only when SIGKVMEXIT is received. This change
prevents from the guest to terminate when unrelated signals are processed
by the thread (for example, when attaching a debugger).

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Exit properly on SMP guests

When shutting down SMP guests only VCPU #0 will receive
a KVM_EXIT_SHUTDOWN. Waiting for all VCPU threads to exit
causes them to hang.

Instead, notify all VCP

kvm tools: Exit properly on SMP guests

When shutting down SMP guests only VCPU #0 will receive
a KVM_EXIT_SHUTDOWN. Waiting for all VCPU threads to exit
causes them to hang.

Instead, notify all VCPU threads once VCPU #0 thread is terminated
so they could also stop properly.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Prefix error() and friends helpers with pr_

To look more familiar with kernel functions.

Suggested-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Ack

kvm tools: Prefix error() and friends helpers with pr_

To look more familiar with kernel functions.

Suggested-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Lookup symbol based on RIP for 'kill -3'

To make debugging easier, look up symbol from guest kernel image based on RIP
when user does 'kill -3' to the hypervisor.

Example output looks as

kvm tools: Lookup symbol based on RIP for 'kill -3'

To make debugging easier, look up symbol from guest kernel image based on RIP
when user does 'kill -3' to the hypervisor.

Example output looks as follows:

Code:
-----
rip: [<ffffffff812cb3a0>] delay_loop+30 (/home/penberg/linux/arch/x86/lib/delay.c:32)

Cc: Asias He <asias.hejun@gmail.com>
Cc: Avi Kivity <avi@redhat.com>
Cc: Cyrill Gorcunov <gorcunov@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Prasad Joshi <prasadjoshi124@gmail.com>
Cc: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Use constants for commonly used mmap flags

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Rename 'self' variables

Give proper names to vars named 'self'.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Fix and improve the CPU register dump debug output code

* Pekka Enberg <penberg@kernel.org> wrote:

> Ingo Molnar reported that 'kill -3' didn't work on his machine:
>
> * Ingo Molnar <

kvm tools: Fix and improve the CPU register dump debug output code

* Pekka Enberg <penberg@kernel.org> wrote:

> Ingo Molnar reported that 'kill -3' didn't work on his machine:
>
> * Ingo Molnar <mingo@elte.hu> wrote:
>
> > This is really cumbersome to debug - is there some good way to get to the RIP
> > that the guest is hanging in? If kvm would print that out to the host console
> > (even if it's just the raw RIP initially) on a kill -3 that would help
> > enormously.
>
> Looks like the code should be doing that already - but the ioctl(KVM_GET_SREGS)
> hangs:
>
> [pid 748] ioctl(6, KVM_GET_SREGS
>
> Avi Kivity pointed out that it's not safe to call KVM_GET_SREGS (or other vcpu
> related ioctls) from other threads:
>
> > is it not OK to call KVM_GET_SREGS from other threads than the one
> > that's doing KVM_RUN?
>
> From Documentation/kvm/api.txt:
>
> - vcpu ioctls: These query and set attributes that control the operation
> of a single virtual cpu.
>
> Only run vcpu ioctls from the same thread that was used to create the
> vcpu.
>
> Fix that up by using pthread_kill() to force the threads that are doing KVM_RUN
> to do the register dumps.
>
> Reported: Ingo Molnar <mingo@elte.hu>
> Cc: Asias He <asias.hejun@gmail.com>
> Cc: Avi Kivity <avi@redhat.com>
> Cc: Cyrill Gorcunov <gorcunov@gmail.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Prasad Joshi <prasadjoshi124@gmail.com>
> Cc: Sasha Levin <levinsasha928@gmail.com>
> Signed-off-by: Pekka Enberg <penberg@kernel.org>
> ---
> tools/kvm/kvm-run.c | 20 +++++++++++++++++---
> 1 files changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/tools/kvm/kvm-run.c b/tools/kvm/kvm-run.c
> index eb50b6a..58e2977 100644
> --- a/tools/kvm/kvm-run.c
> +++ b/tools/kvm/kvm-run.c
> @@ -127,6 +127,18 @@ static const struct option options[] = {
> OPT_END()
> };
>
> +static void handle_sigusr1(int sig)
> +{
> + struct kvm_cpu *cpu = current_kvm_cpu;
> +
> + if (!cpu)
> + return;
> +
> + kvm_cpu__show_registers(cpu);
> + kvm_cpu__show_code(cpu);
> + kvm_cpu__show_page_tables(cpu);
> +}
> +
> static void handle_sigquit(int sig)
> {
> int i;
> @@ -134,9 +146,10 @@ static void handle_sigquit(int sig)
> for (i = 0; i < nrcpus; i++) {
> struct kvm_cpu *cpu = kvm_cpus[i];
>
> - kvm_cpu__show_registers(cpu);
> - kvm_cpu__show_code(cpu);
> - kvm_cpu__show_page_tables(cpu);
> + if (!cpu)
> + continue;
> +
> + pthread_kill(cpu->thread, SIGUSR1);
> }
>
> serial8250__inject_sysrq(kvm);

i can see a couple of problems with the debug printout code, which currently
produces a stream of such dumps for each vcpu:

Registers:
rip: 0000000000000000 rsp: 00000000000016ca flags: 0000000000010002
rax: 0000000000000000 rbx: 0000000000000000 rcx: 0000000000000000
rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000
rbp: 0000000000008000 r8: 0000000000000000 r9: 0000000000000000
r10: 0000000000000000 r11: 0000000000000000 r12: 0000000000000000
r13: 0000000000000000 r14: 0000000000000000 r15: 0000000000000000
cr0: 0000000060000010 cr2: 0000000000000070 cr3: 0000000000000000
cr4: 0000000000000000 cr8: 0000000000000000
Segment registers:
register selector base limit type p dpl db s l g avl
cs f000 00000000000f0000 0000ffff 03 1 3 0 1 0 0 0
ss 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
ds 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
es 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
fs 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
gs 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
tr 0000 0000000000000000 0000ffff 0b 1 0 0 0 0 0 0
ldt 0000 0000000000000000 0000ffff 02 1 0 0 0 0 0 0
gdt 0000000000000000 0000ffff
idt 0000000000000000 0000ffff
[ efer: 0000000000000000 apic base: 00000000fee00900 nmi: enabled ]
Interrupt bitmap:
0000000000000000 0000000000000000 0000000000000000 0000000000000000
Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <cf> eb 0d 90 90 90 90 90 90 90 90 90 90 90 90 90 f6 c4 0e 75 4b
Stack:
0x000016ca: 00 00 00 00 00 00 00 00
0x000016d2: 00 00 00 00 00 00 00 00
0x000016da: 00 00 00 00 00 00 00 00
0x000016e2: 00 00 00 00 00 00 00 00

The problems are:

- This does not work very well on SMP with lots of vcpus, because the printing
is unserialized, resulting in a jumbled mess of an output, all vcpus trying
to print to the console at once, often mixing lines and characters randomly.

- stdout from a signal handler must be flushed, otherwise lines can remain
buffered if someone saves the output via 'tee' for example.

- the dumps from the various CPUs are not distinguishable - they are just
dumped after each other with no identification

- the various printouts are rather hard to parse visually - it's not easy to see
various properties "at a glance" because the dump is visually confusing.

The patch below addresses these concerns, serializes the output, tidies up the
printout, resulting in this new output:

#
# vCPU #0's dump:
#

Registers:
----------
rip: 0000000000000000 rsp: 00000000000008bc flags: 0000000000010002
rax: 0000000000000000 rbx: 0000000000000000 rcx: 0000000000000000
rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000
rbp: 0000000000008000 r8: 0000000000000000 r9: 0000000000000000
r10: 0000000000000000 r11: 0000000000000000 r12: 0000000000000000
r13: 0000000000000000 r14: 0000000000000000 r15: 0000000000000000
cr0: 0000000060000010 cr2: 0000000000000070 cr3: 0000000000000000
cr4: 0000000000000000 cr8: 0000000000000000

Segment registers:
------------------
register selector base limit type p dpl db s l g avl
cs f000 00000000000f0000 0000ffff 03 1 3 0 1 0 0 0
ss 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
ds 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
es 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
fs 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
gs 1000 0000000000010000 0000ffff 03 1 3 0 1 0 0 0
tr 0000 0000000000000000 0000ffff 0b 1 0 0 0 0 0 0
ldt 0000 0000000000000000 0000ffff 02 1 0 0 0 0 0 0
gdt 0000000000000000 0000ffff
idt 0000000000000000 0000ffff

APIC:
-----
efer: 0000000000000000 apic base: 00000000fee00900 nmi: enabled

Interrupt bitmap:
-----------------
0000000000000000 0000000000000000 0000000000000000 0000000000000000

Code:
-----
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <cf> eb 0d 90 90 90 90 90 90 90 90 90 90 90 90 90 f6 c4 0e 75 4b

Stack:
------
0x000008bc: 00 00 00 00 00 00 00 00
0x000008c4: 00 00 00 00 00 00 00 00
0x000008cc: 00 00 00 00 00 00 00 00
0x000008d4: 00 00 00 00 00 00 00 00

Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Enable SMP support

This patch enables SMP support:

[ 0.155072] Brought up 3 CPUs
[ 0.155074] Total of 3 processors activated (15158.58 BogoMIPS).

virtio-console was being loaded n

kvm tools: Enable SMP support

This patch enables SMP support:

[ 0.155072] Brought up 3 CPUs
[ 0.155074] Total of 3 processors activated (15158.58 BogoMIPS).

virtio-console was being loaded no matter the cmdline options
and it was causing some hangs (have to look into that).

I'll send this patch to a larger audience once someone
else can confirm it actually works/doesn't work.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Abolishment of uint*_t types

Clean uint*_t type from the code.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Use system wide msr-index.h instead of own definitions

To eliminate code duplication.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Use per-VCPU threads for execution

This patch makes the core hypervisor to use per-VCPU threads for execution.
NOTE: We only start one thread right now because we're unable to let the gue

kvm tools: Use per-VCPU threads for execution

This patch makes the core hypervisor to use per-VCPU threads for execution.
NOTE: We only start one thread right now because we're unable to let the guest
kernel know about more cores at the moment.

Cc: Asias He <asias.hejun@gmail.com>
Cc: Cyrill Gorcunov <gorcunov@gmail.com>
Tested-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Move CPU initialization to kvm_cpu__start()

Move CPUID setup and CPU reset code to kvm_cpu__start() so that CPU state is
setup in one place.

Cc: Asias He <asias.hejun@gmail.com>
Cc: Cyri

kvm tools: Move CPU initialization to kvm_cpu__start()

Move CPUID setup and CPU reset code to kvm_cpu__start() so that CPU state is
setup in one place.

Cc: Asias He <asias.hejun@gmail.com>
Cc: Cyrill Gorcunov <gorcunov@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Extract kvm_cpu__start() function

In preparation for threaded execution, separate kvm_cpu__start() function so it
can be reused for multiple threads.

Cc: Asias He <asias.hejun@gmail.com>

kvm tools: Extract kvm_cpu__start() function

In preparation for threaded execution, separate kvm_cpu__start() function so it
can be reused for multiple threads.

Cc: Asias He <asias.hejun@gmail.com>
Cc: Cyrill Gorcunov <gorcunov@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Introduce KVM VCPU data structure

In preparation for threaded execution model, this patch introduces a KVM VCPU
data structure 'struct kvm_cpu'.

Cc: Asias He <asias.hejun@gmail.com>
Cc:

kvm tools: Introduce KVM VCPU data structure

In preparation for threaded execution model, this patch introduces a KVM VCPU
data structure 'struct kvm_cpu'.

Cc: Asias He <asias.hejun@gmail.com>
Cc: Cyrill Gorcunov <gorcunov@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...