migration: drop MADVISE_DONT_NEED for incoming zero pages

The madvise for zeroed out pages was introduced when every transferred
zero page was memset to zero and thus allocated. Since commit
211ea74

migration: drop MADVISE_DONT_NEED for incoming zero pages

The madvise for zeroed out pages was introduced when every transferred
zero page was memset to zero and thus allocated. Since commit
211ea740 we check for zeroness of a target page before we memset
it to zero. Additionally we memmap target memory so it is essentially
zero initialized (except for e.g. option roms and bios which are loaded
into target memory although they shouldn't).

It was reported recently that this madvise causes a performance degradation
in some situations. As the madvise should only be called rarely and if it's called
it is likely on a busy page (it was non-zero and changed to zero during migration)
drop it completely.

Reported-By: Zhang Haoyu <haoyu.zhang@huawei.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Merge remote-tracking branch 'mst/tags/for_anthony' into staging

pc,pci,virtio fixes and cleanups

This includes pc and pci cleanups and enhancements,
and a virtio-net bugfix related to softmac prog

Merge remote-tracking branch 'mst/tags/for_anthony' into staging

pc,pci,virtio fixes and cleanups

This includes pc and pci cleanups and enhancements,
and a virtio-net bugfix related to softmac programming.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Sun 29 Sep 2013 01:51:16 AM CDT using RSA key ID D28D5469
# gpg: Can't check signature: public key not found

# By Michael S. Tsirkin (8) and others
# Via Michael S. Tsirkin
* mst/tags/for_anthony:
smbios: Factor out smbios_maybe_add_str()
smbios: Make multiple -smbios type= accumulate sanely
smbios: Improve diagnostics for conflicting entries
smbios: Convert to QemuOpts
smbios: Normalize smbios_entry_add()'s error handling to exit(1)
virtio-net: fix up HMP NIC info string on reset
pci: remove explicit check to 64K ioport size
piix4: disable io on reset
piix: use 64 bit window programmed by guest
q35: use 64 bit window programmed by guest
pci: add helper to retrieve the 64-bit range
range: add min/max operations on ranges
range: add Range to typedefs
q35: make pci window address/size match guest cfg

Message-id: 1380437951-21788-1-git-send-email-mst@redhat.com

show more ...

smbios: Make multiple -smbios type= accumulate sanely

Currently, -smbios type=T,NAME=VAL,... adds one field (T,NAME) with
value VAL to fw_cfg for each unique NAME. If NAME occurs multiple
times, th

smbios: Make multiple -smbios type= accumulate sanely

Currently, -smbios type=T,NAME=VAL,... adds one field (T,NAME) with
value VAL to fw_cfg for each unique NAME. If NAME occurs multiple
times, the last one's VAL is used (before the QemuOpts conversion, the
first one was used).

Multiple -smbios can add multiple fields with the same (T, NAME).
SeaBIOS reads all of them from fw_cfg, but uses only the first field
(T, NAME). The others are ignored.

"First one wins, subsequent ones get ignored silently" isn't nice. We
commonly let the last option win. Useful, because it lets you
-readconfig first, then selectively override with command line
options.

Clean up -smbios to work the common way. Accumulate the settings,
with later ones overwriting earlier ones. Put the result into fw_cfg
(no more useless duplicates).

Bonus cleanup: qemu_uuid_parse() no longer sets SMBIOS system uuid by
side effect.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

show more ...

smbios: Convert to QemuOpts

So that it can be set in config file for -readconfig.

This tightens parsing of -smbios, and makes it more consistent with
other options: unknown parameters are rejected,

smbios: Convert to QemuOpts

So that it can be set in config file for -readconfig.

This tightens parsing of -smbios, and makes it more consistent with
other options: unknown parameters are rejected, numbers with trailing
junk are rejected, when a parameter is given multiple times, last
rather than first wins, ...

MST: drop one chunk to fix build errors

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

show more ...

smbios: Normalize smbios_entry_add()'s error handling to exit(1)

It exits on all error conditions but one, where it returns -1.
Normalize, and return void.

Signed-off-by: Markus Armbruster <armbru@

smbios: Normalize smbios_entry_add()'s error handling to exit(1)

It exits on all error conditions but one, where it returns -1.
Normalize, and return void.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

show more ...

migration: ram_handle_compressed

ram_handle_compressed() should be aware of size > TARGET_PAGE_SIZE.
migration-rdma can call it with larger size.

Signed-off-by: Isaku Yamahata <yamahata@private.ema

migration: ram_handle_compressed

ram_handle_compressed() should be aware of size > TARGET_PAGE_SIZE.
migration-rdma can call it with larger size.

Signed-off-by: Isaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

arch_init: make is_zero_page accept size

Later is_zero_page will be used for non TARGET_PAGE_SIZE
range.
And rename it to is_zero_range as it isn't page size any more.

Signed-off-by: Isaku Yamahata

arch_init: make is_zero_page accept size

Later is_zero_page will be used for non TARGET_PAGE_SIZE
range.
And rename it to is_zero_range as it isn't page size any more.

Signed-off-by: Isaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

arch_init: right return for ram_save_iterate

qemu_file_rate_limit() never return negative value since the refactor
by Commit 1964a39, this patch gets rid of the negative check for it,
adjust bytes_t

arch_init: right return for ram_save_iterate

qemu_file_rate_limit() never return negative value since the refactor
by Commit 1964a39, this patch gets rid of the negative check for it,
adjust bytes_transferred and return value correspondingly in
ram_save_iterate().

Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

cpu: Replace qemu_for_each_cpu()

It was introduced to loop over CPUs from target-independent code, but
since commit 182735efaf956ccab50b6d74a4fed163e0f35660 target-independent
CPUState is used.

A l

cpu: Replace qemu_for_each_cpu()

It was introduced to loop over CPUs from target-independent code, but
since commit 182735efaf956ccab50b6d74a4fed163e0f35660 target-independent
CPUState is used.

A loop can be considered more efficient than function calls in a loop,
and CPU_FOREACH() hides implementation details just as well, so use that
instead.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>

show more ...

Merge remote-tracking branch 'mst/tags/for_anthony' into stable-1.5

pc,pci,virtio fixes and cleanups

This includes pc and pci cleanups, future-proofing of ROM files,
and a virtio bugfix correcting

Merge remote-tracking branch 'mst/tags/for_anthony' into stable-1.5

pc,pci,virtio fixes and cleanups

This includes pc and pci cleanups, future-proofing of ROM files,
and a virtio bugfix correcting splice on virtio console.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Mon 26 Aug 2013 01:34:20 AM CDT using RSA key ID D28D5469
# gpg: Can't check signature: public key not found

# By Markus Armbruster (5) and others
# Via Michael S. Tsirkin
* mst/tags/for_anthony:
virtio: virtqueue_get_avail_bytes: fix desc_pa when loop over the indirect descriptor table
pc_piix: Kill pc_init1() memory region args
pc: pc_compat_1_4() now can call pc_compat_1_5()
pc: Create pc_compat_*() functions
pc: Kill pc_init_pci_1_0()
pc: Don't explode QEMUMachineInitArgs into local variables needlessly
pc: Don't prematurely explode QEMUMachineInitArgs
ppc: Don't duplicate QEMUMachineInitArgs in PPCE500Params
ppc: Don't explode QEMUMachineInitArgs into local variables needlessly
sun4: Don't prematurely explode QEMUMachineInitArgs
q35: Add PCIe switch to example q35 configuration
loader: store FW CFG ROM files in RAM
arch_init: align MR size to target page size
pc: cleanup 1.4 compat support

Message-id: 1377535318-30491-1-git-send-email-mst@redhat.com

show more ...

Merge remote-tracking branch 'stefanha/block' into staging

# By Alex Bligh (32) and others
# Via Stefan Hajnoczi
* stefanha/block: (42 commits)
win32-aio: drop win32_aio_flush_cb()
aio-win32: re

Merge remote-tracking branch 'stefanha/block' into staging

# By Alex Bligh (32) and others
# Via Stefan Hajnoczi
* stefanha/block: (42 commits)
win32-aio: drop win32_aio_flush_cb()
aio-win32: replace incorrect AioHandler->opaque usage with ->e
aio / timers: remove dummy_io_handler_flush from tests/test-aio.c
aio / timers: Remove legacy interface
aio / timers: Switch entire codebase to the new timer API
aio / timers: Add scripts/switch-timer-api
aio / timers: Add test harness for AioContext timers
aio / timers: convert block_job_sleep_ns and co_sleep_ns to new API
aio / timers: Convert rtc_clock to be a QEMUClockType
aio / timers: Remove main_loop_timerlist
aio / timers: Rearrange timer.h & make legacy functions call non-legacy
aio / timers: Add qemu_clock_get_ms and qemu_clock_get_ms
aio / timers: Remove legacy qemu_clock_deadline & qemu_timerlist_deadline
aio / timers: Remove alarm timers
aio / timers: Add documentation and new format calls
aio / timers: Use all timerlists in icount warp calculations
aio / timers: Introduce new API timer_new and friends
aio / timers: On timer modification, qemu_notify or aio_notify
aio / timers: Convert mainloop to use timeout
aio / timers: Convert aio_poll to use AioContext timers' deadline
...

Message-id: 1377202298-22896-1-git-send-email-stefanha@redhat.com
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>

show more ...

aio / timers: Switch entire codebase to the new timer API

This is an autogenerated patch using scripts/switch-timer-api.

Switch the entire code base to using the new timer API.

Note this patch may

aio / timers: Switch entire codebase to the new timer API

This is an autogenerated patch using scripts/switch-timer-api.

Switch the entire code base to using the new timer API.

Note this patch may introduce some line length issues.

Signed-off-by: Alex Bligh <alex@alex.org.uk>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

arch_init: align MR size to target page size

Migration code assumes that each MR is a multiple of TARGET_PAGE_SIZE:
MR size is divided by TARGET_PAGE_SIZE, so if it isn't migration
never completes.

arch_init: align MR size to target page size

Migration code assumes that each MR is a multiple of TARGET_PAGE_SIZE:
MR size is divided by TARGET_PAGE_SIZE, so if it isn't migration
never completes.
But this isn't really required for regions set up with
memory_region_init_ram, since that calls qemu_ram_alloc
which aligns size up using TARGET_PAGE_ALIGN.

Align MR size up to full target page sizes, this way
migration completes even if we create a RAM MR
which is not a full target page size.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>

show more ...

Convert stderr message calling error_get_pretty() to error_report()

Convert stderr messages calling error_get_pretty()
to error_report().

Timestamp is prepended by -msg timstamp option with it.

Pe

Convert stderr message calling error_get_pretty() to error_report()

Convert stderr messages calling error_get_pretty()
to error_report().

Timestamp is prepended by -msg timstamp option with it.

Per Markus's comment below, A conversion from fprintf() to
error_report() is always an improvement, regardless of
error_get_pretty().

http://marc.info/?l=qemu-devel&m=137513283408601&w=2

But, it is not reasonable to convert them at one time
because fprintf() is used everwhere in qemu.

So, it should be done step by step with avoiding regression.

Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...

rdma: send pc.ram

This takes advantages of the previous patches:

1. use the new QEMUFileOps hook 'save_page'

2. call out to the right accessor methods to invoke
the iteration hooks defined in Q

rdma: send pc.ram

This takes advantages of the previous patches:

1. use the new QEMUFileOps hook 'save_page'

2. call out to the right accessor methods to invoke
the iteration hooks defined in QEMUFileOps

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

rdma: introduce ram_handle_compressed()

This gives RDMA shared access to madvise() on the destination side
when an entire chunk is found to be zero.

Reviewed-by: Juan Quintela <quintela@redhat.com>

rdma: introduce ram_handle_compressed()

This gives RDMA shared access to madvise() on the destination side
when an entire chunk is found to be zero.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Force auto-convegence of live migration

If a user chooses to turn on the auto-converge migration capability
these changes detect the lack of convergence and throttle down the
guest. i.e. force the V

Force auto-convegence of live migration

If a user chooses to turn on the auto-converge migration capability
these changes detect the lack of convergence and throttle down the
guest. i.e. force the VCPUs out of the guest for some duration
and let the migration thread catchup and help converge.

Verified the convergence using the following :
- Java Warehouse workload running on a 20VCPU/256G guest(~80% busy)
- OLTP like workload running on a 80VCPU/512G guest (~80% busy)

Sample results with Java warehouse workload : (migrate speed set to 20Gb and
migrate downtime set to 4seconds).

(qemu) info migrate
capabilities: xbzrle: off auto-converge: off <----
Migration status: active
total time: 1487503 milliseconds
expected downtime: 519 milliseconds
transferred ram: 383749347 kbytes
remaining ram: 2753372 kbytes
total ram: 268444224 kbytes
duplicate: 65461532 pages
skipped: 64901568 pages
normal: 95750218 pages
normal bytes: 383000872 kbytes
dirty pages rate: 67551 pages

---

(qemu) info migrate
capabilities: xbzrle: off auto-converge: on <----
Migration status: completed
total time: 241161 milliseconds
downtime: 6373 milliseconds
transferred ram: 28235307 kbytes
remaining ram: 0 kbytes
total ram: 268444224 kbytes
duplicate: 64946416 pages
skipped: 64903523 pages
normal: 7044971 pages
normal bytes: 28179884 kbytes

Signed-off-by: Chegu Vinod <chegu_vinod@hp.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Graphics: Switch to 800x600x32 as default mode

We have stayed at 800x600x15 as default graphics mode for the last 9 years.
If there ever was a reason to be there, surely nobody remembers it.

Howeve

Graphics: Switch to 800x600x32 as default mode

We have stayed at 800x600x15 as default graphics mode for the last 9 years.
If there ever was a reason to be there, surely nobody remembers it.

However, recently non-Linux PPC guests started to show bad effects on 15 bit
color mode. They do work just fine with 32 bits however.

So let's switch to 32 bit color as the default graphic mode.

Reported-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Alexander Graf <agraf@suse.de>

show more ...

arch_init: Fix format string by using RAM_ADDR_FMT

length is a ram_addr_t, so RAM_ADDR_FMT must be used instead of %ld.
This fixes a recently introduced regression for w64 builds.

Using RAM_ADDR_FM

arch_init: Fix format string by using RAM_ADDR_FMT

length is a ram_addr_t, so RAM_ADDR_FMT must be used instead of %ld.
This fixes a recently introduced regression for w64 builds.

Using RAM_ADDR_FMT also changes decimal output to sedecimal.
This is good here because length and block->length should both
use the same base in the error message.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Message-id: 1372359606-2759-1-git-send-email-sw@weilnetz.de
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

show more ...

rdma: introduce qemu_update_position()

RDMA writes happen asynchronously, and thus the performance accounting
also needs to be able to occur asynchronously. This allows anybody
to call into savevm.c

rdma: introduce qemu_update_position()

RDMA writes happen asynchronously, and thus the performance accounting
also needs to be able to occur asynchronously. This allows anybody
to call into savevm.c to update both f->pos as well as into arch_init.c
to update the acct_info structure with up-to-date values when
the RDMA transfer actually completes.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Tested-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

migration: do not overwrite zero pages

on incoming migration do not memset pages to zero if they already read as zero.
this will allocate a new zero page and consume memory unnecessarily. even
if we

migration: do not overwrite zero pages

on incoming migration do not memset pages to zero if they already read as zero.
this will allocate a new zero page and consume memory unnecessarily. even
if we madvise a MADV_DONTNEED later this will only deallocate the memory
asynchronously.

Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Revert "migration: do not sent zero pages in bulk stage"

Not sending zero pages breaks migration if a page is zero
at the source but not at the destination. This can e.g. happen
if different BIOS ve

Revert "migration: do not sent zero pages in bulk stage"

Not sending zero pages breaks migration if a page is zero
at the source but not at the destination. This can e.g. happen
if different BIOS versions are used at source and destination.
It has also been reported that migration on pseries is completely
broken with this patch.

This effectively reverts commit f1c72795af573b24a7da5eb52375c9aba8a37972.

Conflicts:

arch_init.c

Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

arch_init/ram_load: add error message for block length mismatch

Makes it easier to debug situations where the source and target have
different ram blocks in a device and migration fails due to that,

arch_init/ram_load: add error message for block length mismatch

Makes it easier to debug situations where the source and target have
different ram blocks in a device and migration fails due to that, for
instance a BAR size change on a PCI device.

Signed-off-by: Alon Levy <alevy@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Merge remote-tracking branch 'pmaydell/configury.next' into staging

# By Paolo Bonzini (4) and others
# Via Peter Maydell
* pmaydell/configury.next:
ppc: Remove CONFIG_FDT conditionals
microblaz

Merge remote-tracking branch 'pmaydell/configury.next' into staging

# By Paolo Bonzini (4) and others
# Via Peter Maydell
* pmaydell/configury.next:
ppc: Remove CONFIG_FDT conditionals
microblaze: Remove CONFIG_FDT conditionals
arm: Remove CONFIG_FDT conditionals
configure: Require libfdt for arm, ppc, microblaze softmmu targets
configure: dtc: Probe for libfdt_env.h
build: drop TARGET_TYPE
main: use TARGET_ARCH only for the target-specific #define
build: do not use TARGET_ARCH
build: rename TARGET_ARCH2 to TARGET_NAME
Add a stp file for usage from build directory

Message-id: 1371221594-11556-1-git-send-email-peter.maydell@linaro.org
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

show more ...

build: drop TARGET_TYPE

Just use the TARGET_NAME free string.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1370349928-20419-6-git-send-

build: drop TARGET_TYPE

Just use the TARGET_NAME free string.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1370349928-20419-6-git-send-email-pbonzini@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...