arch_init: Add and free data struct for decompression

Define the data structure and variables used to do multiple thread
decompression, and add the code to initialize and free them.

Signed-off-by:

arch_init: Add and free data struct for decompression

Define the data structure and variables used to do multiple thread
decompression, and add the code to initialize and free them.

Signed-off-by: Liang Li <liang.z.li@intel.com>
Signed-off-by: Yang Zhang <yang.z.zhang@intel.com>
Reviewed-by: Dr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

arch_init: Alloc and free data struct for compression

Define the data structure and variables used to do multiple thread
compression, and add the code to initialize and free them.

Signed-off-by: Li

arch_init: Alloc and free data struct for compression

Define the data structure and variables used to do multiple thread
compression, and add the code to initialize and free them.

Signed-off-by: Liang Li <liang.z.li@intel.com>
Signed-off-by: Yang Zhang <yang.z.zhang@intel.com>
Reviewed-by: Dr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

migration: Add the framework of multi-thread decompression

Add the code to create and destroy the multiple threads those will be
used to do data decompression. Left some functions empty just to keep

migration: Add the framework of multi-thread decompression

Add the code to create and destroy the multiple threads those will be
used to do data decompression. Left some functions empty just to keep
clearness, and the code will be added later.

Signed-off-by: Liang Li <liang.z.li@intel.com>
Signed-off-by: Yang Zhang <yang.z.zhang@intel.com>
Reviewed-by: Dr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

migration: Add the framework of multi-thread compression

Add the code to create and destroy the multiple threads those will
be used to do data compression. Left some functions empty to keep
clearnes

migration: Add the framework of multi-thread compression

Add the code to create and destroy the multiple threads those will
be used to do data compression. Left some functions empty to keep
clearness, and the code will be added later.

Signed-off-by: Liang Li <liang.z.li@intel.com>
Signed-off-by: Yang Zhang <yang.z.zhang@intel.com>
Reviewed-by: Dr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

migration: remove last_sent_block from save_page_header

Compression code (still not on tree) want to call this funtion from
outside the migration thread, so we can't write to last_sent_block.

Inst

migration: remove last_sent_block from save_page_header

Compression code (still not on tree) want to call this funtion from
outside the migration thread, so we can't write to last_sent_block.

Instead of reverting full patch:

[PULL 07/11] save_block_hdr: we can recalculate

Just revert the parts that touch last_sent_block.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

show more ...

arch_init: Count the total number of pages by using helper function

There is already a helper function ram_bytes_total(), we can use it to
help counting the total number of pages used by ram blocks.

arch_init: Count the total number of pages by using helper function

There is already a helper function ram_bytes_total(), we can use it to
help counting the total number of pages used by ram blocks.

Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

rename save_block_hdr to save_page_header

It has always been a page header, not a block header. Once there, the
flag argument was only passed to make a bit or with it, just do the or
on the caller.

rename save_block_hdr to save_page_header

It has always been a page header, not a block header. Once there, the
flag argument was only passed to make a bit or with it, just do the or
on the caller.

Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

save_block_hdr: we can recalculate the cont parameter here

No need to pass it through all the callers. Once there, update
last_sent_block here.

Signed-off-by: Juan Quintela <quintela@redhat.com>

save_xbzrle_page: change calling convention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quintela@redha

save_xbzrle_page: change calling convention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

ram_save_page: change calling covention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quintela@redhat.co

ram_save_page: change calling covention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

ram_find_and_save_block: change calling convention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quinte

ram_find_and_save_block: change calling convention

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.

Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

ram: make all save_page functions take a uint64_t parameter

It used to be an int, but then we can't pass directly the
bytes_transferred parameter, that would happen later in the series.

Signed-off-

ram: make all save_page functions take a uint64_t parameter

It used to be an int, but then we can't pass directly the
bytes_transferred parameter, that would happen later in the series.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2015-02-18' into staging

Clean up around error_get_pretty(), qerror_report_err()

# gpg: Signature made Wed Feb 18 10:10:07 2015 GMT usin

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2015-02-18' into staging

Clean up around error_get_pretty(), qerror_report_err()

# gpg: Signature made Wed Feb 18 10:10:07 2015 GMT using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-error-2015-02-18:
qemu-char: Avoid qerror_report_err() outside QMP command handlers
qemu-img: Avoid qerror_report_err() outside QMP command handlers
vl: Avoid qerror_report_err() outside QMP command handlers
tpm: Avoid qerror_report_err() outside QMP command handlers
numa: Avoid qerror_report_err() outside QMP command handlers
net: Avoid qerror_report_err() outside QMP command handlers
monitor: Avoid qerror_report_err() outside QMP command handlers
monitor: Clean up around monitor_handle_fd_param()
error: Use error_report_err() where appropriate
error: New convenience function error_report_err()
vhost-scsi: Improve error reporting for invalid vhostfd

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

error: Use error_report_err() where appropriate

Coccinelle semantic patch:

@@
expression E;
@@
- error_report("%s", error_get_pretty(E));
- error_free(E);
+ error_r

error: Use error_report_err() where appropriate

Coccinelle semantic patch:

@@
expression E;
@@
- error_report("%s", error_get_pretty(E));
- error_free(E);
+ error_report_err(E);
@@
expression E, S;
@@
- error_report("%s", error_get_pretty(E));
+ error_report_err(E);
(
exit(S);
|
abort();
)

Trivial manual touch-ups in block/sheepdog.c.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

Convert ram_list to RCU

Allow "unlocked" reads of the ram_list by using an RCU-enabled QLIST.

The ramlist mutex is kept. call_rcu callbacks are run with the iothread
lock taken, but that may chang

Convert ram_list to RCU

Allow "unlocked" reads of the ram_list by using an RCU-enabled QLIST.

The ramlist mutex is kept. call_rcu callbacks are run with the iothread
lock taken, but that may change in the future. Writers still take the
ramlist mutex, but they no longer need to assume that the iothread lock
is taken.

Readers of the list, instead, no longer require either the iothread
or ramlist mutex, but they need to use rcu_read_lock() and
rcu_read_unlock().

One place in arch_init.c was downgrading from write side to read side
like this:

qemu_mutex_lock_iothread()
qemu_mutex_lock_ramlist()
...
qemu_mutex_unlock_iothread()
...
qemu_mutex_unlock_ramlist()

and the equivalent idiom is:

qemu_mutex_lock_ramlist()
rcu_read_lock()
...
qemu_mutex_unlock_ramlist()
...
rcu_read_unlock()

Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

exec: convert ram_list to QLIST

QLIST has RCU-friendly primitives, so switch to it.

Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzin

exec: convert ram_list to QLIST

QLIST has RCU-friendly primitives, so switch to it.

Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

cosmetic changes preparing for the following patches

Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

xbzrle: optimize XBZRLE to decrease the cache misses

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ra

xbzrle: optimize XBZRLE to decrease the cache misses

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ran in
vm:(migrate bandwidth:1GE and xbzrle cache size 8MB)

the test program:

include <stdlib.h>
include <stdio.h>
int main()
{
char *buf = (char *) calloc(4096, 4096);
while (1) {
int i;
for (i = 0; i < 4096 * 4; i++) {
buf[i * 4096 / 4]++;
}
printf(".");
}
}

before this patch:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":1020,"xbzrle-cache":{"bytes":1108284,
"cache-size":8388608,"cache-miss-rate":0.987013,"pages":18297,"overflow":8,
"cache-miss":1228737},"status":"active","setup-time":10,"total-time":52398,
"ram":{"total":12466991104,"remaining":1695744,"mbps":935.559472,
"transferred":5780760580,"dirty-sync-counter":271,"duplicate":2878530,
"dirty-pages-rate":29130,"skipped":0,"normal-bytes":5748592640,
"normal":1403465}},"id":"libvirt-706"}

18k pages sent compressed in 52 seconds.
cache-miss-rate is 98.7%, totally miss.

after optimizing:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":2054,"xbzrle-cache":{"bytes":5066763,
"cache-size":8388608,"cache-miss-rate":0.485924,"pages":194823,"overflow":0,
"cache-miss":210653},"status":"active","setup-time":11,"total-time":18729,
"ram":{"total":12466991104,"remaining":3895296,"mbps":937.663549,
"transferred":1615042219,"dirty-sync-counter":98,"duplicate":2869840,
"dirty-pages-rate":58781,"skipped":0,"normal-bytes":1588404224,
"normal":387794}},"id":"libvirt-266"}

194k pages sent compressed in 18 seconds.
The value of cache-miss-rate decrease to 48.59%.

Signed-off-by: ChenLiang <chenliang88@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>

show more ...

arch_init: support resizing on incoming migration

If block used_length does not match, try to resize it.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redh

arch_init: support resizing on incoming migration

If block used_length does not match, try to resize it.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

exec: split length -> used_length/max_length

This patch allows us to distinguish between two
length values for each block:
max_length - length of memory block that was allocated
used_length

exec: split length -> used_length/max_length

This patch allows us to distinguish between two
length values for each block:
max_length - length of memory block that was allocated
used_length - length of block used by QEMU/guest

Currently, we set used_length - max_length, unconditionally.
Follow-up patches allow used_length <= max_length.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

migration: static variables will not be reset at second migration

The static variables in migration_bitmap_sync will not be reset in
the case of a second attempted migration.

Signed-off-by: ChenLia

migration: static variables will not be reset at second migration

The static variables in migration_bitmap_sync will not be reset in
the case of a second attempted migration.

Signed-off-by: ChenLiang <chenliang88@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>

show more ...

migration: fix parameter validation on ram load

During migration, the values read from migration stream during ram load
are not validated. Especially offset in host_from_stream_offset() and
also the

migration: fix parameter validation on ram load

During migration, the values read from migration stream during ram load
are not validated. Especially offset in host_from_stream_offset() and
also the length of the writes in the callers of said function.

To fix this, we need to make sure that the [offset, offset + length]
range fits into one of the allocated memory regions.

Validating addr < len should be sufficient since data seems to always be
managed in TARGET_PAGE_SIZE chunks.

Fixes: CVE-2014-7840

Note: follow-up patches add extra checks on each block->host access.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>

show more ...

migration: catch unknown flag combinations in ram_load

this patch extends commit db80fac by not only checking
for unknown flags, but also filtering out unknown flag
combinations.

Suggested-by: Eric

migration: catch unknown flag combinations in ram_load

this patch extends commit db80fac by not only checking
for unknown flags, but also filtering out unknown flag
combinations.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

accel: Remove tcg_available() function

As the function always return 1, it is not needed anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.co

accel: Remove tcg_available() function

As the function always return 1, it is not needed anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

target-tricore: Add target stubs and qom-cpu

Add TriCore target stubs, and QOM cpu, and Maintainer

Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-id: 1409572800-4116-2-g

target-tricore: Add target stubs and qom-cpu

Add TriCore target stubs, and QOM cpu, and Maintainer

Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-id: 1409572800-4116-2-git-send-email-kbastian@mail.uni-paderborn.de
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...