1b35ebdf0SVladimir Sementsov-Ogievskiy /* 2b35ebdf0SVladimir Sementsov-Ogievskiy * Block dirty bitmap postcopy migration 3b35ebdf0SVladimir Sementsov-Ogievskiy * 4b35ebdf0SVladimir Sementsov-Ogievskiy * Copyright IBM, Corp. 2009 5b35ebdf0SVladimir Sementsov-Ogievskiy * Copyright (c) 2016-2017 Virtuozzo International GmbH. All rights reserved. 6b35ebdf0SVladimir Sementsov-Ogievskiy * 7b35ebdf0SVladimir Sementsov-Ogievskiy * Authors: 8b35ebdf0SVladimir Sementsov-Ogievskiy * Liran Schour <lirans@il.ibm.com> 9b35ebdf0SVladimir Sementsov-Ogievskiy * Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> 10b35ebdf0SVladimir Sementsov-Ogievskiy * 11b35ebdf0SVladimir Sementsov-Ogievskiy * This work is licensed under the terms of the GNU GPL, version 2. See 12b35ebdf0SVladimir Sementsov-Ogievskiy * the COPYING file in the top-level directory. 13b35ebdf0SVladimir Sementsov-Ogievskiy * This file is derived from migration/block.c, so it's author and IBM copyright 14b35ebdf0SVladimir Sementsov-Ogievskiy * are here, although content is quite different. 15b35ebdf0SVladimir Sementsov-Ogievskiy * 16b35ebdf0SVladimir Sementsov-Ogievskiy * Contributions after 2012-01-13 are licensed under the terms of the 17b35ebdf0SVladimir Sementsov-Ogievskiy * GNU GPL, version 2 or (at your option) any later version. 18b35ebdf0SVladimir Sementsov-Ogievskiy * 19b35ebdf0SVladimir Sementsov-Ogievskiy * *** 20b35ebdf0SVladimir Sementsov-Ogievskiy * 21b35ebdf0SVladimir Sementsov-Ogievskiy * Here postcopy migration of dirty bitmaps is realized. Only QMP-addressable 22b35ebdf0SVladimir Sementsov-Ogievskiy * bitmaps are migrated. 23b35ebdf0SVladimir Sementsov-Ogievskiy * 24b35ebdf0SVladimir Sementsov-Ogievskiy * Bitmap migration implies creating bitmap with the same name and granularity 25b35ebdf0SVladimir Sementsov-Ogievskiy * in destination QEMU. If the bitmap with the same name (for the same node) 26b35ebdf0SVladimir Sementsov-Ogievskiy * already exists on destination an error will be generated. 27b35ebdf0SVladimir Sementsov-Ogievskiy * 28b35ebdf0SVladimir Sementsov-Ogievskiy * format of migration: 29b35ebdf0SVladimir Sementsov-Ogievskiy * 30b35ebdf0SVladimir Sementsov-Ogievskiy * # Header (shared for different chunk types) 31b35ebdf0SVladimir Sementsov-Ogievskiy * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags) 32b35ebdf0SVladimir Sementsov-Ogievskiy * [ 1 byte: node name size ] \ flags & DEVICE_NAME 33b35ebdf0SVladimir Sementsov-Ogievskiy * [ n bytes: node name ] / 34b35ebdf0SVladimir Sementsov-Ogievskiy * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME 35b35ebdf0SVladimir Sementsov-Ogievskiy * [ n bytes: bitmap name ] / 36b35ebdf0SVladimir Sementsov-Ogievskiy * 37b35ebdf0SVladimir Sementsov-Ogievskiy * # Start of bitmap migration (flags & START) 38b35ebdf0SVladimir Sementsov-Ogievskiy * header 39b35ebdf0SVladimir Sementsov-Ogievskiy * be64: granularity 40b35ebdf0SVladimir Sementsov-Ogievskiy * 1 byte: bitmap flags (corresponds to BdrvDirtyBitmap) 41b35ebdf0SVladimir Sementsov-Ogievskiy * bit 0 - bitmap is enabled 42b35ebdf0SVladimir Sementsov-Ogievskiy * bit 1 - bitmap is persistent 43b35ebdf0SVladimir Sementsov-Ogievskiy * bit 2 - bitmap is autoloading 44b35ebdf0SVladimir Sementsov-Ogievskiy * bits 3-7 - reserved, must be zero 45b35ebdf0SVladimir Sementsov-Ogievskiy * 46b35ebdf0SVladimir Sementsov-Ogievskiy * # Complete of bitmap migration (flags & COMPLETE) 47b35ebdf0SVladimir Sementsov-Ogievskiy * header 48b35ebdf0SVladimir Sementsov-Ogievskiy * 49b35ebdf0SVladimir Sementsov-Ogievskiy * # Data chunk of bitmap migration 50b35ebdf0SVladimir Sementsov-Ogievskiy * header 51b35ebdf0SVladimir Sementsov-Ogievskiy * be64: start sector 52b35ebdf0SVladimir Sementsov-Ogievskiy * be32: number of sectors 53b35ebdf0SVladimir Sementsov-Ogievskiy * [ be64: buffer size ] \ ! (flags & ZEROES) 54b35ebdf0SVladimir Sementsov-Ogievskiy * [ n bytes: buffer ] / 55b35ebdf0SVladimir Sementsov-Ogievskiy * 56b35ebdf0SVladimir Sementsov-Ogievskiy * The last chunk in stream should contain flags & EOS. The chunk may skip 57b35ebdf0SVladimir Sementsov-Ogievskiy * device and/or bitmap names, assuming them to be the same with the previous 58b35ebdf0SVladimir Sementsov-Ogievskiy * chunk. 59b35ebdf0SVladimir Sementsov-Ogievskiy */ 60b35ebdf0SVladimir Sementsov-Ogievskiy 61b35ebdf0SVladimir Sementsov-Ogievskiy #include "qemu/osdep.h" 62b35ebdf0SVladimir Sementsov-Ogievskiy #include "block/block.h" 63b35ebdf0SVladimir Sementsov-Ogievskiy #include "block/block_int.h" 64b35ebdf0SVladimir Sementsov-Ogievskiy #include "sysemu/block-backend.h" 6554d31236SMarkus Armbruster #include "sysemu/runstate.h" 66b35ebdf0SVladimir Sementsov-Ogievskiy #include "qemu/main-loop.h" 67b35ebdf0SVladimir Sementsov-Ogievskiy #include "qemu/error-report.h" 68b35ebdf0SVladimir Sementsov-Ogievskiy #include "migration/misc.h" 69b35ebdf0SVladimir Sementsov-Ogievskiy #include "migration/migration.h" 7053d37d36SMichael S. Tsirkin #include "qemu-file.h" 71b35ebdf0SVladimir Sementsov-Ogievskiy #include "migration/vmstate.h" 72b35ebdf0SVladimir Sementsov-Ogievskiy #include "migration/register.h" 73b35ebdf0SVladimir Sementsov-Ogievskiy #include "qemu/hbitmap.h" 74b35ebdf0SVladimir Sementsov-Ogievskiy #include "qemu/cutils.h" 75b35ebdf0SVladimir Sementsov-Ogievskiy #include "qapi/error.h" 76b35ebdf0SVladimir Sementsov-Ogievskiy #include "trace.h" 77b35ebdf0SVladimir Sementsov-Ogievskiy 78b35ebdf0SVladimir Sementsov-Ogievskiy #define CHUNK_SIZE (1 << 10) 79b35ebdf0SVladimir Sementsov-Ogievskiy 80b35ebdf0SVladimir Sementsov-Ogievskiy /* Flags occupy one, two or four bytes (Big Endian). The size is determined as 81b35ebdf0SVladimir Sementsov-Ogievskiy * follows: 82b35ebdf0SVladimir Sementsov-Ogievskiy * in first (most significant) byte bit 8 is clear --> one byte 83b35ebdf0SVladimir Sementsov-Ogievskiy * in first byte bit 8 is set --> two or four bytes, depending on second 84b35ebdf0SVladimir Sementsov-Ogievskiy * byte: 85b35ebdf0SVladimir Sementsov-Ogievskiy * | in second byte bit 8 is clear --> two bytes 86b35ebdf0SVladimir Sementsov-Ogievskiy * | in second byte bit 8 is set --> four bytes 87b35ebdf0SVladimir Sementsov-Ogievskiy */ 88b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_EOS 0x01 89b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_ZEROES 0x02 90b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04 91b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08 92b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_START 0x10 93b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20 94b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_FLAG_BITS 0x40 95b35ebdf0SVladimir Sementsov-Ogievskiy 96b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_EXTRA_FLAGS 0x80 97b35ebdf0SVladimir Sementsov-Ogievskiy 98b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_START_FLAG_ENABLED 0x01 99b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT 0x02 100b35ebdf0SVladimir Sementsov-Ogievskiy /* 0x04 was "AUTOLOAD" flags on elder versions, no it is ignored */ 101b35ebdf0SVladimir Sementsov-Ogievskiy #define DIRTY_BITMAP_MIG_START_FLAG_RESERVED_MASK 0xf8 102b35ebdf0SVladimir Sementsov-Ogievskiy 103b35ebdf0SVladimir Sementsov-Ogievskiy typedef struct DirtyBitmapMigBitmapState { 104b35ebdf0SVladimir Sementsov-Ogievskiy /* Written during setup phase. */ 105b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *bs; 106b35ebdf0SVladimir Sementsov-Ogievskiy const char *node_name; 107b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *bitmap; 108b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t total_sectors; 109b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t sectors_per_chunk; 110b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry; 111b35ebdf0SVladimir Sementsov-Ogievskiy uint8_t flags; 112b35ebdf0SVladimir Sementsov-Ogievskiy 113b35ebdf0SVladimir Sementsov-Ogievskiy /* For bulk phase. */ 114b35ebdf0SVladimir Sementsov-Ogievskiy bool bulk_completed; 115b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t cur_sector; 116b35ebdf0SVladimir Sementsov-Ogievskiy } DirtyBitmapMigBitmapState; 117b35ebdf0SVladimir Sementsov-Ogievskiy 118b35ebdf0SVladimir Sementsov-Ogievskiy typedef struct DirtyBitmapMigState { 119b58deb34SPaolo Bonzini QSIMPLEQ_HEAD(, DirtyBitmapMigBitmapState) dbms_list; 120b35ebdf0SVladimir Sementsov-Ogievskiy 121b35ebdf0SVladimir Sementsov-Ogievskiy bool bulk_completed; 122b35ebdf0SVladimir Sementsov-Ogievskiy bool no_bitmaps; 123b35ebdf0SVladimir Sementsov-Ogievskiy 124b35ebdf0SVladimir Sementsov-Ogievskiy /* for send_bitmap_bits() */ 125b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *prev_bs; 126b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *prev_bitmap; 127b35ebdf0SVladimir Sementsov-Ogievskiy } DirtyBitmapMigState; 128b35ebdf0SVladimir Sementsov-Ogievskiy 129b35ebdf0SVladimir Sementsov-Ogievskiy typedef struct DirtyBitmapLoadState { 130b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t flags; 131b35ebdf0SVladimir Sementsov-Ogievskiy char node_name[256]; 132b35ebdf0SVladimir Sementsov-Ogievskiy char bitmap_name[256]; 133b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *bs; 134b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *bitmap; 135b35ebdf0SVladimir Sementsov-Ogievskiy } DirtyBitmapLoadState; 136b35ebdf0SVladimir Sementsov-Ogievskiy 137b35ebdf0SVladimir Sementsov-Ogievskiy static DirtyBitmapMigState dirty_bitmap_mig_state; 138b35ebdf0SVladimir Sementsov-Ogievskiy 139b35ebdf0SVladimir Sementsov-Ogievskiy typedef struct DirtyBitmapLoadBitmapState { 140b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *bs; 141b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *bitmap; 142b35ebdf0SVladimir Sementsov-Ogievskiy bool migrated; 143b35ebdf0SVladimir Sementsov-Ogievskiy } DirtyBitmapLoadBitmapState; 144b35ebdf0SVladimir Sementsov-Ogievskiy static GSList *enabled_bitmaps; 145b35ebdf0SVladimir Sementsov-Ogievskiy QemuMutex finish_lock; 146b35ebdf0SVladimir Sementsov-Ogievskiy 147b35ebdf0SVladimir Sementsov-Ogievskiy void init_dirty_bitmap_incoming_migration(void) 148b35ebdf0SVladimir Sementsov-Ogievskiy { 149b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_init(&finish_lock); 150b35ebdf0SVladimir Sementsov-Ogievskiy } 151b35ebdf0SVladimir Sementsov-Ogievskiy 152b35ebdf0SVladimir Sementsov-Ogievskiy static uint32_t qemu_get_bitmap_flags(QEMUFile *f) 153b35ebdf0SVladimir Sementsov-Ogievskiy { 154b35ebdf0SVladimir Sementsov-Ogievskiy uint8_t flags = qemu_get_byte(f); 155b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) { 156b35ebdf0SVladimir Sementsov-Ogievskiy flags = flags << 8 | qemu_get_byte(f); 157b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) { 158b35ebdf0SVladimir Sementsov-Ogievskiy flags = flags << 16 | qemu_get_be16(f); 159b35ebdf0SVladimir Sementsov-Ogievskiy } 160b35ebdf0SVladimir Sementsov-Ogievskiy } 161b35ebdf0SVladimir Sementsov-Ogievskiy 162b35ebdf0SVladimir Sementsov-Ogievskiy return flags; 163b35ebdf0SVladimir Sementsov-Ogievskiy } 164b35ebdf0SVladimir Sementsov-Ogievskiy 165b35ebdf0SVladimir Sementsov-Ogievskiy static void qemu_put_bitmap_flags(QEMUFile *f, uint32_t flags) 166b35ebdf0SVladimir Sementsov-Ogievskiy { 167b35ebdf0SVladimir Sementsov-Ogievskiy /* The code currently do not send flags more than one byte */ 168b35ebdf0SVladimir Sementsov-Ogievskiy assert(!(flags & (0xffffff00 | DIRTY_BITMAP_MIG_EXTRA_FLAGS))); 169b35ebdf0SVladimir Sementsov-Ogievskiy 170b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_byte(f, flags); 171b35ebdf0SVladimir Sementsov-Ogievskiy } 172b35ebdf0SVladimir Sementsov-Ogievskiy 173b35ebdf0SVladimir Sementsov-Ogievskiy static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbms, 174b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t additional_flags) 175b35ebdf0SVladimir Sementsov-Ogievskiy { 176b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *bs = dbms->bs; 177b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *bitmap = dbms->bitmap; 178b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t flags = additional_flags; 179b35ebdf0SVladimir Sementsov-Ogievskiy trace_send_bitmap_header_enter(); 180b35ebdf0SVladimir Sementsov-Ogievskiy 181b35ebdf0SVladimir Sementsov-Ogievskiy if (bs != dirty_bitmap_mig_state.prev_bs) { 182b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.prev_bs = bs; 183b35ebdf0SVladimir Sementsov-Ogievskiy flags |= DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME; 184b35ebdf0SVladimir Sementsov-Ogievskiy } 185b35ebdf0SVladimir Sementsov-Ogievskiy 186b35ebdf0SVladimir Sementsov-Ogievskiy if (bitmap != dirty_bitmap_mig_state.prev_bitmap) { 187b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.prev_bitmap = bitmap; 188b35ebdf0SVladimir Sementsov-Ogievskiy flags |= DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME; 189b35ebdf0SVladimir Sementsov-Ogievskiy } 190b35ebdf0SVladimir Sementsov-Ogievskiy 191b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_bitmap_flags(f, flags); 192b35ebdf0SVladimir Sementsov-Ogievskiy 193b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { 194b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_counted_string(f, dbms->node_name); 195b35ebdf0SVladimir Sementsov-Ogievskiy } 196b35ebdf0SVladimir Sementsov-Ogievskiy 197b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { 198b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_counted_string(f, bdrv_dirty_bitmap_name(bitmap)); 199b35ebdf0SVladimir Sementsov-Ogievskiy } 200b35ebdf0SVladimir Sementsov-Ogievskiy } 201b35ebdf0SVladimir Sementsov-Ogievskiy 202b35ebdf0SVladimir Sementsov-Ogievskiy static void send_bitmap_start(QEMUFile *f, DirtyBitmapMigBitmapState *dbms) 203b35ebdf0SVladimir Sementsov-Ogievskiy { 204b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START); 205b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap)); 206b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_byte(f, dbms->flags); 207b35ebdf0SVladimir Sementsov-Ogievskiy } 208b35ebdf0SVladimir Sementsov-Ogievskiy 209b35ebdf0SVladimir Sementsov-Ogievskiy static void send_bitmap_complete(QEMUFile *f, DirtyBitmapMigBitmapState *dbms) 210b35ebdf0SVladimir Sementsov-Ogievskiy { 211b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE); 212b35ebdf0SVladimir Sementsov-Ogievskiy } 213b35ebdf0SVladimir Sementsov-Ogievskiy 214b35ebdf0SVladimir Sementsov-Ogievskiy static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMigBitmapState *dbms, 215b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t start_sector, uint32_t nr_sectors) 216b35ebdf0SVladimir Sementsov-Ogievskiy { 217b35ebdf0SVladimir Sementsov-Ogievskiy /* align for buffer_is_zero() */ 218b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t align = 4 * sizeof(long); 219b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t unaligned_size = 220b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_serialization_size( 221b35ebdf0SVladimir Sementsov-Ogievskiy dbms->bitmap, start_sector << BDRV_SECTOR_BITS, 222b35ebdf0SVladimir Sementsov-Ogievskiy (uint64_t)nr_sectors << BDRV_SECTOR_BITS); 223b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t buf_size = QEMU_ALIGN_UP(unaligned_size, align); 224b35ebdf0SVladimir Sementsov-Ogievskiy uint8_t *buf = g_malloc0(buf_size); 225b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t flags = DIRTY_BITMAP_MIG_FLAG_BITS; 226b35ebdf0SVladimir Sementsov-Ogievskiy 227b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_serialize_part( 228b35ebdf0SVladimir Sementsov-Ogievskiy dbms->bitmap, buf, start_sector << BDRV_SECTOR_BITS, 229b35ebdf0SVladimir Sementsov-Ogievskiy (uint64_t)nr_sectors << BDRV_SECTOR_BITS); 230b35ebdf0SVladimir Sementsov-Ogievskiy 231b35ebdf0SVladimir Sementsov-Ogievskiy if (buffer_is_zero(buf, buf_size)) { 232b35ebdf0SVladimir Sementsov-Ogievskiy g_free(buf); 233b35ebdf0SVladimir Sementsov-Ogievskiy buf = NULL; 234b35ebdf0SVladimir Sementsov-Ogievskiy flags |= DIRTY_BITMAP_MIG_FLAG_ZEROES; 235b35ebdf0SVladimir Sementsov-Ogievskiy } 236b35ebdf0SVladimir Sementsov-Ogievskiy 237b35ebdf0SVladimir Sementsov-Ogievskiy trace_send_bitmap_bits(flags, start_sector, nr_sectors, buf_size); 238b35ebdf0SVladimir Sementsov-Ogievskiy 239b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_header(f, dbms, flags); 240b35ebdf0SVladimir Sementsov-Ogievskiy 241b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_be64(f, start_sector); 242b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_be32(f, nr_sectors); 243b35ebdf0SVladimir Sementsov-Ogievskiy 244b35ebdf0SVladimir Sementsov-Ogievskiy /* if a block is zero we need to flush here since the network 245b35ebdf0SVladimir Sementsov-Ogievskiy * bandwidth is now a lot higher than the storage device bandwidth. 246b35ebdf0SVladimir Sementsov-Ogievskiy * thus if we queue zero blocks we slow down the migration. */ 247b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) { 248b35ebdf0SVladimir Sementsov-Ogievskiy qemu_fflush(f); 249b35ebdf0SVladimir Sementsov-Ogievskiy } else { 250b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_be64(f, buf_size); 251b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_buffer(f, buf, buf_size); 252b35ebdf0SVladimir Sementsov-Ogievskiy } 253b35ebdf0SVladimir Sementsov-Ogievskiy 254b35ebdf0SVladimir Sementsov-Ogievskiy g_free(buf); 255b35ebdf0SVladimir Sementsov-Ogievskiy } 256b35ebdf0SVladimir Sementsov-Ogievskiy 257b35ebdf0SVladimir Sementsov-Ogievskiy /* Called with iothread lock taken. */ 258b35ebdf0SVladimir Sementsov-Ogievskiy static void dirty_bitmap_mig_cleanup(void) 259b35ebdf0SVladimir Sementsov-Ogievskiy { 260b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms; 261b35ebdf0SVladimir Sementsov-Ogievskiy 262b35ebdf0SVladimir Sementsov-Ogievskiy while ((dbms = QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != NULL) { 263b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry); 26427a1b301SJohn Snow bdrv_dirty_bitmap_set_busy(dbms->bitmap, false); 265b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_unref(dbms->bs); 266b35ebdf0SVladimir Sementsov-Ogievskiy g_free(dbms); 267b35ebdf0SVladimir Sementsov-Ogievskiy } 268b35ebdf0SVladimir Sementsov-Ogievskiy } 269b35ebdf0SVladimir Sementsov-Ogievskiy 270b35ebdf0SVladimir Sementsov-Ogievskiy /* Called with iothread lock taken. */ 271b35ebdf0SVladimir Sementsov-Ogievskiy static int init_dirty_bitmap_migration(void) 272b35ebdf0SVladimir Sementsov-Ogievskiy { 273b35ebdf0SVladimir Sementsov-Ogievskiy BlockDriverState *bs; 274b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *bitmap; 275b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms; 2763ae96d66SJohn Snow Error *local_err = NULL; 277b35ebdf0SVladimir Sementsov-Ogievskiy 278b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.bulk_completed = false; 279b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.prev_bs = NULL; 280b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.prev_bitmap = NULL; 281b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.no_bitmaps = false; 282b35ebdf0SVladimir Sementsov-Ogievskiy 283592203e7SJohn Snow for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) { 284592203e7SJohn Snow const char *name = bdrv_get_device_or_node_name(bs); 285b35ebdf0SVladimir Sementsov-Ogievskiy 286b35ebdf0SVladimir Sementsov-Ogievskiy for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap; 287b35ebdf0SVladimir Sementsov-Ogievskiy bitmap = bdrv_dirty_bitmap_next(bs, bitmap)) 288b35ebdf0SVladimir Sementsov-Ogievskiy { 289b35ebdf0SVladimir Sementsov-Ogievskiy if (!bdrv_dirty_bitmap_name(bitmap)) { 290b35ebdf0SVladimir Sementsov-Ogievskiy continue; 291b35ebdf0SVladimir Sementsov-Ogievskiy } 292b35ebdf0SVladimir Sementsov-Ogievskiy 293592203e7SJohn Snow if (!name || strcmp(name, "") == 0) { 294b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Found bitmap '%s' in unnamed node %p. It can't " 295b35ebdf0SVladimir Sementsov-Ogievskiy "be migrated", bdrv_dirty_bitmap_name(bitmap), bs); 296b35ebdf0SVladimir Sementsov-Ogievskiy goto fail; 297b35ebdf0SVladimir Sementsov-Ogievskiy } 298b35ebdf0SVladimir Sementsov-Ogievskiy 2993ae96d66SJohn Snow if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, 3003ae96d66SJohn Snow &local_err)) { 3013ae96d66SJohn Snow error_report_err(local_err); 3029c98f145SVladimir Sementsov-Ogievskiy goto fail; 3039c98f145SVladimir Sementsov-Ogievskiy } 3049c98f145SVladimir Sementsov-Ogievskiy 305b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_ref(bs); 30627a1b301SJohn Snow bdrv_dirty_bitmap_set_busy(bitmap, true); 307b35ebdf0SVladimir Sementsov-Ogievskiy 308b35ebdf0SVladimir Sementsov-Ogievskiy dbms = g_new0(DirtyBitmapMigBitmapState, 1); 309b35ebdf0SVladimir Sementsov-Ogievskiy dbms->bs = bs; 310592203e7SJohn Snow dbms->node_name = name; 311b35ebdf0SVladimir Sementsov-Ogievskiy dbms->bitmap = bitmap; 312b35ebdf0SVladimir Sementsov-Ogievskiy dbms->total_sectors = bdrv_nb_sectors(bs); 313b35ebdf0SVladimir Sementsov-Ogievskiy dbms->sectors_per_chunk = CHUNK_SIZE * 8 * 314b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; 315b35ebdf0SVladimir Sementsov-Ogievskiy if (bdrv_dirty_bitmap_enabled(bitmap)) { 316b35ebdf0SVladimir Sementsov-Ogievskiy dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_ENABLED; 317b35ebdf0SVladimir Sementsov-Ogievskiy } 318796a3798SEric Blake if (bdrv_dirty_bitmap_get_persistence(bitmap)) { 319b35ebdf0SVladimir Sementsov-Ogievskiy dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT; 320b35ebdf0SVladimir Sementsov-Ogievskiy } 321b35ebdf0SVladimir Sementsov-Ogievskiy 322b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list, 323b35ebdf0SVladimir Sementsov-Ogievskiy dbms, entry); 324b35ebdf0SVladimir Sementsov-Ogievskiy } 325b35ebdf0SVladimir Sementsov-Ogievskiy } 326b35ebdf0SVladimir Sementsov-Ogievskiy 3279c98f145SVladimir Sementsov-Ogievskiy /* unset migration flags here, to not roll back it */ 328b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { 329c4e4b0faSJohn Snow bdrv_dirty_bitmap_skip_store(dbms->bitmap, true); 330b35ebdf0SVladimir Sementsov-Ogievskiy } 331b35ebdf0SVladimir Sementsov-Ogievskiy 332b35ebdf0SVladimir Sementsov-Ogievskiy if (QSIMPLEQ_EMPTY(&dirty_bitmap_mig_state.dbms_list)) { 333b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.no_bitmaps = true; 334b35ebdf0SVladimir Sementsov-Ogievskiy } 335b35ebdf0SVladimir Sementsov-Ogievskiy 336b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 337b35ebdf0SVladimir Sementsov-Ogievskiy 338b35ebdf0SVladimir Sementsov-Ogievskiy fail: 339b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_cleanup(); 340b35ebdf0SVladimir Sementsov-Ogievskiy 341b35ebdf0SVladimir Sementsov-Ogievskiy return -1; 342b35ebdf0SVladimir Sementsov-Ogievskiy } 343b35ebdf0SVladimir Sementsov-Ogievskiy 344b35ebdf0SVladimir Sementsov-Ogievskiy /* Called with no lock taken. */ 345b35ebdf0SVladimir Sementsov-Ogievskiy static void bulk_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *dbms) 346b35ebdf0SVladimir Sementsov-Ogievskiy { 347b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t nr_sectors = MIN(dbms->total_sectors - dbms->cur_sector, 348b35ebdf0SVladimir Sementsov-Ogievskiy dbms->sectors_per_chunk); 349b35ebdf0SVladimir Sementsov-Ogievskiy 350b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_bits(f, dbms, dbms->cur_sector, nr_sectors); 351b35ebdf0SVladimir Sementsov-Ogievskiy 352b35ebdf0SVladimir Sementsov-Ogievskiy dbms->cur_sector += nr_sectors; 353b35ebdf0SVladimir Sementsov-Ogievskiy if (dbms->cur_sector >= dbms->total_sectors) { 354b35ebdf0SVladimir Sementsov-Ogievskiy dbms->bulk_completed = true; 355b35ebdf0SVladimir Sementsov-Ogievskiy } 356b35ebdf0SVladimir Sementsov-Ogievskiy } 357b35ebdf0SVladimir Sementsov-Ogievskiy 358b35ebdf0SVladimir Sementsov-Ogievskiy /* Called with no lock taken. */ 359b35ebdf0SVladimir Sementsov-Ogievskiy static void bulk_phase(QEMUFile *f, bool limit) 360b35ebdf0SVladimir Sementsov-Ogievskiy { 361b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms; 362b35ebdf0SVladimir Sementsov-Ogievskiy 363b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { 364b35ebdf0SVladimir Sementsov-Ogievskiy while (!dbms->bulk_completed) { 365b35ebdf0SVladimir Sementsov-Ogievskiy bulk_phase_send_chunk(f, dbms); 366b35ebdf0SVladimir Sementsov-Ogievskiy if (limit && qemu_file_rate_limit(f)) { 367b35ebdf0SVladimir Sementsov-Ogievskiy return; 368b35ebdf0SVladimir Sementsov-Ogievskiy } 369b35ebdf0SVladimir Sementsov-Ogievskiy } 370b35ebdf0SVladimir Sementsov-Ogievskiy } 371b35ebdf0SVladimir Sementsov-Ogievskiy 372b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_state.bulk_completed = true; 373b35ebdf0SVladimir Sementsov-Ogievskiy } 374b35ebdf0SVladimir Sementsov-Ogievskiy 375b35ebdf0SVladimir Sementsov-Ogievskiy /* for SaveVMHandlers */ 376b35ebdf0SVladimir Sementsov-Ogievskiy static void dirty_bitmap_save_cleanup(void *opaque) 377b35ebdf0SVladimir Sementsov-Ogievskiy { 378b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_cleanup(); 379b35ebdf0SVladimir Sementsov-Ogievskiy } 380b35ebdf0SVladimir Sementsov-Ogievskiy 381b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque) 382b35ebdf0SVladimir Sementsov-Ogievskiy { 383b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_save_iterate(migration_in_postcopy()); 384b35ebdf0SVladimir Sementsov-Ogievskiy 385b35ebdf0SVladimir Sementsov-Ogievskiy if (migration_in_postcopy() && !dirty_bitmap_mig_state.bulk_completed) { 386b35ebdf0SVladimir Sementsov-Ogievskiy bulk_phase(f, true); 387b35ebdf0SVladimir Sementsov-Ogievskiy } 388b35ebdf0SVladimir Sementsov-Ogievskiy 389b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); 390b35ebdf0SVladimir Sementsov-Ogievskiy 391b35ebdf0SVladimir Sementsov-Ogievskiy return dirty_bitmap_mig_state.bulk_completed; 392b35ebdf0SVladimir Sementsov-Ogievskiy } 393b35ebdf0SVladimir Sementsov-Ogievskiy 394b35ebdf0SVladimir Sementsov-Ogievskiy /* Called with iothread lock taken. */ 395b35ebdf0SVladimir Sementsov-Ogievskiy 396b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) 397b35ebdf0SVladimir Sementsov-Ogievskiy { 398b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms; 399b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_save_complete_enter(); 400b35ebdf0SVladimir Sementsov-Ogievskiy 401b35ebdf0SVladimir Sementsov-Ogievskiy if (!dirty_bitmap_mig_state.bulk_completed) { 402b35ebdf0SVladimir Sementsov-Ogievskiy bulk_phase(f, false); 403b35ebdf0SVladimir Sementsov-Ogievskiy } 404b35ebdf0SVladimir Sementsov-Ogievskiy 405b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { 406b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_complete(f, dbms); 407b35ebdf0SVladimir Sementsov-Ogievskiy } 408b35ebdf0SVladimir Sementsov-Ogievskiy 409b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); 410b35ebdf0SVladimir Sementsov-Ogievskiy 411b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_save_complete_finish(); 412b35ebdf0SVladimir Sementsov-Ogievskiy 413b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_mig_cleanup(); 414b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 415b35ebdf0SVladimir Sementsov-Ogievskiy } 416b35ebdf0SVladimir Sementsov-Ogievskiy 417b35ebdf0SVladimir Sementsov-Ogievskiy static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, 418b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t max_size, 419b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t *res_precopy_only, 420b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t *res_compatible, 421b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t *res_postcopy_only) 422b35ebdf0SVladimir Sementsov-Ogievskiy { 423b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms; 424b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t pending = 0; 425b35ebdf0SVladimir Sementsov-Ogievskiy 426b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_lock_iothread(); 427b35ebdf0SVladimir Sementsov-Ogievskiy 428b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { 429b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t gran = bdrv_dirty_bitmap_granularity(dbms->bitmap); 430b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t sectors = dbms->bulk_completed ? 0 : 431b35ebdf0SVladimir Sementsov-Ogievskiy dbms->total_sectors - dbms->cur_sector; 432b35ebdf0SVladimir Sementsov-Ogievskiy 433b35ebdf0SVladimir Sementsov-Ogievskiy pending += DIV_ROUND_UP(sectors * BDRV_SECTOR_SIZE, gran); 434b35ebdf0SVladimir Sementsov-Ogievskiy } 435b35ebdf0SVladimir Sementsov-Ogievskiy 436b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_unlock_iothread(); 437b35ebdf0SVladimir Sementsov-Ogievskiy 438b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_save_pending(pending, max_size); 439b35ebdf0SVladimir Sementsov-Ogievskiy 440b35ebdf0SVladimir Sementsov-Ogievskiy *res_postcopy_only += pending; 441b35ebdf0SVladimir Sementsov-Ogievskiy } 442b35ebdf0SVladimir Sementsov-Ogievskiy 443b35ebdf0SVladimir Sementsov-Ogievskiy /* First occurrence of this bitmap. It should be created if doesn't exist */ 444b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_load_start(QEMUFile *f, DirtyBitmapLoadState *s) 445b35ebdf0SVladimir Sementsov-Ogievskiy { 446b35ebdf0SVladimir Sementsov-Ogievskiy Error *local_err = NULL; 447b35ebdf0SVladimir Sementsov-Ogievskiy uint32_t granularity = qemu_get_be32(f); 448b35ebdf0SVladimir Sementsov-Ogievskiy uint8_t flags = qemu_get_byte(f); 449b35ebdf0SVladimir Sementsov-Ogievskiy 450b35ebdf0SVladimir Sementsov-Ogievskiy if (s->bitmap) { 451b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Bitmap with the same name ('%s') already exists on " 452b35ebdf0SVladimir Sementsov-Ogievskiy "destination", bdrv_dirty_bitmap_name(s->bitmap)); 453b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 454b35ebdf0SVladimir Sementsov-Ogievskiy } else { 455b35ebdf0SVladimir Sementsov-Ogievskiy s->bitmap = bdrv_create_dirty_bitmap(s->bs, granularity, 456b35ebdf0SVladimir Sementsov-Ogievskiy s->bitmap_name, &local_err); 457b35ebdf0SVladimir Sementsov-Ogievskiy if (!s->bitmap) { 458b35ebdf0SVladimir Sementsov-Ogievskiy error_report_err(local_err); 459b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 460b35ebdf0SVladimir Sementsov-Ogievskiy } 461b35ebdf0SVladimir Sementsov-Ogievskiy } 462b35ebdf0SVladimir Sementsov-Ogievskiy 463b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_START_FLAG_RESERVED_MASK) { 464b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Unknown flags in migrated dirty bitmap header: %x", 465b35ebdf0SVladimir Sementsov-Ogievskiy flags); 466b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 467b35ebdf0SVladimir Sementsov-Ogievskiy } 468b35ebdf0SVladimir Sementsov-Ogievskiy 469b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT) { 470796a3798SEric Blake bdrv_dirty_bitmap_set_persistence(s->bitmap, true); 471b35ebdf0SVladimir Sementsov-Ogievskiy } 472b35ebdf0SVladimir Sementsov-Ogievskiy 473b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_disable_dirty_bitmap(s->bitmap); 474b35ebdf0SVladimir Sementsov-Ogievskiy if (flags & DIRTY_BITMAP_MIG_START_FLAG_ENABLED) { 475b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapLoadBitmapState *b; 476b35ebdf0SVladimir Sementsov-Ogievskiy 477b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_create_successor(s->bs, s->bitmap, &local_err); 478b35ebdf0SVladimir Sementsov-Ogievskiy if (local_err) { 479b35ebdf0SVladimir Sementsov-Ogievskiy error_report_err(local_err); 480b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 481b35ebdf0SVladimir Sementsov-Ogievskiy } 482b35ebdf0SVladimir Sementsov-Ogievskiy 483b35ebdf0SVladimir Sementsov-Ogievskiy b = g_new(DirtyBitmapLoadBitmapState, 1); 484b35ebdf0SVladimir Sementsov-Ogievskiy b->bs = s->bs; 485b35ebdf0SVladimir Sementsov-Ogievskiy b->bitmap = s->bitmap; 486b35ebdf0SVladimir Sementsov-Ogievskiy b->migrated = false; 487b35ebdf0SVladimir Sementsov-Ogievskiy enabled_bitmaps = g_slist_prepend(enabled_bitmaps, b); 488b35ebdf0SVladimir Sementsov-Ogievskiy } 489b35ebdf0SVladimir Sementsov-Ogievskiy 490b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 491b35ebdf0SVladimir Sementsov-Ogievskiy } 492b35ebdf0SVladimir Sementsov-Ogievskiy 493b35ebdf0SVladimir Sementsov-Ogievskiy void dirty_bitmap_mig_before_vm_start(void) 494b35ebdf0SVladimir Sementsov-Ogievskiy { 495b35ebdf0SVladimir Sementsov-Ogievskiy GSList *item; 496b35ebdf0SVladimir Sementsov-Ogievskiy 497b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_lock(&finish_lock); 498b35ebdf0SVladimir Sementsov-Ogievskiy 499b35ebdf0SVladimir Sementsov-Ogievskiy for (item = enabled_bitmaps; item; item = g_slist_next(item)) { 500b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapLoadBitmapState *b = item->data; 501b35ebdf0SVladimir Sementsov-Ogievskiy 502b35ebdf0SVladimir Sementsov-Ogievskiy if (b->migrated) { 50358f72b96SVladimir Sementsov-Ogievskiy bdrv_enable_dirty_bitmap_locked(b->bitmap); 504b35ebdf0SVladimir Sementsov-Ogievskiy } else { 505b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_enable_successor(b->bitmap); 506b35ebdf0SVladimir Sementsov-Ogievskiy } 507b35ebdf0SVladimir Sementsov-Ogievskiy 508b35ebdf0SVladimir Sementsov-Ogievskiy g_free(b); 509b35ebdf0SVladimir Sementsov-Ogievskiy } 510b35ebdf0SVladimir Sementsov-Ogievskiy 511b35ebdf0SVladimir Sementsov-Ogievskiy g_slist_free(enabled_bitmaps); 512b35ebdf0SVladimir Sementsov-Ogievskiy enabled_bitmaps = NULL; 513b35ebdf0SVladimir Sementsov-Ogievskiy 514b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_unlock(&finish_lock); 515b35ebdf0SVladimir Sementsov-Ogievskiy } 516b35ebdf0SVladimir Sementsov-Ogievskiy 517b35ebdf0SVladimir Sementsov-Ogievskiy static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *s) 518b35ebdf0SVladimir Sementsov-Ogievskiy { 519b35ebdf0SVladimir Sementsov-Ogievskiy GSList *item; 520b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_complete(); 521b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_deserialize_finish(s->bitmap); 522b35ebdf0SVladimir Sementsov-Ogievskiy 523b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_lock(&finish_lock); 524b35ebdf0SVladimir Sementsov-Ogievskiy 525b35ebdf0SVladimir Sementsov-Ogievskiy for (item = enabled_bitmaps; item; item = g_slist_next(item)) { 526b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapLoadBitmapState *b = item->data; 527b35ebdf0SVladimir Sementsov-Ogievskiy 528b35ebdf0SVladimir Sementsov-Ogievskiy if (b->bitmap == s->bitmap) { 529b35ebdf0SVladimir Sementsov-Ogievskiy b->migrated = true; 530b35ebdf0SVladimir Sementsov-Ogievskiy break; 531b35ebdf0SVladimir Sementsov-Ogievskiy } 532b35ebdf0SVladimir Sementsov-Ogievskiy } 533b35ebdf0SVladimir Sementsov-Ogievskiy 53450a47257SJohn Snow if (bdrv_dirty_bitmap_has_successor(s->bitmap)) { 535b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_lock(s->bitmap); 536b35ebdf0SVladimir Sementsov-Ogievskiy if (enabled_bitmaps == NULL) { 537b35ebdf0SVladimir Sementsov-Ogievskiy /* in postcopy */ 538b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_reclaim_dirty_bitmap_locked(s->bs, s->bitmap, &error_abort); 53958f72b96SVladimir Sementsov-Ogievskiy bdrv_enable_dirty_bitmap_locked(s->bitmap); 540b35ebdf0SVladimir Sementsov-Ogievskiy } else { 541b35ebdf0SVladimir Sementsov-Ogievskiy /* target not started, successor must be empty */ 542b35ebdf0SVladimir Sementsov-Ogievskiy int64_t count = bdrv_get_dirty_count(s->bitmap); 543b35ebdf0SVladimir Sementsov-Ogievskiy BdrvDirtyBitmap *ret = bdrv_reclaim_dirty_bitmap_locked(s->bs, 544b35ebdf0SVladimir Sementsov-Ogievskiy s->bitmap, 545b35ebdf0SVladimir Sementsov-Ogievskiy NULL); 546b35ebdf0SVladimir Sementsov-Ogievskiy /* bdrv_reclaim_dirty_bitmap can fail only on no successor (it 547b35ebdf0SVladimir Sementsov-Ogievskiy * must be) or on merge fail, but merge can't fail when second 548b35ebdf0SVladimir Sementsov-Ogievskiy * bitmap is empty 549b35ebdf0SVladimir Sementsov-Ogievskiy */ 550b35ebdf0SVladimir Sementsov-Ogievskiy assert(ret == s->bitmap && 551b35ebdf0SVladimir Sementsov-Ogievskiy count == bdrv_get_dirty_count(s->bitmap)); 552b35ebdf0SVladimir Sementsov-Ogievskiy } 553b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_unlock(s->bitmap); 554b35ebdf0SVladimir Sementsov-Ogievskiy } 555b35ebdf0SVladimir Sementsov-Ogievskiy 556b35ebdf0SVladimir Sementsov-Ogievskiy qemu_mutex_unlock(&finish_lock); 557b35ebdf0SVladimir Sementsov-Ogievskiy } 558b35ebdf0SVladimir Sementsov-Ogievskiy 559b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s) 560b35ebdf0SVladimir Sementsov-Ogievskiy { 561b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t first_byte = qemu_get_be64(f) << BDRV_SECTOR_BITS; 562b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t nr_bytes = (uint64_t)qemu_get_be32(f) << BDRV_SECTOR_BITS; 563b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_bits_enter(first_byte >> BDRV_SECTOR_BITS, 564b35ebdf0SVladimir Sementsov-Ogievskiy nr_bytes >> BDRV_SECTOR_BITS); 565b35ebdf0SVladimir Sementsov-Ogievskiy 566b35ebdf0SVladimir Sementsov-Ogievskiy if (s->flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) { 567b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_bits_zeroes(); 568b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_byte, nr_bytes, 569b35ebdf0SVladimir Sementsov-Ogievskiy false); 570b35ebdf0SVladimir Sementsov-Ogievskiy } else { 571b35ebdf0SVladimir Sementsov-Ogievskiy size_t ret; 572b35ebdf0SVladimir Sementsov-Ogievskiy uint8_t *buf; 573b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t buf_size = qemu_get_be64(f); 574b35ebdf0SVladimir Sementsov-Ogievskiy uint64_t needed_size = 575b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_serialization_size(s->bitmap, 576b35ebdf0SVladimir Sementsov-Ogievskiy first_byte, nr_bytes); 577b35ebdf0SVladimir Sementsov-Ogievskiy 578b35ebdf0SVladimir Sementsov-Ogievskiy if (needed_size > buf_size || 579b35ebdf0SVladimir Sementsov-Ogievskiy buf_size > QEMU_ALIGN_UP(needed_size, 4 * sizeof(long)) 580b35ebdf0SVladimir Sementsov-Ogievskiy /* Here used same alignment as in send_bitmap_bits */ 581b35ebdf0SVladimir Sementsov-Ogievskiy ) { 582b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Migrated bitmap granularity doesn't " 583b35ebdf0SVladimir Sementsov-Ogievskiy "match the destination bitmap '%s' granularity", 584b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_name(s->bitmap)); 585b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 586b35ebdf0SVladimir Sementsov-Ogievskiy } 587b35ebdf0SVladimir Sementsov-Ogievskiy 588b35ebdf0SVladimir Sementsov-Ogievskiy buf = g_malloc(buf_size); 589b35ebdf0SVladimir Sementsov-Ogievskiy ret = qemu_get_buffer(f, buf, buf_size); 590b35ebdf0SVladimir Sementsov-Ogievskiy if (ret != buf_size) { 591b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Failed to read bitmap bits"); 59216a22278SVladimir Sementsov-Ogievskiy g_free(buf); 593b35ebdf0SVladimir Sementsov-Ogievskiy return -EIO; 594b35ebdf0SVladimir Sementsov-Ogievskiy } 595b35ebdf0SVladimir Sementsov-Ogievskiy 596b35ebdf0SVladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_deserialize_part(s->bitmap, buf, first_byte, nr_bytes, 597b35ebdf0SVladimir Sementsov-Ogievskiy false); 598b35ebdf0SVladimir Sementsov-Ogievskiy g_free(buf); 599b35ebdf0SVladimir Sementsov-Ogievskiy } 600b35ebdf0SVladimir Sementsov-Ogievskiy 601b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 602b35ebdf0SVladimir Sementsov-Ogievskiy } 603b35ebdf0SVladimir Sementsov-Ogievskiy 604b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_load_header(QEMUFile *f, DirtyBitmapLoadState *s) 605b35ebdf0SVladimir Sementsov-Ogievskiy { 606b35ebdf0SVladimir Sementsov-Ogievskiy Error *local_err = NULL; 607b35ebdf0SVladimir Sementsov-Ogievskiy bool nothing; 608b35ebdf0SVladimir Sementsov-Ogievskiy s->flags = qemu_get_bitmap_flags(f); 609b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_header(s->flags); 610b35ebdf0SVladimir Sementsov-Ogievskiy 611b35ebdf0SVladimir Sementsov-Ogievskiy nothing = s->flags == (s->flags & DIRTY_BITMAP_MIG_FLAG_EOS); 612b35ebdf0SVladimir Sementsov-Ogievskiy 613b35ebdf0SVladimir Sementsov-Ogievskiy if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { 614b35ebdf0SVladimir Sementsov-Ogievskiy if (!qemu_get_counted_string(f, s->node_name)) { 615b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Unable to read node name string"); 616b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 617b35ebdf0SVladimir Sementsov-Ogievskiy } 618b35ebdf0SVladimir Sementsov-Ogievskiy s->bs = bdrv_lookup_bs(s->node_name, s->node_name, &local_err); 619b35ebdf0SVladimir Sementsov-Ogievskiy if (!s->bs) { 620b35ebdf0SVladimir Sementsov-Ogievskiy error_report_err(local_err); 621b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 622b35ebdf0SVladimir Sementsov-Ogievskiy } 623b35ebdf0SVladimir Sementsov-Ogievskiy } else if (!s->bs && !nothing) { 624b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Error: block device name is not set"); 625b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 626b35ebdf0SVladimir Sementsov-Ogievskiy } 627b35ebdf0SVladimir Sementsov-Ogievskiy 628b35ebdf0SVladimir Sementsov-Ogievskiy if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { 629b35ebdf0SVladimir Sementsov-Ogievskiy if (!qemu_get_counted_string(f, s->bitmap_name)) { 630b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Unable to read bitmap name string"); 631b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 632b35ebdf0SVladimir Sementsov-Ogievskiy } 633b35ebdf0SVladimir Sementsov-Ogievskiy s->bitmap = bdrv_find_dirty_bitmap(s->bs, s->bitmap_name); 634b35ebdf0SVladimir Sementsov-Ogievskiy 635b35ebdf0SVladimir Sementsov-Ogievskiy /* bitmap may be NULL here, it wouldn't be an error if it is the 636b35ebdf0SVladimir Sementsov-Ogievskiy * first occurrence of the bitmap */ 637b35ebdf0SVladimir Sementsov-Ogievskiy if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) { 638b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Error: unknown dirty bitmap " 639b35ebdf0SVladimir Sementsov-Ogievskiy "'%s' for block device '%s'", 640b35ebdf0SVladimir Sementsov-Ogievskiy s->bitmap_name, s->node_name); 641b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 642b35ebdf0SVladimir Sementsov-Ogievskiy } 643b35ebdf0SVladimir Sementsov-Ogievskiy } else if (!s->bitmap && !nothing) { 644b35ebdf0SVladimir Sementsov-Ogievskiy error_report("Error: block device name is not set"); 645b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 646b35ebdf0SVladimir Sementsov-Ogievskiy } 647b35ebdf0SVladimir Sementsov-Ogievskiy 648b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 649b35ebdf0SVladimir Sementsov-Ogievskiy } 650b35ebdf0SVladimir Sementsov-Ogievskiy 651b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) 652b35ebdf0SVladimir Sementsov-Ogievskiy { 653b35ebdf0SVladimir Sementsov-Ogievskiy static DirtyBitmapLoadState s; 654b35ebdf0SVladimir Sementsov-Ogievskiy int ret = 0; 655b35ebdf0SVladimir Sementsov-Ogievskiy 656b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_enter(); 657b35ebdf0SVladimir Sementsov-Ogievskiy 658b35ebdf0SVladimir Sementsov-Ogievskiy if (version_id != 1) { 659b35ebdf0SVladimir Sementsov-Ogievskiy return -EINVAL; 660b35ebdf0SVladimir Sementsov-Ogievskiy } 661b35ebdf0SVladimir Sementsov-Ogievskiy 662b35ebdf0SVladimir Sementsov-Ogievskiy do { 663b35ebdf0SVladimir Sementsov-Ogievskiy ret = dirty_bitmap_load_header(f, &s); 664a36f6ff4SVladimir Sementsov-Ogievskiy if (ret < 0) { 665a36f6ff4SVladimir Sementsov-Ogievskiy return ret; 666a36f6ff4SVladimir Sementsov-Ogievskiy } 667b35ebdf0SVladimir Sementsov-Ogievskiy 668b35ebdf0SVladimir Sementsov-Ogievskiy if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) { 669b35ebdf0SVladimir Sementsov-Ogievskiy ret = dirty_bitmap_load_start(f, &s); 670b35ebdf0SVladimir Sementsov-Ogievskiy } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) { 671b35ebdf0SVladimir Sementsov-Ogievskiy dirty_bitmap_load_complete(f, &s); 672b35ebdf0SVladimir Sementsov-Ogievskiy } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_BITS) { 673b35ebdf0SVladimir Sementsov-Ogievskiy ret = dirty_bitmap_load_bits(f, &s); 674b35ebdf0SVladimir Sementsov-Ogievskiy } 675b35ebdf0SVladimir Sementsov-Ogievskiy 676b35ebdf0SVladimir Sementsov-Ogievskiy if (!ret) { 677b35ebdf0SVladimir Sementsov-Ogievskiy ret = qemu_file_get_error(f); 678b35ebdf0SVladimir Sementsov-Ogievskiy } 679b35ebdf0SVladimir Sementsov-Ogievskiy 680b35ebdf0SVladimir Sementsov-Ogievskiy if (ret) { 681b35ebdf0SVladimir Sementsov-Ogievskiy return ret; 682b35ebdf0SVladimir Sementsov-Ogievskiy } 683b35ebdf0SVladimir Sementsov-Ogievskiy } while (!(s.flags & DIRTY_BITMAP_MIG_FLAG_EOS)); 684b35ebdf0SVladimir Sementsov-Ogievskiy 685b35ebdf0SVladimir Sementsov-Ogievskiy trace_dirty_bitmap_load_success(); 686b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 687b35ebdf0SVladimir Sementsov-Ogievskiy } 688b35ebdf0SVladimir Sementsov-Ogievskiy 689b35ebdf0SVladimir Sementsov-Ogievskiy static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque) 690b35ebdf0SVladimir Sementsov-Ogievskiy { 691b35ebdf0SVladimir Sementsov-Ogievskiy DirtyBitmapMigBitmapState *dbms = NULL; 692b35ebdf0SVladimir Sementsov-Ogievskiy if (init_dirty_bitmap_migration() < 0) { 693b35ebdf0SVladimir Sementsov-Ogievskiy return -1; 694b35ebdf0SVladimir Sementsov-Ogievskiy } 695b35ebdf0SVladimir Sementsov-Ogievskiy 696b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { 697b35ebdf0SVladimir Sementsov-Ogievskiy send_bitmap_start(f, dbms); 698b35ebdf0SVladimir Sementsov-Ogievskiy } 699b35ebdf0SVladimir Sementsov-Ogievskiy qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); 700b35ebdf0SVladimir Sementsov-Ogievskiy 701b35ebdf0SVladimir Sementsov-Ogievskiy return 0; 702b35ebdf0SVladimir Sementsov-Ogievskiy } 703b35ebdf0SVladimir Sementsov-Ogievskiy 704b35ebdf0SVladimir Sementsov-Ogievskiy static bool dirty_bitmap_is_active(void *opaque) 705b35ebdf0SVladimir Sementsov-Ogievskiy { 706b35ebdf0SVladimir Sementsov-Ogievskiy return migrate_dirty_bitmaps() && !dirty_bitmap_mig_state.no_bitmaps; 707b35ebdf0SVladimir Sementsov-Ogievskiy } 708b35ebdf0SVladimir Sementsov-Ogievskiy 709b35ebdf0SVladimir Sementsov-Ogievskiy static bool dirty_bitmap_is_active_iterate(void *opaque) 710b35ebdf0SVladimir Sementsov-Ogievskiy { 711b35ebdf0SVladimir Sementsov-Ogievskiy return dirty_bitmap_is_active(opaque) && !runstate_is_running(); 712b35ebdf0SVladimir Sementsov-Ogievskiy } 713b35ebdf0SVladimir Sementsov-Ogievskiy 714b35ebdf0SVladimir Sementsov-Ogievskiy static bool dirty_bitmap_has_postcopy(void *opaque) 715b35ebdf0SVladimir Sementsov-Ogievskiy { 716b35ebdf0SVladimir Sementsov-Ogievskiy return true; 717b35ebdf0SVladimir Sementsov-Ogievskiy } 718b35ebdf0SVladimir Sementsov-Ogievskiy 719b35ebdf0SVladimir Sementsov-Ogievskiy static SaveVMHandlers savevm_dirty_bitmap_handlers = { 720b35ebdf0SVladimir Sementsov-Ogievskiy .save_setup = dirty_bitmap_save_setup, 721b35ebdf0SVladimir Sementsov-Ogievskiy .save_live_complete_postcopy = dirty_bitmap_save_complete, 722b35ebdf0SVladimir Sementsov-Ogievskiy .save_live_complete_precopy = dirty_bitmap_save_complete, 723b35ebdf0SVladimir Sementsov-Ogievskiy .has_postcopy = dirty_bitmap_has_postcopy, 724b35ebdf0SVladimir Sementsov-Ogievskiy .save_live_pending = dirty_bitmap_save_pending, 725b35ebdf0SVladimir Sementsov-Ogievskiy .save_live_iterate = dirty_bitmap_save_iterate, 726b35ebdf0SVladimir Sementsov-Ogievskiy .is_active_iterate = dirty_bitmap_is_active_iterate, 727b35ebdf0SVladimir Sementsov-Ogievskiy .load_state = dirty_bitmap_load, 728b35ebdf0SVladimir Sementsov-Ogievskiy .save_cleanup = dirty_bitmap_save_cleanup, 729b35ebdf0SVladimir Sementsov-Ogievskiy .is_active = dirty_bitmap_is_active, 730b35ebdf0SVladimir Sementsov-Ogievskiy }; 731b35ebdf0SVladimir Sementsov-Ogievskiy 732b35ebdf0SVladimir Sementsov-Ogievskiy void dirty_bitmap_mig_init(void) 733b35ebdf0SVladimir Sementsov-Ogievskiy { 734b35ebdf0SVladimir Sementsov-Ogievskiy QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); 735b35ebdf0SVladimir Sementsov-Ogievskiy 736*ce62df53SDr. David Alan Gilbert register_savevm_live("dirty-bitmap", 0, 1, 737b35ebdf0SVladimir Sementsov-Ogievskiy &savevm_dirty_bitmap_handlers, 738b35ebdf0SVladimir Sementsov-Ogievskiy &dirty_bitmap_mig_state); 739b35ebdf0SVladimir Sementsov-Ogievskiy } 740