1..
2    Copyright (C) 2017 Red Hat Inc.
3
4    This work is licensed under the terms of the GNU GPL, version 2 or
5    later.  See the COPYING file in the top-level directory.
6
7============================
8Live Block Device Operations
9============================
10
11QEMU Block Layer currently (as of QEMU 2.9) supports four major kinds of
12live block device jobs -- stream, commit, mirror, and backup.  These can
13be used to manipulate disk image chains to accomplish certain tasks,
14namely: live copy data from backing files into overlays; shorten long
15disk image chains by merging data from overlays into backing files; live
16synchronize data from a disk image chain (including current active disk)
17to another target image; and point-in-time (and incremental) backups of
18a block device.  Below is a description of the said block (QMP)
19primitives, and some (non-exhaustive list of) examples to illustrate
20their use.
21
22.. note::
23    The file ``qapi/block-core.json`` in the QEMU source tree has the
24    canonical QEMU API (QAPI) schema documentation for the QMP
25    primitives discussed here.
26
27.. todo (kashyapc):: Remove the ".. contents::" directive when Sphinx is
28                     integrated.
29
30.. contents::
31
32Disk image backing chain notation
33---------------------------------
34
35A simple disk image chain.  (This can be created live using QMP
36``blockdev-snapshot-sync``, or offline via ``qemu-img``)::
37
38                   (Live QEMU)
39                        |
40                        .
41                        V
42
43            [A] <----- [B]
44
45    (backing file)    (overlay)
46
47The arrow can be read as: Image [A] is the backing file of disk image
48[B].  And live QEMU is currently writing to image [B], consequently, it
49is also referred to as the "active layer".
50
51There are two kinds of terminology that are common when referring to
52files in a disk image backing chain:
53
54(1) Directional: 'base' and 'top'.  Given the simple disk image chain
55    above, image [A] can be referred to as 'base', and image [B] as
56    'top'.  (This terminology can be seen in in QAPI schema file,
57    block-core.json.)
58
59(2) Relational: 'backing file' and 'overlay'.  Again, taking the same
60    simple disk image chain from the above, disk image [A] is referred
61    to as the backing file, and image [B] as overlay.
62
63   Throughout this document, we will use the relational terminology.
64
65.. important::
66    The overlay files can generally be any format that supports a
67    backing file, although QCOW2 is the preferred format and the one
68    used in this document.
69
70
71Brief overview of live block QMP primitives
72-------------------------------------------
73
74The following are the four different kinds of live block operations that
75QEMU block layer supports.
76
77(1) ``block-stream``: Live copy of data from backing files into overlay
78    files.
79
80    .. note:: Once the 'stream' operation has finished, three things to
81              note:
82
83                (a) QEMU rewrites the backing chain to remove
84                    reference to the now-streamed and redundant backing
85                    file;
86
87                (b) the streamed file *itself* won't be removed by QEMU,
88                    and must be explicitly discarded by the user;
89
90                (c) the streamed file remains valid -- i.e. further
91                    overlays can be created based on it.  Refer the
92                    ``block-stream`` section further below for more
93                    details.
94
95(2) ``block-commit``: Live merge of data from overlay files into backing
96    files (with the optional goal of removing the overlay file from the
97    chain).  Since QEMU 2.0, this includes "active ``block-commit``"
98    (i.e. merge the current active layer into the base image).
99
100    .. note:: Once the 'commit' operation has finished, there are three
101              things to note here as well:
102
103                (a) QEMU rewrites the backing chain to remove reference
104                    to now-redundant overlay images that have been
105                    committed into a backing file;
106
107                (b) the committed file *itself* won't be removed by QEMU
108                    -- it ought to be manually removed;
109
110                (c) however, unlike in the case of ``block-stream``, the
111                    intermediate images will be rendered invalid -- i.e.
112                    no more further overlays can be created based on
113                    them.  Refer the ``block-commit`` section further
114                    below for more details.
115
116(3) ``drive-mirror`` (and ``blockdev-mirror``): Synchronize a running
117    disk to another image.
118
119(4) ``drive-backup`` (and ``blockdev-backup``): Point-in-time (live) copy
120    of a block device to a destination.
121
122
123.. _`Interacting with a QEMU instance`:
124
125Interacting with a QEMU instance
126--------------------------------
127
128To show some example invocations of command-line, we will use the
129following invocation of QEMU, with a QMP server running over UNIX
130socket:
131
132.. parsed-literal::
133
134  $ |qemu_system| -display none -no-user-config -nodefaults \\
135    -m 512 -blockdev \\
136    node-name=node-A,driver=qcow2,file.driver=file,file.node-name=file,file.filename=./a.qcow2 \\
137    -device virtio-blk,drive=node-A,id=virtio0 \\
138    -monitor stdio -qmp unix:/tmp/qmp-sock,server=on,wait=off
139
140The ``-blockdev`` command-line option, used above, is available from
141QEMU 2.9 onwards.  In the above invocation, notice the ``node-name``
142parameter that is used to refer to the disk image a.qcow2 ('node-A') --
143this is a cleaner way to refer to a disk image (as opposed to referring
144to it by spelling out file paths).  So, we will continue to designate a
145``node-name`` to each further disk image created (either via
146``blockdev-snapshot-sync``, or ``blockdev-add``) as part of the disk
147image chain, and continue to refer to the disks using their
148``node-name`` (where possible, because ``block-commit`` does not yet, as
149of QEMU 2.9, accept ``node-name`` parameter) when performing various
150block operations.
151
152To interact with the QEMU instance launched above, we will use the
153``qmp-shell`` utility (located at: ``qemu/scripts/qmp``, as part of the
154QEMU source directory), which takes key-value pairs for QMP commands.
155Invoke it as below (which will also print out the complete raw JSON
156syntax for reference -- examples in the following sections)::
157
158    $ ./qmp-shell -v -p /tmp/qmp-sock
159    (QEMU)
160
161.. note::
162    In the event we have to repeat a certain QMP command, we will: for
163    the first occurrence of it, show the ``qmp-shell`` invocation, *and*
164    the corresponding raw JSON QMP syntax; but for subsequent
165    invocations, present just the ``qmp-shell`` syntax, and omit the
166    equivalent JSON output.
167
168
169Example disk image chain
170------------------------
171
172We will use the below disk image chain (and occasionally spelling it
173out where appropriate) when discussing various primitives::
174
175    [A] <-- [B] <-- [C] <-- [D]
176
177Where [A] is the original base image; [B] and [C] are intermediate
178overlay images; image [D] is the active layer -- i.e. live QEMU is
179writing to it.  (The rule of thumb is: live QEMU will always be pointing
180to the rightmost image in a disk image chain.)
181
182The above image chain can be created by invoking
183``blockdev-snapshot-sync`` commands as following (which shows the
184creation of overlay image [B]) using the ``qmp-shell`` (our invocation
185also prints the raw JSON invocation of it)::
186
187    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
188    {
189        "execute": "blockdev-snapshot-sync",
190        "arguments": {
191            "node-name": "node-A",
192            "snapshot-file": "b.qcow2",
193            "format": "qcow2",
194            "snapshot-node-name": "node-B"
195        }
196    }
197
198Here, "node-A" is the name QEMU internally uses to refer to the base
199image [A] -- it is the backing file, based on which the overlay image,
200[B], is created.
201
202To create the rest of the overlay images, [C], and [D] (omitting the raw
203JSON output for brevity)::
204
205    (QEMU) blockdev-snapshot-sync node-name=node-B snapshot-file=c.qcow2 snapshot-node-name=node-C format=qcow2
206    (QEMU) blockdev-snapshot-sync node-name=node-C snapshot-file=d.qcow2 snapshot-node-name=node-D format=qcow2
207
208
209A note on points-in-time vs file names
210--------------------------------------
211
212In our disk image chain::
213
214    [A] <-- [B] <-- [C] <-- [D]
215
216We have *three* points in time and an active layer:
217
218- Point 1: Guest state when [B] was created is contained in file [A]
219- Point 2: Guest state when [C] was created is contained in [A] + [B]
220- Point 3: Guest state when [D] was created is contained in
221  [A] + [B] + [C]
222- Active layer: Current guest state is contained in [A] + [B] + [C] +
223  [D]
224
225Therefore, be aware with naming choices:
226
227- Naming a file after the time it is created is misleading -- the
228  guest data for that point in time is *not* contained in that file
229  (as explained earlier)
230- Rather, think of files as a *delta* from the backing file
231
232
233Live block streaming --- ``block-stream``
234-----------------------------------------
235
236The ``block-stream`` command allows you to do live copy data from backing
237files into overlay images.
238
239Given our original example disk image chain from earlier::
240
241    [A] <-- [B] <-- [C] <-- [D]
242
243The disk image chain can be shortened in one of the following different
244ways (not an exhaustive list).
245
246.. _`Case-1`:
247
248(1) Merge everything into the active layer: I.e. copy all contents from
249    the base image, [A], and overlay images, [B] and [C], into [D],
250    *while* the guest is running.  The resulting chain will be a
251    standalone image, [D] -- with contents from [A], [B] and [C] merged
252    into it (where live QEMU writes go to)::
253
254        [D]
255
256.. _`Case-2`:
257
258(2) Taking the same example disk image chain mentioned earlier, merge
259    only images [B] and [C] into [D], the active layer.  The result will
260    be contents of images [B] and [C] will be copied into [D], and the
261    backing file pointer of image [D] will be adjusted to point to image
262    [A].  The resulting chain will be::
263
264        [A] <-- [D]
265
266.. _`Case-3`:
267
268(3) Intermediate streaming (available since QEMU 2.8): Starting afresh
269    with the original example disk image chain, with a total of four
270    images, it is possible to copy contents from image [B] into image
271    [C].  Once the copy is finished, image [B] can now be (optionally)
272    discarded; and the backing file pointer of image [C] will be
273    adjusted to point to [A].  I.e. after performing "intermediate
274    streaming" of [B] into [C], the resulting image chain will be (where
275    live QEMU is writing to [D])::
276
277        [A] <-- [C] <-- [D]
278
279
280QMP invocation for ``block-stream``
281~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
282
283For `Case-1`_, to merge contents of all the backing files into the
284active layer, where 'node-D' is the current active image (by default
285``block-stream`` will flatten the entire chain); ``qmp-shell`` (and its
286corresponding JSON output)::
287
288    (QEMU) block-stream device=node-D job-id=job0
289    {
290        "execute": "block-stream",
291        "arguments": {
292            "device": "node-D",
293            "job-id": "job0"
294        }
295    }
296
297For `Case-2`_, merge contents of the images [B] and [C] into [D], where
298image [D] ends up referring to image [A] as its backing file::
299
300    (QEMU) block-stream device=node-D base-node=node-A job-id=job0
301
302And for `Case-3`_, of "intermediate" streaming", merge contents of
303images [B] into [C], where [C] ends up referring to [A] as its backing
304image::
305
306    (QEMU) block-stream device=node-C base-node=node-A job-id=job0
307
308Progress of a ``block-stream`` operation can be monitored via the QMP
309command::
310
311    (QEMU) query-block-jobs
312    {
313        "execute": "query-block-jobs",
314        "arguments": {}
315    }
316
317
318Once the ``block-stream`` operation has completed, QEMU will emit an
319event, ``BLOCK_JOB_COMPLETED``.  The intermediate overlays remain valid,
320and can now be (optionally) discarded, or retained to create further
321overlays based on them.  Finally, the ``block-stream`` jobs can be
322restarted at anytime.
323
324
325Live block commit --- ``block-commit``
326--------------------------------------
327
328The ``block-commit`` command lets you merge live data from overlay
329images into backing file(s).  Since QEMU 2.0, this includes "live active
330commit" (i.e. it is possible to merge the "active layer", the right-most
331image in a disk image chain where live QEMU will be writing to, into the
332base image).  This is analogous to ``block-stream``, but in the opposite
333direction.
334
335Again, starting afresh with our example disk image chain, where live
336QEMU is writing to the right-most image in the chain, [D]::
337
338    [A] <-- [B] <-- [C] <-- [D]
339
340The disk image chain can be shortened in one of the following ways:
341
342.. _`block-commit_Case-1`:
343
344(1) Commit content from only image [B] into image [A].  The resulting
345    chain is the following, where image [C] is adjusted to point at [A]
346    as its new backing file::
347
348        [A] <-- [C] <-- [D]
349
350(2) Commit content from images [B] and [C] into image [A].  The
351    resulting chain, where image [D] is adjusted to point to image [A]
352    as its new backing file::
353
354        [A] <-- [D]
355
356.. _`block-commit_Case-3`:
357
358(3) Commit content from images [B], [C], and the active layer [D] into
359    image [A].  The resulting chain (in this case, a consolidated single
360    image)::
361
362        [A]
363
364(4) Commit content from image only image [C] into image [B].  The
365    resulting chain::
366
367	[A] <-- [B] <-- [D]
368
369(5) Commit content from image [C] and the active layer [D] into image
370    [B].  The resulting chain::
371
372	[A] <-- [B]
373
374
375QMP invocation for ``block-commit``
376~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
377
378For :ref:`Case-1 <block-commit_Case-1>`, to merge contents only from
379image [B] into image [A], the invocation is as follows::
380
381    (QEMU) block-commit device=node-D base=a.qcow2 top=b.qcow2 job-id=job0
382    {
383        "execute": "block-commit",
384        "arguments": {
385            "device": "node-D",
386            "job-id": "job0",
387            "top": "b.qcow2",
388            "base": "a.qcow2"
389        }
390    }
391
392Once the above ``block-commit`` operation has completed, a
393``BLOCK_JOB_COMPLETED`` event will be issued, and no further action is
394required.  As the end result, the backing file of image [C] is adjusted
395to point to image [A], and the original 4-image chain will end up being
396transformed to::
397
398    [A] <-- [C] <-- [D]
399
400.. note::
401    The intermediate image [B] is invalid (as in: no more further
402    overlays based on it can be created).
403
404    Reasoning: An intermediate image after a 'stream' operation still
405    represents that old point-in-time, and may be valid in that context.
406    However, an intermediate image after a 'commit' operation no longer
407    represents any point-in-time, and is invalid in any context.
408
409
410However, :ref:`Case-3 <block-commit_Case-3>` (also called: "active
411``block-commit``") is a *two-phase* operation: In the first phase, the
412content from the active overlay, along with the intermediate overlays,
413is copied into the backing file (also called the base image).  In the
414second phase, adjust the said backing file as the current active image
415-- possible via issuing the command ``block-job-complete``.  Optionally,
416the ``block-commit`` operation can be cancelled by issuing the command
417``block-job-cancel``, but be careful when doing this.
418
419Once the ``block-commit`` operation has completed, the event
420``BLOCK_JOB_READY`` will be emitted, signalling that the synchronization
421has finished.  Now the job can be gracefully completed by issuing the
422command ``block-job-complete`` -- until such a command is issued, the
423'commit' operation remains active.
424
425The following is the flow for :ref:`Case-3 <block-commit_Case-3>` to
426convert a disk image chain such as this::
427
428    [A] <-- [B] <-- [C] <-- [D]
429
430Into::
431
432    [A]
433
434Where content from all the subsequent overlays, [B], and [C], including
435the active layer, [D], is committed back to [A] -- which is where live
436QEMU is performing all its current writes).
437
438Start the "active ``block-commit``" operation::
439
440    (QEMU) block-commit device=node-D base=a.qcow2 top=d.qcow2 job-id=job0
441    {
442        "execute": "block-commit",
443        "arguments": {
444            "device": "node-D",
445            "job-id": "job0",
446            "top": "d.qcow2",
447            "base": "a.qcow2"
448        }
449    }
450
451
452Once the synchronization has completed, the event ``BLOCK_JOB_READY`` will
453be emitted.
454
455Then, optionally query for the status of the active block operations.
456We can see the 'commit' job is now ready to be completed, as indicated
457by the line *"ready": true*::
458
459    (QEMU) query-block-jobs
460    {
461        "execute": "query-block-jobs",
462        "arguments": {}
463    }
464    {
465        "return": [
466            {
467                "busy": false,
468                "type": "commit",
469                "len": 1376256,
470                "paused": false,
471                "ready": true,
472                "io-status": "ok",
473                "offset": 1376256,
474                "device": "job0",
475                "speed": 0
476            }
477        ]
478    }
479
480Gracefully complete the 'commit' block device job::
481
482    (QEMU) block-job-complete device=job0
483    {
484        "execute": "block-job-complete",
485        "arguments": {
486            "device": "job0"
487        }
488    }
489    {
490        "return": {}
491    }
492
493Finally, once the above job is completed, an event
494``BLOCK_JOB_COMPLETED`` will be emitted.
495
496.. note::
497    The invocation for rest of the cases (2, 4, and 5), discussed in the
498    previous section, is omitted for brevity.
499
500
501Live disk synchronization --- ``drive-mirror`` and ``blockdev-mirror``
502----------------------------------------------------------------------
503
504Synchronize a running disk image chain (all or part of it) to a target
505image.
506
507Again, given our familiar disk image chain::
508
509    [A] <-- [B] <-- [C] <-- [D]
510
511The ``drive-mirror`` (and its newer equivalent ``blockdev-mirror``)
512allows you to copy data from the entire chain into a single target image
513(which can be located on a different host), [E].
514
515.. note::
516
517    When you cancel an in-progress 'mirror' job *before* the source and
518    target are synchronized, ``block-job-cancel`` will emit the event
519    ``BLOCK_JOB_CANCELLED``.  However, note that if you cancel a
520    'mirror' job *after* it has indicated (via the event
521    ``BLOCK_JOB_READY``) that the source and target have reached
522    synchronization, then the event emitted by ``block-job-cancel``
523    changes to ``BLOCK_JOB_COMPLETED``.
524
525    Besides the 'mirror' job, the "active ``block-commit``" is the only
526    other block device job that emits the event ``BLOCK_JOB_READY``.
527    The rest of the block device jobs ('stream', "non-active
528    ``block-commit``", and 'backup') end automatically.
529
530So there are two possible actions to take, after a 'mirror' job has
531emitted the event ``BLOCK_JOB_READY``, indicating that the source and
532target have reached synchronization:
533
534(1) Issuing the command ``block-job-cancel`` (after it emits the event
535    ``BLOCK_JOB_COMPLETED``) will create a point-in-time (which is at
536    the time of *triggering* the cancel command) copy of the entire disk
537    image chain (or only the top-most image, depending on the ``sync``
538    mode), contained in the target image [E]. One use case for this is
539    live VM migration with non-shared storage.
540
541(2) Issuing the command ``block-job-complete`` (after it emits the event
542    ``BLOCK_JOB_COMPLETED``) will adjust the guest device (i.e. live
543    QEMU) to point to the target image, [E], causing all the new writes
544    from this point on to happen there.
545
546About synchronization modes: The synchronization mode determines
547*which* part of the disk image chain will be copied to the target.
548Currently, there are four different kinds:
549
550(1) ``full`` -- Synchronize the content of entire disk image chain to
551    the target
552
553(2) ``top`` -- Synchronize only the contents of the top-most disk image
554    in the chain to the target
555
556(3) ``none`` -- Synchronize only the new writes from this point on.
557
558    .. note:: In the case of ``drive-backup`` (or ``blockdev-backup``),
559              the behavior of ``none`` synchronization mode is different.
560              Normally, a ``backup`` job consists of two parts: Anything
561              that is overwritten by the guest is first copied out to
562              the backup, and in the background the whole image is
563              copied from start to end. With ``sync=none``, it's only
564              the first part.
565
566(4) ``incremental`` -- Synchronize content that is described by the
567    dirty bitmap
568
569.. note::
570    Refer to the :doc:`bitmaps` document in the QEMU source
571    tree to learn about the detailed workings of the ``incremental``
572    synchronization mode.
573
574
575QMP invocation for ``drive-mirror``
576~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
577
578To copy the contents of the entire disk image chain, from [A] all the
579way to [D], to a new target (``drive-mirror`` will create the destination
580file, if it doesn't already exist), call it [E]::
581
582    (QEMU) drive-mirror device=node-D target=e.qcow2 sync=full job-id=job0
583    {
584        "execute": "drive-mirror",
585        "arguments": {
586            "device": "node-D",
587            "job-id": "job0",
588            "target": "e.qcow2",
589            "sync": "full"
590        }
591    }
592
593The ``"sync": "full"``, from the above, means: copy the *entire* chain
594to the destination.
595
596Following the above, querying for active block jobs will show that a
597'mirror' job is "ready" to be completed (and QEMU will also emit an
598event, ``BLOCK_JOB_READY``)::
599
600    (QEMU) query-block-jobs
601    {
602        "execute": "query-block-jobs",
603        "arguments": {}
604    }
605    {
606        "return": [
607            {
608                "busy": false,
609                "type": "mirror",
610                "len": 21757952,
611                "paused": false,
612                "ready": true,
613                "io-status": "ok",
614                "offset": 21757952,
615                "device": "job0",
616                "speed": 0
617            }
618        ]
619    }
620
621And, as noted in the previous section, there are two possible actions
622at this point:
623
624(a) Create a point-in-time snapshot by ending the synchronization.  The
625    point-in-time is at the time of *ending* the sync.  (The result of
626    the following being: the target image, [E], will be populated with
627    content from the entire chain, [A] to [D])::
628
629        (QEMU) block-job-cancel device=job0
630        {
631            "execute": "block-job-cancel",
632            "arguments": {
633                "device": "job0"
634            }
635        }
636
637(b) Or, complete the operation and pivot the live QEMU to the target
638    copy::
639
640        (QEMU) block-job-complete device=job0
641
642In either of the above cases, if you once again run the
643`query-block-jobs` command, there should not be any active block
644operation.
645
646Comparing 'commit' and 'mirror': In both then cases, the overlay images
647can be discarded.  However, with 'commit', the *existing* base image
648will be modified (by updating it with contents from overlays); while in
649the case of 'mirror', a *new* target image is populated with the data
650from the disk image chain.
651
652
653QMP invocation for live storage migration with ``drive-mirror`` + NBD
654~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
655
656Live storage migration (without shared storage setup) is one of the most
657common use-cases that takes advantage of the ``drive-mirror`` primitive
658and QEMU's built-in Network Block Device (NBD) server.  Here's a quick
659walk-through of this setup.
660
661Given the disk image chain::
662
663    [A] <-- [B] <-- [C] <-- [D]
664
665Instead of copying content from the entire chain, synchronize *only* the
666contents of the *top*-most disk image (i.e. the active layer), [D], to a
667target, say, [TargetDisk].
668
669.. important::
670    The destination host must already have the contents of the backing
671    chain, involving images [A], [B], and [C], visible via other means
672    -- whether by ``cp``, ``rsync``, or by some storage array-specific
673    command.)
674
675Sometimes, this is also referred to as "shallow copy" -- because only
676the "active layer", and not the rest of the image chain, is copied to
677the destination.
678
679.. note::
680    In this example, for the sake of simplicity, we'll be using the same
681    ``localhost`` as both source and destination.
682
683As noted earlier, on the destination host the contents of the backing
684chain -- from images [A] to [C] -- are already expected to exist in some
685form (e.g. in a file called, ``Contents-of-A-B-C.qcow2``).  Now, on the
686destination host, let's create a target overlay image (with the image
687``Contents-of-A-B-C.qcow2`` as its backing file), to which the contents
688of image [D] (from the source QEMU) will be mirrored to::
689
690    $ qemu-img create -f qcow2 -b ./Contents-of-A-B-C.qcow2 \
691        -F qcow2 ./target-disk.qcow2
692
693And start the destination QEMU (we already have the source QEMU running
694-- discussed in the section: `Interacting with a QEMU instance`_)
695instance, with the following invocation.  (As noted earlier, for
696simplicity's sake, the destination QEMU is started on the same host, but
697it could be located elsewhere):
698
699.. parsed-literal::
700
701  $ |qemu_system| -display none -no-user-config -nodefaults \\
702    -m 512 -blockdev \\
703    node-name=node-TargetDisk,driver=qcow2,file.driver=file,file.node-name=file,file.filename=./target-disk.qcow2 \\
704    -device virtio-blk,drive=node-TargetDisk,id=virtio0 \\
705    -S -monitor stdio -qmp unix:./qmp-sock2,server=on,wait=off \\
706    -incoming tcp:localhost:6666
707
708Given the disk image chain on source QEMU::
709
710    [A] <-- [B] <-- [C] <-- [D]
711
712On the destination host, it is expected that the contents of the chain
713``[A] <-- [B] <-- [C]`` are *already* present, and therefore copy *only*
714the content of image [D].
715
716(1) [On *destination* QEMU] As part of the first step, start the
717    built-in NBD server on a given host (local host, represented by
718    ``::``)and port::
719
720        (QEMU) nbd-server-start addr={"type":"inet","data":{"host":"::","port":"49153"}}
721        {
722            "execute": "nbd-server-start",
723            "arguments": {
724                "addr": {
725                    "data": {
726                        "host": "::",
727                        "port": "49153"
728                    },
729                    "type": "inet"
730                }
731            }
732        }
733
734(2) [On *destination* QEMU] And export the destination disk image using
735    QEMU's built-in NBD server::
736
737        (QEMU) nbd-server-add device=node-TargetDisk writable=true
738        {
739            "execute": "nbd-server-add",
740            "arguments": {
741                "device": "node-TargetDisk"
742            }
743        }
744
745(3) [On *source* QEMU] Then, invoke ``drive-mirror`` (NB: since we're
746    running ``drive-mirror`` with ``mode=existing`` (meaning:
747    synchronize to a pre-created file, therefore 'existing', file on the
748    target host), with the synchronization mode as 'top' (``"sync:
749    "top"``)::
750
751        (QEMU) drive-mirror device=node-D target=nbd:localhost:49153:exportname=node-TargetDisk sync=top mode=existing job-id=job0
752        {
753            "execute": "drive-mirror",
754            "arguments": {
755                "device": "node-D",
756                "mode": "existing",
757                "job-id": "job0",
758                "target": "nbd:localhost:49153:exportname=node-TargetDisk",
759                "sync": "top"
760            }
761        }
762
763(4) [On *source* QEMU] Once ``drive-mirror`` copies the entire data, and the
764    event ``BLOCK_JOB_READY`` is emitted, issue ``block-job-cancel`` to
765    gracefully end the synchronization, from source QEMU::
766
767        (QEMU) block-job-cancel device=job0
768        {
769            "execute": "block-job-cancel",
770            "arguments": {
771                "device": "job0"
772            }
773        }
774
775(5) [On *destination* QEMU] Then, stop the NBD server::
776
777        (QEMU) nbd-server-stop
778        {
779            "execute": "nbd-server-stop",
780            "arguments": {}
781        }
782
783(6) [On *destination* QEMU] Finally, resume the guest vCPUs by issuing the
784    QMP command `cont`::
785
786        (QEMU) cont
787        {
788            "execute": "cont",
789            "arguments": {}
790        }
791
792.. note::
793    Higher-level libraries (e.g. libvirt) automate the entire above
794    process (although note that libvirt does not allow same-host
795    migrations to localhost for other reasons).
796
797
798Notes on ``blockdev-mirror``
799~~~~~~~~~~~~~~~~~~~~~~~~~~~~
800
801The ``blockdev-mirror`` command is equivalent in core functionality to
802``drive-mirror``, except that it operates at node-level in a BDS graph.
803
804Also: for ``blockdev-mirror``, the 'target' image needs to be explicitly
805created (using ``qemu-img``) and attach it to live QEMU via
806``blockdev-add``, which assigns a name to the to-be created target node.
807
808E.g. the sequence of actions to create a point-in-time backup of an
809entire disk image chain, to a target, using ``blockdev-mirror`` would be:
810
811(0) Create the QCOW2 overlays, to arrive at a backing chain of desired
812    depth
813
814(1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
815
816(2) Attach the above created file (``e.qcow2``), run-time, using
817    ``blockdev-add`` to QEMU
818
819(3) Perform ``blockdev-mirror`` (use ``"sync": "full"`` to copy the
820    entire chain to the target).  And notice the event
821    ``BLOCK_JOB_READY``
822
823(4) Optionally, query for active block jobs, there should be a 'mirror'
824    job ready to be completed
825
826(5) Gracefully complete the 'mirror' block device job, and notice the
827    the event ``BLOCK_JOB_COMPLETED``
828
829(6) Shutdown the guest by issuing the QMP ``quit`` command so that
830    caches are flushed
831
832(7) Then, finally, compare the contents of the disk image chain, and
833    the target copy with ``qemu-img compare``.  You should notice:
834    "Images are identical"
835
836
837QMP invocation for ``blockdev-mirror``
838~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
839
840Given the disk image chain::
841
842    [A] <-- [B] <-- [C] <-- [D]
843
844To copy the contents of the entire disk image chain, from [A] all the
845way to [D], to a new target, call it [E].  The following is the flow.
846
847Create the overlay images, [B], [C], and [D]::
848
849    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
850    (QEMU) blockdev-snapshot-sync node-name=node-B snapshot-file=c.qcow2 snapshot-node-name=node-C format=qcow2
851    (QEMU) blockdev-snapshot-sync node-name=node-C snapshot-file=d.qcow2 snapshot-node-name=node-D format=qcow2
852
853Create the target image, [E]::
854
855    $ qemu-img create -f qcow2 e.qcow2 39M
856
857Add the above created target image to QEMU, via ``blockdev-add``::
858
859    (QEMU) blockdev-add driver=qcow2 node-name=node-E file={"driver":"file","filename":"e.qcow2"}
860    {
861        "execute": "blockdev-add",
862        "arguments": {
863            "node-name": "node-E",
864            "driver": "qcow2",
865            "file": {
866                "driver": "file",
867                "filename": "e.qcow2"
868            }
869        }
870    }
871
872Perform ``blockdev-mirror``, and notice the event ``BLOCK_JOB_READY``::
873
874    (QEMU) blockdev-mirror device=node-B target=node-E sync=full job-id=job0
875    {
876        "execute": "blockdev-mirror",
877        "arguments": {
878            "device": "node-D",
879            "job-id": "job0",
880            "target": "node-E",
881            "sync": "full"
882        }
883    }
884
885Query for active block jobs, there should be a 'mirror' job ready::
886
887    (QEMU) query-block-jobs
888    {
889        "execute": "query-block-jobs",
890        "arguments": {}
891    }
892    {
893        "return": [
894            {
895                "busy": false,
896                "type": "mirror",
897                "len": 21561344,
898                "paused": false,
899                "ready": true,
900                "io-status": "ok",
901                "offset": 21561344,
902                "device": "job0",
903                "speed": 0
904            }
905        ]
906    }
907
908Gracefully complete the block device job operation, and notice the
909event ``BLOCK_JOB_COMPLETED``::
910
911    (QEMU) block-job-complete device=job0
912    {
913        "execute": "block-job-complete",
914        "arguments": {
915            "device": "job0"
916        }
917    }
918    {
919        "return": {}
920    }
921
922Shutdown the guest, by issuing the ``quit`` QMP command::
923
924    (QEMU) quit
925    {
926        "execute": "quit",
927        "arguments": {}
928    }
929
930
931Live disk backup --- ``drive-backup`` and ``blockdev-backup``
932-------------------------------------------------------------
933
934The ``drive-backup`` (and its newer equivalent ``blockdev-backup``) allows
935you to create a point-in-time snapshot.
936
937In this case, the point-in-time is when you *start* the ``drive-backup``
938(or its newer equivalent ``blockdev-backup``) command.
939
940
941QMP invocation for ``drive-backup``
942~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
943
944Yet again, starting afresh with our example disk image chain::
945
946    [A] <-- [B] <-- [C] <-- [D]
947
948To create a target image [E], with content populated from image [A] to
949[D], from the above chain, the following is the syntax.  (If the target
950image does not exist, ``drive-backup`` will create it)::
951
952    (QEMU) drive-backup device=node-D sync=full target=e.qcow2 job-id=job0
953    {
954        "execute": "drive-backup",
955        "arguments": {
956            "device": "node-D",
957            "job-id": "job0",
958            "sync": "full",
959            "target": "e.qcow2"
960        }
961    }
962
963Once the above ``drive-backup`` has completed, a ``BLOCK_JOB_COMPLETED`` event
964will be issued, indicating the live block device job operation has
965completed, and no further action is required.
966
967
968Notes on ``blockdev-backup``
969~~~~~~~~~~~~~~~~~~~~~~~~~~~~
970
971The ``blockdev-backup`` command is equivalent in functionality to
972``drive-backup``, except that it operates at node-level in a Block Driver
973State (BDS) graph.
974
975E.g. the sequence of actions to create a point-in-time backup
976of an entire disk image chain, to a target, using ``blockdev-backup``
977would be:
978
979(0) Create the QCOW2 overlays, to arrive at a backing chain of desired
980    depth
981
982(1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
983
984(2) Attach the above created file (``e.qcow2``), run-time, using
985    ``blockdev-add`` to QEMU
986
987(3) Perform ``blockdev-backup`` (use ``"sync": "full"`` to copy the
988    entire chain to the target).  And notice the event
989    ``BLOCK_JOB_COMPLETED``
990
991(4) Shutdown the guest, by issuing the QMP ``quit`` command, so that
992    caches are flushed
993
994(5) Then, finally, compare the contents of the disk image chain, and
995    the target copy with ``qemu-img compare``.  You should notice:
996    "Images are identical"
997
998The following section shows an example QMP invocation for
999``blockdev-backup``.
1000
1001QMP invocation for ``blockdev-backup``
1002~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1003
1004Given a disk image chain of depth 1 where image [B] is the active
1005overlay (live QEMU is writing to it)::
1006
1007    [A] <-- [B]
1008
1009The following is the procedure to copy the content from the entire chain
1010to a target image (say, [E]), which has the full content from [A] and
1011[B].
1012
1013Create the overlay [B]::
1014
1015    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
1016    {
1017        "execute": "blockdev-snapshot-sync",
1018        "arguments": {
1019            "node-name": "node-A",
1020            "snapshot-file": "b.qcow2",
1021            "format": "qcow2",
1022            "snapshot-node-name": "node-B"
1023        }
1024    }
1025
1026
1027Create a target image that will contain the copy::
1028
1029    $ qemu-img create -f qcow2 e.qcow2 39M
1030
1031Then add it to QEMU via ``blockdev-add``::
1032
1033    (QEMU) blockdev-add driver=qcow2 node-name=node-E file={"driver":"file","filename":"e.qcow2"}
1034    {
1035        "execute": "blockdev-add",
1036        "arguments": {
1037            "node-name": "node-E",
1038            "driver": "qcow2",
1039            "file": {
1040                "driver": "file",
1041                "filename": "e.qcow2"
1042            }
1043        }
1044    }
1045
1046Then invoke ``blockdev-backup`` to copy the contents from the entire
1047image chain, consisting of images [A] and [B] to the target image
1048'e.qcow2'::
1049
1050    (QEMU) blockdev-backup device=node-B target=node-E sync=full job-id=job0
1051    {
1052        "execute": "blockdev-backup",
1053        "arguments": {
1054            "device": "node-B",
1055            "job-id": "job0",
1056            "target": "node-E",
1057            "sync": "full"
1058        }
1059    }
1060
1061Once the above 'backup' operation has completed, the event,
1062``BLOCK_JOB_COMPLETED`` will be emitted, signalling successful
1063completion.
1064
1065Next, query for any active block device jobs (there should be none)::
1066
1067    (QEMU) query-block-jobs
1068    {
1069        "execute": "query-block-jobs",
1070        "arguments": {}
1071    }
1072
1073Shutdown the guest::
1074
1075    (QEMU) quit
1076    {
1077            "execute": "quit",
1078                "arguments": {}
1079    }
1080            "return": {}
1081    }
1082
1083.. note::
1084    The above step is really important; if forgotten, an error, "Failed
1085    to get shared "write" lock on e.qcow2", will be thrown when you do
1086    ``qemu-img compare`` to verify the integrity of the disk image
1087    with the backup content.
1088
1089
1090The end result will be the image 'e.qcow2' containing a
1091point-in-time backup of the disk image chain -- i.e. contents from
1092images [A] and [B] at the time the ``blockdev-backup`` command was
1093initiated.
1094
1095One way to confirm the backup disk image contains the identical content
1096with the disk image chain is to compare the backup and the contents of
1097the chain, you should see "Images are identical".  (NB: this is assuming
1098QEMU was launched with ``-S`` option, which will not start the CPUs at
1099guest boot up)::
1100
1101    $ qemu-img compare b.qcow2 e.qcow2
1102    Warning: Image size mismatch!
1103    Images are identical.
1104
1105NOTE: The "Warning: Image size mismatch!" is expected, as we created the
1106target image (e.qcow2) with 39M size.
1107