blockdev: Split monitor reference from BB creation

Before this patch, blk_new() automatically assigned a name to the new
BlockBackend and considered it referenced by the monitor. This patch
removes

blockdev: Split monitor reference from BB creation

Before this patch, blk_new() automatically assigned a name to the new
BlockBackend and considered it referenced by the monitor. This patch
removes the implicit monitor_add_blk() call from blk_new() (and
consequently the monitor_remove_blk() call from blk_delete(), too) and
thus blk_new() (and related functions) no longer take a BB name
argument.

In fact, there is only a single point where blk_new()/blk_new_open() is
called and the new BB is monitor-owned, and that is in blockdev_init().
Besides thus relieving us from having to invent names for all of the BBs
we use in qemu-img, this fixes a bug where qemu cannot create a new
image if there already is a monitor-owned BB named "image".

If a BB and its BDS tree are created in a single operation, as of this
patch the BDS tree will be created before the BB is given a name
(whereas it was the other way around before). This results in minor
change to the output of iotest 087, whose reference output is amended
accordingly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-io: use no_argument/required_argument constants

When declaring the 'struct option' array, use the standard
constants no_argument/required_argument, instead of magic
values 0 and 1.

Reviewed-by

qemu-io: use no_argument/required_argument constants

When declaring the 'struct option' array, use the standard
constants no_argument/required_argument, instead of magic
values 0 and 1.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-io: allow specifying image as a set of options args

Currently qemu-io allows an image filename to be passed on the
command line, but unless using the JSON format, it does not have
a way to set

qemu-io: allow specifying image as a set of options args

Currently qemu-io allows an image filename to be passed on the
command line, but unless using the JSON format, it does not have
a way to set any options except the format eg

qemu-io https://127.0.0.1/images/centos7.iso
qemu-io /home/berrange/demo.qcow2

By contrast when using the interactive shell, it is possible to
use --option with the 'open' command, or to omit the filename.

This adds a --image-opts arg that indicates that the positional
filename should be interpreted as a full option string, not
just a filename.

qemu-io --image-opts driver=https,url=https://127.0.0.1/images,sslverify=off
qemu-io --image-opts driver=qcow2,file.filename=/home/berrange/demo.qcow2

This flag is mutually exclusive with the '-f' flag and with
the '-o' flag to the 'open' command

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-io: add support for --object command line arg

Allow creation of user creatable object types with qemu-io
via a new --object command line arg. This will be used to supply
passwords and/or encryp

qemu-io: add support for --object command line arg

Allow creation of user creatable object types with qemu-io
via a new --object command line arg. This will be used to supply
passwords and/or encryption keys to the various block driver
backends via the recently added 'secret' object type.

# printf letmein > mypasswd.txt
# qemu-io --object secret,id=sec0,file=mypasswd.txt \
...other args...

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

nbd: convert block client to use I/O channels for connection setup

This converts the NBD block driver client to use the QIOChannelSocket
class for initial connection setup. The NbdClientSession stru

nbd: convert block client to use I/O channels for connection setup

This converts the NBD block driver client to use the QIOChannelSocket
class for initial connection setup. The NbdClientSession struct has
two pointers, one to the master QIOChannelSocket providing the raw
data channel, and one to a QIOChannel which is the current channel
used for I/O. Initially the two point to the same object, but when
TLS support is added, they will point to different objects.

The qemu-img & qemu-io tools now need to use MODULE_INIT_QOM to
ensure the QIOChannel object classes are registered. The qemu-nbd
tool already did this.

In this initial conversion though, all I/O is still actually done
using the raw POSIX sockets APIs.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1455129674-17255-4-git-send-email-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

trace: split trace_init_file out of trace_init_backends

This is cleaner, and improves error reporting with -daemonize.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Denis V. Lun

trace: split trace_init_file out of trace_init_backends

This is cleaner, and improves error reporting with -daemonize.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Message-id: 1452174932-28657-4-git-send-email-den@openvz.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

trace: split trace_init_events out of trace_init_backends

This is cleaner and has two advantages. First, it improves error
reporting with -daemonize. Second, multiple "-trace events" options
now c

trace: split trace_init_events out of trace_init_backends

This is cleaner and has two advantages. First, it improves error
reporting with -daemonize. Second, multiple "-trace events" options
now cumulate.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Message-id: 1452174932-28657-3-git-send-email-den@openvz.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: Clean up includes

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

Signed-off

block: Clean up includes

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-io qemu-nbd: Use error_report() etc. instead of fprintf()

Just three instances left.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id

qemu-io qemu-nbd: Use error_report() etc. instead of fprintf()

Just three instances left.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1450452927-8346-16-git-send-email-armbru@redhat.com>

show more ...

qemu-io: Remove duplicate 'open' error message

qemu_opts_parse_noisily() already prints an error message with the exact
reason why the parsing failed. No need to add another less specific one.

Sign

qemu-io: Remove duplicate 'open' error message

qemu_opts_parse_noisily() already prints an error message with the exact
reason why the parsing failed. No need to add another less specific one.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

show more ...

qerror: Move #include out of qerror.h

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by:

qerror: Move #include out of qerror.h

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...

QemuOpts: Wean off qerror_report_err()

qerror_report_err() is a transitional interface to help with
converting existing monitor commands to QMP. It should not be used
elsewhere.

The only remaining

QemuOpts: Wean off qerror_report_err()

qerror_report_err() is a transitional interface to help with
converting existing monitor commands to QMP. It should not be used
elsewhere.

The only remaining user in qemu-option.c is qemu_opts_parse(). Is it
used in QMP context? If not, we can simply replace
qerror_report_err() by error_report_err().

The uses in qemu-img.c, qemu-io.c, qemu-nbd.c and under tests/ are
clearly not in QMP context.

The uses in vl.c aren't either, because the only QMP command handlers
there are qmp_query_status() and qmp_query_machines(), and they don't
call it.

Remaining uses:

* drive_def(): Command line -drive and such, HMP drive_add and pci_add

* hmp_chardev_add(): HMP chardev-add

* monitor_parse_command(): HMP core

* tmp_config_parse(): Command line -tpmdev

* net_host_device_add(): HMP host_net_add

* net_client_parse(): Command line -net and -netdev

* qemu_global_option(): Command line -global

* vnc_parse_func(): Command line -display, -vnc, default display, HMP
change, QMP change. Bummer.

* qemu_pci_hot_add_nic(): HMP pci_add

* usb_net_init(): Command line -usbdevice, HMP usb_add

Propagate errors through qemu_opts_parse(). Create a convenience
function qemu_opts_parse_noisily() that passes errors to
error_report_err(). Switch all non-QMP users outside tests to it.

That leaves vnc_parse_func(). Propagate errors through it. Since I'm
touching it anyway, rename it to vnc_parse().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...

qemu-io: prompt for encryption keys when required

The qemu-io tool does not check if the image is encrypted so
historically would silently corrupt the sectors by writing
plain text data into them in

qemu-io: prompt for encryption keys when required

The qemu-io tool does not check if the image is encrypted so
historically would silently corrupt the sectors by writing
plain text data into them instead of cipher text. The earlier
commit turns this mistake into a fatal abort, so check for
encryption and prompt for key when required.

This enables us to add unit tests to ensure we don't break
the ability of qemu-img to convert existing encrypted qcow2
files into a non-encrypted format.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-io: Use getopt() correctly

POSIX says getopt() returns -1 on completion. While Linux happens
to define EOF as -1, this definition is not required by POSIX, and
there may be platforms where che

qemu-io: Use getopt() correctly

POSIX says getopt() returns -1 on completion. While Linux happens
to define EOF as -1, this definition is not required by POSIX, and
there may be platforms where checking for EOF instead of -1 would
lead to an infinite loop.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

error: Use error_report_err() where appropriate

Coccinelle semantic patch:

@@
expression E;
@@
- error_report("%s", error_get_pretty(E));
- error_free(E);
+ error_r

error: Use error_report_err() where appropriate

Coccinelle semantic patch:

@@
expression E;
@@
- error_report("%s", error_get_pretty(E));
- error_free(E);
+ error_report_err(E);
@@
expression E, S;
@@
- error_report("%s", error_get_pretty(E));
+ error_report_err(E);
(
exit(S);
|
abort();
)

Trivial manual touch-ups in block/sheepdog.c.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

qemu-io: Use BlockBackend

qemu-io should behave like a guest, therefore it should use BlockBackend
to access the block layer.

There are a couple of places where that is infeasible: First, the
bdrv_

qemu-io: Use BlockBackend

qemu-io should behave like a guest, therefore it should use BlockBackend
to access the block layer.

There are a couple of places where that is infeasible: First, the
bdrv_debug_* functions could theoretically be mirrored in the
BlockBackend, but since these are functions internal to the block layer,
they should not be visible externally (qemu-io as a test tool is exempt
from this).

Second, bdrv_get_info() and bdrv_get_specific_info() work on a single
BDS alone, therefore they should stay BDS-specific.

Third, bdrv_is_allocated() mainly works on a single BDS as well. Some
data may be passed through from the BDS's file (if sectors which are
apparently allocated in the file are not really allocated there but just
zero).

[Fixed conflicts around block_acct_start() usage from Fam Zheng's
"qemu-io: Account IO by aio_read and aio_write" commit. Use
BlockBackend and blk_get_stats() instead of BlockDriverState.
--Stefan]

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-14-git-send-email-mreitz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

qemu-io: Remove "growable" option

Remove "growable" option from the "open" command and from the qemu-io
command line. qemu-io is about to be converted to BlockBackend which
will make sure that no re

qemu-io: Remove "growable" option

Remove "growable" option from the "open" command and from the qemu-io
command line. qemu-io is about to be converted to BlockBackend which
will make sure that no request exceeds the image size, so the only way
to keep "growable" would be to use BlockBackend if it is not given and
to directly access the BDS if it is.

qemu-io is a debugging tool, therefore removing a rarely used option
will have only a very small impact, if any. There was only one
qemu-iotest which used the option; since it is not critical, this patch
just removes it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-13-git-send-email-mreitz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

qemu-io: Use blk_new_open() in openfile()

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-12-git-send-email-mreitz@redhat.com
Si

qemu-io: Use blk_new_open() in openfile()

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-12-git-send-email-mreitz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

qemu-io: Allow explicitly specifying format

This adds a -f option to qemu-io which allows to explicitly specify the
block driver to use for the given image.

Signed-off-by: Kevin Wolf <kwolf@redhat.

qemu-io: Allow explicitly specifying format

This adds a -f option to qemu-io which allows to explicitly specify the
block driver to use for the given image.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1416497234-29880-2-git-send-email-kwolf@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Make BlockBackend own its BlockDriverState

On BlockBackend destruction, unref its BlockDriverState. Replaces the
callers' unrefs.

This turns the pointer from BlockBackend to BlockDriverStat

block: Make BlockBackend own its BlockDriverState

On BlockBackend destruction, unref its BlockDriverState. Replaces the
callers' unrefs.

This turns the pointer from BlockBackend to BlockDriverState into a
strong reference, managed with bdrv_ref() / bdrv_unref(). The
back-pointer remains weak.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Connect BlockBackend to BlockDriverState

Convenience function blk_new_with_bs() creates a BlockBackend with its
BlockDriverState. Callers have to unref both. The commit after next
will reli

block: Connect BlockBackend to BlockDriverState

Convenience function blk_new_with_bs() creates a BlockBackend with its
BlockDriverState. Callers have to unref both. The commit after next
will relieve them of the need to unref the BlockDriverState.

Complication: due to the silly way drive_del works, we need a way to
hide a BlockBackend, just like bdrv_make_anon(). To emphasize its
"special" status, give the function a suitably off-putting name:
blk_hide_on_behalf_of_do_drive_del(). Unfortunately, hiding turns the
BlockBackend's name into the empty string. Can't avoid that without
breaking the blk->bs->device_name equals blk->name invariant.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: New BlockBackend

A block device consists of a frontend device model and a backend.

A block backend has a tree of block drivers doing the actual work.
The tree is managed by the block layer.

block: New BlockBackend

A block device consists of a frontend device model and a backend.

A block backend has a tree of block drivers doing the actual work.
The tree is managed by the block layer.

We currently use a single abstraction BlockDriverState both for tree
nodes and the backend as a whole. Drawbacks:

* Its API includes both stuff that makes sense only at the block
backend level (root of the tree) and stuff that's only for use
within the block layer. This makes the API bigger and more complex
than necessary. Moreover, it's not obvious which interfaces are
meant for device models, and which really aren't.

* Since device models keep a reference to their backend, the backend
object can't just be destroyed. But for media change, we need to
replace the tree. Our solution is to make the BlockDriverState
generic, with actual driver state in a separate object, pointed to
by member opaque. That lets us replace the tree by deinitializing
and reinitializing its root. This special need of the root makes
the data structure awkward everywhere in the tree.

The general plan is to separate the APIs into "block backend", for use
by device models, monitor and whatever other code dealing with block
backends, and "block driver", for use by the block layer and whatever
other code (if any) dealing with trees and tree nodes.

Code dealing with block backends, device models in particular, should
become completely oblivious of BlockDriverState. This should let us
clean up both APIs, and the tree data structures.

This commit is a first step. It creates a minimal "block backend"
API: type BlockBackend and functions to create, destroy and find them.

BlockBackend objects are created and destroyed exactly when root
BlockDriverState objects are created and destroyed. "Root" in the
sense of "in bdrv_states". They're not yet used for anything; that'll
come shortly.

A root BlockDriverState is created with bdrv_new_root(), so where to
create a BlockBackend is obvious. Where these roots get destroyed
isn't always as obvious.

It is obvious in qemu-img.c, qemu-io.c and qemu-nbd.c, and in error
paths of blockdev_init(), blk_connect(). That leaves destruction of
objects successfully created by blockdev_init() and blk_connect().

blockdev_init() is used only by drive_new() and qmp_blockdev_add().
Objects created by the latter are currently indestructible (see commit
48f364d "blockdev: Refuse to drive_del something added with
blockdev-add" and commit 2d246f0 "blockdev: Introduce
DriveInfo.enable_auto_del"). Objects created by the former get
destroyed by drive_del().

Objects created by blk_connect() get destroyed by blk_disconnect().

BlockBackend is reference-counted. Its reference count never exceeds
one so far, but that's going to change.

In drive_del(), the BB's reference count is surely one now. The BDS's
reference count is greater than one when something else is holding a
reference, such as a block job. In this case, the BB is destroyed
right away, but the BDS lives on until all extra references get
dropped.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Split bdrv_new_root() off bdrv_new()

Creating an anonymous BDS can't fail. Make that obvious.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

block: Split bdrv_new_root() off bdrv_new()

Creating an anonymous BDS can't fail. Make that obvious.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Benoît Canet <benoit.canet@nodalink.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

async: aio_context_new(): Handle event_notifier_init failure

On a system with a low limit of open files the initialization
of the event notifier could fail and QEMU exits without printing any
error

async: aio_context_new(): Handle event_notifier_init failure

On a system with a low limit of open files the initialization
of the event notifier could fail and QEMU exits without printing any
error information to the user.

The problem can be easily reproduced by enforcing a low limit of open
files and start QEMU with enough I/O threads to hit this limit.

The same problem raises, without the creation of I/O threads, while
QEMU initializes the main event loop by enforcing an even lower limit of
open files.

This commit adds an error message on failure:

# qemu [...] -object iothread,id=iothread0 -object iothread,id=iothread1
qemu: Failed to initialize event notifier: Too many open files in system

Signed-off-by: Chrysostomos Nanakos <cnanakos@grnet.gr>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

qemu-io: Clean up openfile() after commit 2e40134

Commit 6db9560 split off the growable case so it can use
bdrv_file_open() instead of bdrv_open() then. Growable BDSes become
anonymous. Weird.

Co

qemu-io: Clean up openfile() after commit 2e40134

Commit 6db9560 split off the growable case so it can use
bdrv_file_open() instead of bdrv_open() then. Growable BDSes become
anonymous. Weird.

Commit 2e40134 folded bdrv_file_open() back into bdrv_open() with new
flag BDRV_O_PROTOCOL. We still have two bdrv_open() calls, and
growable BDSes remain anonymous.

Circle back to before commit 6db9560: just one call, not anonymous.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...