scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it defa

scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it default to 'REPLY' is a bashism. If your system
sh is dash, then it will print an error message during build:

qemu/pc-bios/s390-ccw/../../scripts/git-submodule.sh: 106: read: arg count

Specify the variable name explicitly.

Fixes: fdb8fd8cb915647b ("git-submodule: allow partial update of .git-submodule-status")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20230720153038.1587196-1-peter.maydell@linaro.org

show more ...

scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it defa

scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it default to 'REPLY' is a bashism. If your system
sh is dash, then it will print an error message during build:

qemu/pc-bios/s390-ccw/../../scripts/git-submodule.sh: 106: read: arg count

Specify the variable name explicitly.

Fixes: fdb8fd8cb915647b ("git-submodule: allow partial update of .git-submodule-status")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20230720153038.1587196-1-peter.maydell@linaro.org

show more ...

scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it defa

scripts/git-submodule.sh: Don't rely on non-POSIX 'read' behaviour

The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it default to 'REPLY' is a bashism. If your system
sh is dash, then it will print an error message during build:

qemu/pc-bios/s390-ccw/../../scripts/git-submodule.sh: 106: read: arg count

Specify the variable name explicitly.

Fixes: fdb8fd8cb915647b ("git-submodule: allow partial update of .git-submodule-status")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20230720153038.1587196-1-peter.maydell@linaro.org

show more ...

git-submodule.sh: allow running in validate mode without previous update

The call to git-submodule.sh done in configure may happen without a
previous checkout of the roms/SLOF submodule, or even wit

git-submodule.sh: allow running in validate mode without previous update

The call to git-submodule.sh done in configure may happen without a
previous checkout of the roms/SLOF submodule, or even without a
previous run of the script.

So, handle creating a .git-submodule-status file even in validate
mode. If git is absent, ensure that all passed directories exists
(because you should be in a fresh untar and will not have stale
arguments to git-submodule.sh) but do no other checks. If git
is present, ensure that .git-submodule-status contains an entry
for all submodules passed on the command line.

With this change, "ignore" mode is not needed anymore.

Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Fixes: b11f9bd96f4 ("configure: move SLOF submodule handling to pc-bios/s390-ccw", 2023-06-06)
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: remove --with-git-submodules=

Reuse --enable/--disable-download to control git submodules as well.
Adjust the error messages of git-submodule.sh to refer to the new
option.

Reviewed-by:

configure: remove --with-git-submodules=

Reuse --enable/--disable-download to control git submodules as well.
Adjust the error messages of git-submodule.sh to refer to the new
option.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: log submodule update from git-submodule.sh

Print exactly which submodules have been updated, by reusing the logic of
"git-submodule.sh validate" after executing "git submodule update --init'.

build: log submodule update from git-submodule.sh

Print exactly which submodules have been updated, by reusing the logic of
"git-submodule.sh validate" after executing "git submodule update --init'.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

git-submodule: allow partial update of .git-submodule-status

Allow a specific subdirectory to run git-submodule.sh with only a
subset of submodules, without removing the others from the
.git-submodu

git-submodule: allow partial update of .git-submodule-status

Allow a specific subdirectory to run git-submodule.sh with only a
subset of submodules, without removing the others from the
.git-submodule-status file.

This also allows scripts/git-submodule.sh to be more lenient:
validating an empty set of submodules is not a mistake.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: remove --with-git= option

The scenario for which --with-git= was introduced was to use a SOCKS proxy
such as tsocks. However, this was back in 2017 when QEMU's submodules
used the git://

configure: remove --with-git= option

The scenario for which --with-git= was introduced was to use a SOCKS proxy
such as tsocks. However, this was back in 2017 when QEMU's submodules
used the git:// protocol, and it is not as important when using the
"smart HTTP" backend; for example, neither "meson subprojects download"
nor scripts/checkpatch.pl obey the GIT environment variable.

So remove the knob, but test for the presence of git in the configure and
git-submodule.sh scripts, and suggest using --with-git-submodules=validate
+ a manual invocation of git-submodule.sh when git does not work. Hopefully
in the future the GIT environment variable will be supported by Meson.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

scripts: check if .git exists before checking submodule status

Currently we check status of each submodule, before actually checking
if we're in a git repo. These status commands will all fail, but

scripts: check if .git exists before checking submodule status

Currently we check status of each submodule, before actually checking
if we're in a git repo. These status commands will all fail, but we
are hiding their output so we don't see it currently.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

scripts: check if .git exists before checking submodule status

Currently we check status of each submodule, before actually checking
if we're in a git repo. These status commands will all fail, but

scripts: check if .git exists before checking submodule status

Currently we check status of each submodule, before actually checking
if we're in a git repo. These status commands will all fail, but we
are hiding their output so we don't see it currently.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

configure: replace --enable/disable-git-update with --with-git-submodules

Replace the --enable-git-update and --disable-git-update configure params
with the param --with-git-submodules=(update|valid

configure: replace --enable/disable-git-update with --with-git-submodules

Replace the --enable-git-update and --disable-git-update configure params
with the param --with-git-submodules=(update|validate|ignore) to
allow 3 options for building from a git repo.

This is needed because downstream packagers, e.g. Debian, Ubuntu, etc,
also keep the source code in git, but do not want to enable the
'git_update' mode; with the current code, that's not possible even
if the downstream package specifies --disable-git-update.

The previous parameters are deprecated but still available; the
--enable-git-update parameter maps to --with-git-submodules=update and
--disable-git-update parameter maps to --with-git-submodules=validate.

The configure script behavior is slightly modified, where previously
the dtc, capstone, and slirp submodules were not validated when
--disable-git-update was specified (but were updated with git-update
enabled), now they are validated when using --with-git-submodules=validate
and are only ignored when using --with-git-submodules=ignore.

Signed-off-by: Dan Streetman <ddstreet@canonical.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

git: Make submodule check only needed modules

If one is compiling more than one tree from the same source, it is
possible that they need different submodules. Change the check to see
that all modul

git: Make submodule check only needed modules

If one is compiling more than one tree from the same source, it is
possible that they need different submodules. Change the check to see
that all modules that we are interested in are updated, discarding the
ones that we don't care about.

Signed-off-by: Juan Quintela <quintela@redhat.com>

---

v1->v2:
patchw insists in not using modules

show more ...

git-submodule.sh: Modern shell scripting (use $() instead of ``)

Various shell files contain a mix between obsolete ``
and modern $(); It would be nice to convert to using $()
everywhere.

Signed-of

git-submodule.sh: Modern shell scripting (use $() instead of ``)

Various shell files contain a mix between obsolete ``
and modern $(); It would be nice to convert to using $()
everywhere.

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

build: fix typo in error message

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Fixes: f62bb

build: fix typo in error message

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Fixes: f62bbee55d503f639ee9498878ebf42ff4f4299a

show more ...

build: fix typo in error message

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Michael Tokare

build: fix typo in error message

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>

show more ...

build: delay check for empty git submodule list

We short circuit the git submodule update when passed an empty module list.
This accidentally causes the 'status' command to write to the status file.

build: delay check for empty git submodule list

We short circuit the git submodule update when passed an empty module list.
This accidentally causes the 'status' command to write to the status file. The
test needs to be delayed into the individual commands to avoid this premature
writing of the status file.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

show more ...

build: don't fail if given a git submodule which does not exist

If going back in time in git history, across a commit that introduces a new
submodule, the 'git-submodule.sh' script will fail, causin

build: don't fail if given a git submodule which does not exist

If going back in time in git history, across a commit that introduces a new
submodule, the 'git-submodule.sh' script will fail, causing rebuild to fail.

This is because config-host.mak contains a GIT_SUBMODULES variable that lists
a submodule that only exists in the later commit. config-host.mak won't get
repopulated until config.status is invoked, but make won't get this far due to
the submodule error.

This change makes 'git-submodule.sh' check whether each module is known to git
and drops any which are not present. A warning message will be printed when any
submodule is dropped in this manner.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

show more ...

build: allow automatic git submodule updates to be disabled

Some people building QEMU use VPATH builds where the source directory is on a
read-only volume. In such a case 'scripts/git-submodules.sh

build: allow automatic git submodule updates to be disabled

Some people building QEMU use VPATH builds where the source directory is on a
read-only volume. In such a case 'scripts/git-submodules.sh update' will always
fail and users are required to run it manually themselves on their original
writable source directory.

While this is already supported, it is nice to give users a command line flag
to configure to permanently disable automatic submodule updates, as it means
they won't get hard to diagnose failures from git-submodules.sh at an arbitrary
later date.

This patch thus introduces a flag '--disable-git-update' which will prevent
'make' from ever running 'scripts/git-submodules.sh update'. It will still run
the 'status' command to determine if a submodule update is needed, but when it
does this it'll simply stop and print a message instructing the developer what
todo. eg

$ ./configure --target-list=x86_64-softmmu --disable-git-update
...snip...

$ make
GEN config-host.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h

GIT submodule checkout is out of date. Please run
scripts/git-submodule.sh update ui/keycodemapdb
from the source directory checkout /home/berrange/src/virt/qemu

make: *** [Makefile:31: git-submodule-update] Error 1

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

show more ...

build: don't create temporary files in source dir

There are cases where users do VPATH builds with the source directory being on
a read-only volume. In such a case they have to manually run the comm

build: don't create temporary files in source dir

There are cases where users do VPATH builds with the source directory being on
a read-only volume. In such a case they have to manually run the command
'git-submodule.sh ...modules...' ahead of time. When checking for status we
should not then write into the source dir.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

show more ...

build: allow setting a custom GIT binary for transparent proxying

Some users can't run a bare 'git' command, due to need for a transparent
proxying solution such as 'tsocks'. This adds an argument t

build: allow setting a custom GIT binary for transparent proxying

Some users can't run a bare 'git' command, due to need for a transparent
proxying solution such as 'tsocks'. This adds an argument to configure to
let users specify such a thing:

./configure --with-git="tsocks git"

The submodule script is also updated to give the user a hint about using this
flag, if we fail to checkout modules.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

show more ...

scripts: don't throw away stderr when checking out git submodules

The stderr from git is important if git fails to checkout modules
due to network problems, or other unexpected errors.

Signed-off-b

scripts: don't throw away stderr when checking out git submodules

The stderr from git is important if git fails to checkout modules
due to network problems, or other unexpected errors.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20171020130748.22983-1-berrange@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

show more ...

build: automatically handle GIT submodule checkout for dtc

Currently if DTC is required by configure and not available in the host
OS install, we exit with an error message telling the user to check

build: automatically handle GIT submodule checkout for dtc

Currently if DTC is required by configure and not available in the host
OS install, we exit with an error message telling the user to checkout a
git submodule or install the library.

This introduces automatic handling of the git submodule checkout process
and enables it for dtc. This only runs if building from GIT, so users of
release tarballs still need the system library install. The current state
of the git checkout is stashed in .git-submodule-status, and a helper
program is used to determine if this state matches the desired submodule
state. A dependency against 'Makefile' ensures that the submodule state
is refreshed at the start of the build process

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 20170929101201.21039-2-berrange@redhat.com

[ kraxel: use /bin/sh not bash for scripts/git-submodule.sh ]
[ kraxel: fix Makefile dependencies ]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

[fixup] Makefile dep

show more ...