s390x/pci: only limit DMA aperture if vfio DMA limit reported

If the host kernel lacks vfio DMA limit reporting, do not attempt
to shrink the guest DMA aperture.

Fixes: df202e3ff3 ("s390x/pci: shri

s390x/pci: only limit DMA aperture if vfio DMA limit reported

If the host kernel lacks vfio DMA limit reporting, do not attempt
to shrink the guest DMA aperture.

Fixes: df202e3ff3 ("s390x/pci: shrink DMA aperture to be bound by vfio DMA limit")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20231110175108.465851-3-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: bypass vfio DMA counting when using cdev

The current code assumes that there is always a vfio group, but
that's no longer guaranteed with the iommufd backend when using
cdev. In this cas

s390x/pci: bypass vfio DMA counting when using cdev

The current code assumes that there is always a vfio group, but
that's no longer guaranteed with the iommufd backend when using
cdev. In this case, we don't need to track the vfio dma limit
anyway.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20231110175108.465851-2-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: only limit DMA aperture if vfio DMA limit reported

If the host kernel lacks vfio DMA limit reporting, do not attempt
to shrink the guest DMA aperture.

Fixes: df202e3ff3 ("s390x/pci: shri

s390x/pci: only limit DMA aperture if vfio DMA limit reported

If the host kernel lacks vfio DMA limit reporting, do not attempt
to shrink the guest DMA aperture.

Fixes: df202e3ff3 ("s390x/pci: shrink DMA aperture to be bound by vfio DMA limit")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20231110175108.465851-3-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: bypass vfio DMA counting when using cdev

The current code assumes that there is always a vfio group, but
that's no longer guaranteed with the iommufd backend when using
cdev. In this cas

s390x/pci: bypass vfio DMA counting when using cdev

The current code assumes that there is always a vfio group, but
that's no longer guaranteed with the iommufd backend when using
cdev. In this case, we don't need to track the vfio dma limit
anyway.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20231110175108.465851-2-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

vfio: Implement a common device info helper

A common helper implementing the realloc algorithm for handling
capabilities.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Cédric

vfio: Implement a common device info helper

A common helper implementing the realloc algorithm for handling
capabilities.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Robin Voetter <robin@streamhpc.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>

show more ...

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we g

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we get a termination signal for QEMU) to trigger firmware errors
because firmware believes we are attempting to invalidate entries
that are still in-use by the guest OS (when in fact that guest is
in the process of being terminated or rebooted).
To alleviate this, register both a shutdown notifier (for unexpected
termination cases e.g. virsh destroy) as well as a reset callback
(for cases like guest OS reboot). For each of these scenarios, trigger
PCI device reset; this is enough to indicate to firmware that the IOMMU
is no longer in-use by the guest OS, making it safe to invalidate any
associated IOMMU entries.

Fixes: 15d0e7942d3b ("s390x/pci: don't fence interpreted devices without MSI-X")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221209195700.263824-1-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
[thuth: Adjusted the hunk in s390-pci-vfio.c due to different context]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached. Let's go a step further and also limit the size of
the supported DMA aperture reported to the guest based upon the
initial vfio DMA limit reported for the container (if less than
than the size reported by the firmware/host zPCI layer). This
avoids processing sections of the guest DMA table during global
refresh that, for common use cases, will never be used anway, and
makes exhausting the vfio DMA limit due to mismatch between guest
aperture size and host limit far less likely and more indicitive
of an error.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221028194758.204007-4-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we g

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we get a termination signal for QEMU) to trigger firmware errors
because firmware believes we are attempting to invalidate entries
that are still in-use by the guest OS (when in fact that guest is
in the process of being terminated or rebooted).
To alleviate this, register both a shutdown notifier (for unexpected
termination cases e.g. virsh destroy) as well as a reset callback
(for cases like guest OS reboot). For each of these scenarios, trigger
PCI device reset; this is enough to indicate to firmware that the IOMMU
is no longer in-use by the guest OS, making it safe to invalidate any
associated IOMMU entries.

Fixes: 15d0e7942d3b ("s390x/pci: don't fence interpreted devices without MSI-X")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221209195700.263824-1-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
[thuth: Adjusted the hunk in s390-pci-vfio.c due to different context]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached. Let's go a step further and also limit the size of
the supported DMA aperture reported to the guest based upon the
initial vfio DMA limit reported for the container (if less than
than the size reported by the firmware/host zPCI layer). This
avoids processing sections of the guest DMA table during global
refresh that, for common use cases, will never be used anway, and
makes exhausting the vfio DMA limit due to mismatch between guest
aperture size and host limit far less likely and more indicitive
of an error.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221028194758.204007-4-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we g

s390x/pci: reset ISM passthrough devices on shutdown and system reset

ISM device firmware stores unique state information that can
can cause a wholesale unmap of the associated IOMMU (e.g. when
we get a termination signal for QEMU) to trigger firmware errors
because firmware believes we are attempting to invalidate entries
that are still in-use by the guest OS (when in fact that guest is
in the process of being terminated or rebooted).
To alleviate this, register both a shutdown notifier (for unexpected
termination cases e.g. virsh destroy) as well as a reset callback
(for cases like guest OS reboot). For each of these scenarios, trigger
PCI device reset; this is enough to indicate to firmware that the IOMMU
is no longer in-use by the guest OS, making it safe to invalidate any
associated IOMMU entries.

Fixes: 15d0e7942d3b ("s390x/pci: don't fence interpreted devices without MSI-X")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221209195700.263824-1-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
[thuth: Adjusted the hunk in s390-pci-vfio.c due to different context]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached

s390x/pci: shrink DMA aperture to be bound by vfio DMA limit

Currently, s390x-pci performs accounting against the vfio DMA
limit and triggers the guest to clean up mappings when the limit
is reached. Let's go a step further and also limit the size of
the supported DMA aperture reported to the guest based upon the
initial vfio DMA limit reported for the container (if less than
than the size reported by the firmware/host zPCI layer). This
avoids processing sections of the guest DMA table during global
refresh that, for common use cases, will never be used anway, and
makes exhausting the vfio DMA limit due to mismatch between guest
aperture size and host limit far less likely and more indicitive
of an error.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-Id: <20221028194758.204007-4-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more common spelling here.

Message-Id: <20221111182828.282251-1-thuth@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more common spelling here.

Message-Id: <20221111182828.282251-1-thuth@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more

s390x: Fix spelling errors

Fix typos (discovered with the 'codespell' utility).
Note: Though "migrateable" still seems to be a valid spelling, we change
it to "migratable" since this is the way more common spelling here.

Message-Id: <20221111182828.282251-1-thuth@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmw

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmware-reported
maximum) or handled via userspace intercept (host kernel API maximum).
Choose the best available value during group creation.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-8-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some grou

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some group
values may be different. If we run out of simulated PCI groups, subsequent
intercept devices just get the default group.
Furthermore, if we encounter any PCI groups from hostdevs that are marked
as simulated, let's just assign them to the default group to avoid
conflicts between host simulated groups and our own simulated groups.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-7-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the
vfio CLP capabilities chain.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-3-mjrosato@linux.ibm.com>
[thuth: Replace free(info) with g_free(info)]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmw

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmware-reported
maximum) or handled via userspace intercept (host kernel API maximum).
Choose the best available value during group creation.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-8-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some grou

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some group
values may be different. If we run out of simulated PCI groups, subsequent
intercept devices just get the default group.
Furthermore, if we encounter any PCI groups from hostdevs that are marked
as simulated, let's just assign them to the default group to avoid
conflicts between host simulated groups and our own simulated groups.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-7-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the
vfio CLP capabilities chain.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-3-mjrosato@linux.ibm.com>
[thuth: Replace free(info) with g_free(info)]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmw

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmware-reported
maximum) or handled via userspace intercept (host kernel API maximum).
Choose the best available value during group creation.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-8-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some grou

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some group
values may be different. If we run out of simulated PCI groups, subsequent
intercept devices just get the default group.
Furthermore, if we encounter any PCI groups from hostdevs that are marked
as simulated, let's just assign them to the default group to avoid
conflicts between host simulated groups and our own simulated groups.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-7-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the

s390x/pci: add routine to get host function handle from CLP info

In order to interface with the underlying host zPCI device, we need
to know its function handle. Add a routine to grab this from the
vfio CLP capabilities chain.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-3-mjrosato@linux.ibm.com>
[thuth: Replace free(info) with g_free(info)]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmw

s390x/pci: reflect proper maxstbl for groups of interpreted devices

The maximum supported store block length might be different depending
on whether the instruction is interpretively executed (firmware-reported
maximum) or handled via userspace intercept (host kernel API maximum).
Choose the best available value during group creation.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-8-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some grou

s390x/pci: let intercept devices have separate PCI groups

Let's use the reserved pool of simulated PCI groups to allow intercept
devices to have separate groups from interpreted devices as some group
values may be different. If we run out of simulated PCI groups, subsequent
intercept devices just get the default group.
Furthermore, if we encounter any PCI groups from hostdevs that are marked
as simulated, let's just assign them to the default group to avoid
conflicts between host simulated groups and our own simulated groups.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <20220902172737.170349-7-mjrosato@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...