media: v4l2-subdev: Store frame interval in subdev state

Subdev states store all standard pad configuration data, except for
frame intervals. Fix it by adding interval fields in the
v4l2_subdev_pad_

media: v4l2-subdev: Store frame interval in subdev state

Subdev states store all standard pad configuration data, except for
frame intervals. Fix it by adding interval fields in the
v4l2_subdev_pad_config and v4l2_subdev_stream_config structures, with
corresponding accessor functions and a helper function to implement the
.get_frame_interval() operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: v4l2-subdev: Turn .[gs]_frame_interval into pad operations

The subdev .[gs]_frame_interval are video operations, but they operate
on pads (and even on streams). Not only is this confusing, it

media: v4l2-subdev: Turn .[gs]_frame_interval into pad operations

The subdev .[gs]_frame_interval are video operations, but they operate
on pads (and even on streams). Not only is this confusing, it causes
practical issues for drivers as the operations don't receive a subdev
state pointer, requiring manual state handling.

To improve the situation, turn the operations into pad operations, and
extend them to receive a state pointer like other pad operations.

While at it, rename the operations to .[gs]et_frame_interval at the same
time to match the naming scheme of other pad operations. This isn't
strictly necessary, but given that all drivers using those operations
need to be modified, handling the rename separately would generate more
churn for very little gain (if at all).

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de> # for imx-media
Reviewed-by: Luca Ceresoli <luca.ceresoli@bootlin.com> # for tegra-video
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: v4l: subdev: Move out subdev state lock macros outside CONFIG_MEDIA_CONTROLLER

The subdev state locking macros and macros to get the active state are
currently behind CONFIG_MEDIA_CONTROLLER.

media: v4l: subdev: Move out subdev state lock macros outside CONFIG_MEDIA_CONTROLLER

The subdev state locking macros and macros to get the active state are
currently behind CONFIG_MEDIA_CONTROLLER. This makes sense, as there can
be no subdev state without MC.

However, we have code paths common to MC and non-MC cases which call
subdev operations that have subdev state as a parameter. In the non-MC
case the state parameter would always be NULL.

Thus it makes sense to allow, e.g.:

v4l2_subdev_call_state_active(sd, pad, get_fmt, fmt)

which for non-MC case would call the subdev passing NULL as the state.

This currently fails:
https://lore.kernel.org/oe-kbuild-all/202312061101.PLrz5NnJ-lkp@intel.com/

Fix the issue by moving the related macros to be outside
CONFIG_MEDIA_CONTROLLER. The v4l2_subdev_lock_state() and
v4l2_subdev_unlock_state() macros will crash if given NULL as the state,
but the other macros behave correctly even when there's no active state,
and they will only call the lock/unlock macros if there is a state.

An alternative fix would be to make another version of
v4l2_subdev_call_state_try() with ifdefs, which would not use any state
macros and would always pass NULL as the state. But having two version
of a macro/function is always more confusing than having just one, so I
went this way.

So, this fixes the v4l2_subdev_call_state_active() macro. But we also
have v4l2_subdev_call_state_try(). It would be possible to fix that
macro by additionally creating "no-op" variants of the state alloc and
free functions. However, v4l2_subdev_call_state_try() is only used by a
single driver (stm32-dcmi), which selects MC, and the macro is supposed
to be removed as soon as the users have been converted away from the
macro. Thus I have not touched the state alloc/free functions, and I
think it makes sense to keep alloc/free functions available only if
there's actually something that can be allocated or freed.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Link: https://lore.kernel.org/r/20231208-v4l2-state-mc-fix-v1-1-a0c8162557c6@ideasonboard.com
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: v4l2-subdev: Rename .init_cfg() operation to .init_state()

The subdev .init_cfg() operation is affected by two issues:

- It has long been extended to initialize a whole v4l2_subdev_state
i

media: v4l2-subdev: Rename .init_cfg() operation to .init_state()

The subdev .init_cfg() operation is affected by two issues:

- It has long been extended to initialize a whole v4l2_subdev_state
instead of just a v4l2_subdev_pad_config, but its name has stuck
around.

- Despite operating on a whole subdev state and not being directly
exposed to the subdev users (either in-kernel or through the userspace
API), .init_cfg() is categorized as a subdev pad operation.

This participates in making the subdev API confusing for new developers.
Fix it by renaming the operation to .init_state(), and make it a subdev
internal operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Michael Riesch <michael.riesch@wolfvision.net> # for imx415
Acked-by: Shuah Khan <skhan@linuxfoundation.org> # for vimc
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
[Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: v4l: subdev: Remove stream-unaware sub-device state access

Remove stream-unaware sub-device state access functions and macros. These
are no longer used.

[Sakari Ailus: Resolve a minor confli

media: v4l: subdev: Remove stream-unaware sub-device state access

Remove stream-unaware sub-device state access functions and macros. These
are no longer used.

[Sakari Ailus: Resolve a minor conflict in removed code.]

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: subdev: Always compile sub-device state access functions

Compile sub-device state information access functions
v4l2_subdev_state_get_{format,crop,compose} unconditionally as they are
now

media: v4l: subdev: Always compile sub-device state access functions

Compile sub-device state information access functions
v4l2_subdev_state_get_{format,crop,compose} unconditionally as they are
now also used by plain V4L2 drivers.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: subdev: Make stream argument optional in state access functions

The sub-device state access functions take three arguments: sub-device
state, pad and stream. The stream is not relevant f

media: v4l: subdev: Make stream argument optional in state access functions

The sub-device state access functions take three arguments: sub-device
state, pad and stream. The stream is not relevant for the majority of
drivers and having to specify 0 for the stream is considered a nuisance.

Provide a two-argument macros for these state access functions to cover
the needs of stream-unaware users.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: subdev: Rename sub-device state information access functions

Rename the sub-devices state information access functions, removing
"_stream" from them and replacing "format" by "ffmt". Thi

media: v4l: subdev: Rename sub-device state information access functions

Rename the sub-devices state information access functions, removing
"_stream" from them and replacing "format" by "ffmt". This makes them
shorter and so more convenient to use. No other sets of functions will be
needed to access this information.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: subdev: Also return pads array information on stream functions

There are two sets of functions that return information from sub-device
state, one for stream-unaware users and another for

media: v4l: subdev: Also return pads array information on stream functions

There are two sets of functions that return information from sub-device
state, one for stream-unaware users and another for stream-aware users.
Add support for stream-aware functions to return format, crop and compose
information from pad-based array that are functionally equivalent to the
old, stream-unaware ones.

Also check state is non-NULL, in order to guard against old drivers
potentially calling this with NULL state for active formats or selection
rectangles.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: subdev: Store the sub-device in the sub-device state

Store the sub-device in the sub-device state. This will be needed in e.g.
validating pad number when retrieving information for non-s

media: v4l: subdev: Store the sub-device in the sub-device state

Store the sub-device in the sub-device state. This will be needed in e.g.
validating pad number when retrieving information for non-stream-aware
users. There are expected to be more needs for this in the future.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l2-subdev: Fix references to pad config

V4L2 subdev operations have moved from operating on a
v4l2_subdev_pad_config to a v4l2_subdev_state a long time ago. Fix the
few remaining incorrect

media: v4l2-subdev: Fix references to pad config

V4L2 subdev operations have moved from operating on a
v4l2_subdev_pad_config to a v4l2_subdev_state a long time ago. Fix the
few remaining incorrect references to pad config in the documentation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l2-subdev: Drop outdated comment for v4l2_subdev_pad_config

The v4l2_subdev_pad_config structure is not passed to subdev operations
anymore. Drop an outdated comment to refers to that old m

media: v4l2-subdev: Drop outdated comment for v4l2_subdev_pad_config

The v4l2_subdev_pad_config structure is not passed to subdev operations
anymore. Drop an outdated comment to refers to that old mechanism.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l2-subdev: Rename pad config 'try_*' fields

The try_fmt, try_crop and try_compose fields of the
v4l2_subdev_pad_config structure are misnamed (for historical reason) as
they also store data

media: v4l2-subdev: Rename pad config 'try_*' fields

The try_fmt, try_crop and try_compose fields of the
v4l2_subdev_pad_config structure are misnamed (for historical reason) as
they also store data for the subdev active configuration. Rename them to
format, crop and compose respectively and update the accessor helpers.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: use sizeof() instead of V4L2_SUBDEV_NAME_SIZE

Don't rely on a define, let the compiler use the actual
field size.

Remove all uses of the V4L2_SUBDEV_NAME_SIZE define and also
drop the define

media: use sizeof() instead of V4L2_SUBDEV_NAME_SIZE

Don't rely on a define, let the compiler use the actual
field size.

Remove all uses of the V4L2_SUBDEV_NAME_SIZE define and also
drop the define itself.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Tested-by: Luca Ceresoli <luca.ceresoli@bootlin.com>

show more ...

media: v4l2-subdev.h: increase struct v4l2_subdev name size

This resolves a lot of the string truncate compiler warnings.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: v4l2-subdev: Document and enforce .s_stream() requirements

The subdev .s_stream() operation must not be called to start an already
started subdev, or stop an already stopped one. This require

media: v4l2-subdev: Document and enforce .s_stream() requirements

The subdev .s_stream() operation must not be called to start an already
started subdev, or stop an already stopped one. This requirement has
never been formally documented. Fix it, and catch possible offenders
with a WARN_ON() in the call_s_stream() wrapper.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: v4l2-subdev: Document that routing support depends on streams

Routing support, through the subdev .set_routing() operation, requires
the subdev to support streams. This is however not clearly

media: v4l2-subdev: Document that routing support depends on streams

Routing support, through the subdev .set_routing() operation, requires
the subdev to support streams. This is however not clearly documented
anywhere. Fix it by expanding the operation's documentation to indicate
that subdevs must set the V4L2_SUBDEV_FL_STREAMS flag.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

show more ...

media: subdev: Constify v4l2_subdev_set_routing_with_fmt() param

The routing parameter of v4l2_subdev_set_routing_with_fmt() is missing
'const'. Add it.

Signed-off-by: Tomi Valkeinen <tomi.valkeine

media: subdev: Constify v4l2_subdev_set_routing_with_fmt() param

The routing parameter of v4l2_subdev_set_routing_with_fmt() is missing
'const'. Add it.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: async: Support fwnode endpoint list matching for subdevs

Support matching V4L2 async sub-devices based on particular fwnode
endpoint. This makes it possible to instantiate multiple V4L2

media: v4l: async: Support fwnode endpoint list matching for subdevs

Support matching V4L2 async sub-devices based on particular fwnode
endpoint. This makes it possible to instantiate multiple V4L2 sub-devices
based on given fwnode endpoints from a single device, based on driver
needs.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Tested-by: Philipp Zabel <p.zabel@pengutronix.de> # imx6qp
Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> # rcar + adv746x
Tested-by: Aishwarya Kothari <aishwarya.kothari@toradex.com> # Apalis i.MX6Q with TC358743
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> # Renesas RZ/G2L SMARC
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: async: Allow multiple connections between entities

When the v4l2-async framework was introduced, the use case for it was to
connect a camera sensor with a parallel receiver. Both tended

media: v4l: async: Allow multiple connections between entities

When the v4l2-async framework was introduced, the use case for it was to
connect a camera sensor with a parallel receiver. Both tended to be rather
simple devices with a single connection between them.

The framework has been since improved in multiple ways but there are
limitations that have remained, for instance the assumption an async
sub-device is connected towards a single notifier and via a single link
only.

This patch enables connecting a sub-device to one or more notifiers
simultaneously, with one or more connections per notifier. The notifier
information is moved from the sub-device to the connection and the
connections in sub-device are no longer a pointer but a linked list.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Tested-by: Philipp Zabel <p.zabel@pengutronix.de> # imx6qp
Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> # rcar + adv746x
Tested-by: Aishwarya Kothari <aishwarya.kothari@toradex.com> # Apalis i.MX6Q with TC358743
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> # Renesas RZ/G2L SMARC
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: async: Rename v4l2_async_subdev as v4l2_async_connection

Rename v4l2_async_subdev as v4l2_async_connection, in order to
differentiate between the sub-devices and their connections: one
s

media: v4l: async: Rename v4l2_async_subdev as v4l2_async_connection

Rename v4l2_async_subdev as v4l2_async_connection, in order to
differentiate between the sub-devices and their connections: one
sub-device can have many connections but the V4L2 async framework has so
far allowed just a single one. Connections in this context will later
translate into either MC ancillary or data links.

This patch prepares changing that relation by changing existing users of
v4l2_async_subdev to switch to v4l2_async_connection. Async sub-devices
themselves will not be needed anymore

Additionally, __v4l2_async_nf_add_subdev() has been renamed
__v4l2_async_nf_add_connection().

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Tested-by: Philipp Zabel <p.zabel@pengutronix.de> # imx6qp
Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> # rcar + adv746x
Tested-by: Aishwarya Kothari <aishwarya.kothari@toradex.com> # Apalis i.MX6Q with TC358743
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> # Renesas RZ/G2L SMARC
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l: async: Clean up list heads and entries

The naming of list heads and list entries is confusing as they're named
similarly. Use _list for list head and _entry for list entries.

Signed-off

media: v4l: async: Clean up list heads and entries

The naming of list heads and list entries is confusing as they're named
similarly. Use _list for list head and _entry for list entries.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Tested-by: Philipp Zabel <p.zabel@pengutronix.de> # imx6qp
Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> # rcar + adv746x
Tested-by: Aishwarya Kothari <aishwarya.kothari@toradex.com> # Apalis i.MX6Q with TC358743
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> # Renesas RZ/G2L SMARC
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l2-subdev: Fix missing kerneldoc for client_caps

Add missing kernel doc for the new 'client_caps' field in struct
v4l2_subdev_fh.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard

media: v4l2-subdev: Fix missing kerneldoc for client_caps

Add missing kernel doc for the new 'client_caps' field in struct
v4l2_subdev_fh.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Fixes: f57fa2959244 ("media: v4l2-subdev: Add new ioctl for client capabilities")
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: v4l2-subdev: Add new ioctl for client capabilities

Add new ioctls to set and get subdev client capabilities. Client in this
context means the userspace application which opens the subdev devi

media: v4l2-subdev: Add new ioctl for client capabilities

Add new ioctls to set and get subdev client capabilities. Client in this
context means the userspace application which opens the subdev device
node. The client capabilities are stored in the file handle of the
opened subdev device node, and the client must set the capabilities for
each opened subdev.

For now we only add a single flag, V4L2_SUBDEV_CLIENT_CAP_STREAMS, which
indicates that the client is streams-aware.

The reason for needing such a flag is as follows:

Many structs passed via ioctls, e.g. struct v4l2_subdev_format, contain
reserved fields (usually a single array field). These reserved fields
can be used to extend the ioctl. The userspace is required to zero the
reserved fields.

We recently added a new 'stream' field to many of these structs, and the
space for the field was taken from these reserved arrays. The assumption
was that these new 'stream' fields are always initialized to zero if the
userspace does not use them. This was a mistake, as, as mentioned above,
the userspace is required to zero the _reserved_ fields. In other words,
there is no requirement to zero this new stream field, and if the
userspace doesn't use the field (which is the case for all userspace
applications at the moment), the field may contain random data.

This shows that the way the reserved fields are defined in v4l2 is, in
my opinion, somewhat broken, but there is nothing to do about that.

To fix this issue we need a way for the userspace to tell the kernel
that the userspace has indeed set the 'stream' field, and it's fine for
the kernel to access it. This is achieved with the new ioctl, which the
userspace should usually use right after opening the subdev device node.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: subdev: Add V4L2_SUBDEV_ROUTING_NO_MULTIPLEXING

A common case with subdev routing is that on the subdevice just before
the DMA engines (video nodes), no multiplexing is allowed on the source

media: subdev: Add V4L2_SUBDEV_ROUTING_NO_MULTIPLEXING

A common case with subdev routing is that on the subdevice just before
the DMA engines (video nodes), no multiplexing is allowed on the source
pads, as the DMA engine can only handle a single stream.

In some other situations one might also want to do the same check on the
sink side.

Add new routing validation flags to check these:
V4L2_SUBDEV_ROUTING_NO_SINK_MULTIPLEXING and
V4L2_SUBDEV_ROUTING_NO_SOURCE_MULTIPLEXING.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...