132e2eae2SMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 232e2eae2SMauro Carvalho Chehab 332e2eae2SMauro Carvalho Chehabi.MX Video Capture Driver 432e2eae2SMauro Carvalho Chehab========================= 532e2eae2SMauro Carvalho Chehab 632e2eae2SMauro Carvalho ChehabIntroduction 732e2eae2SMauro Carvalho Chehab------------ 832e2eae2SMauro Carvalho Chehab 932e2eae2SMauro Carvalho ChehabThe Freescale i.MX5/6 contains an Image Processing Unit (IPU), which 1032e2eae2SMauro Carvalho Chehabhandles the flow of image frames to and from capture devices and 1132e2eae2SMauro Carvalho Chehabdisplay devices. 1232e2eae2SMauro Carvalho Chehab 1332e2eae2SMauro Carvalho ChehabFor image capture, the IPU contains the following internal subunits: 1432e2eae2SMauro Carvalho Chehab 1532e2eae2SMauro Carvalho Chehab- Image DMA Controller (IDMAC) 1632e2eae2SMauro Carvalho Chehab- Camera Serial Interface (CSI) 1732e2eae2SMauro Carvalho Chehab- Image Converter (IC) 1832e2eae2SMauro Carvalho Chehab- Sensor Multi-FIFO Controller (SMFC) 1932e2eae2SMauro Carvalho Chehab- Image Rotator (IRT) 2032e2eae2SMauro Carvalho Chehab- Video De-Interlacing or Combining Block (VDIC) 2132e2eae2SMauro Carvalho Chehab 2232e2eae2SMauro Carvalho ChehabThe IDMAC is the DMA controller for transfer of image frames to and from 2332e2eae2SMauro Carvalho Chehabmemory. Various dedicated DMA channels exist for both video capture and 2432e2eae2SMauro Carvalho Chehabdisplay paths. During transfer, the IDMAC is also capable of vertical 2532e2eae2SMauro Carvalho Chehabimage flip, 8x8 block transfer (see IRT description), pixel component 2632e2eae2SMauro Carvalho Chehabre-ordering (for example UYVY to YUYV) within the same colorspace, and 2732e2eae2SMauro Carvalho Chehabpacked <--> planar conversion. The IDMAC can also perform a simple 2832e2eae2SMauro Carvalho Chehabde-interlacing by interweaving even and odd lines during transfer 2932e2eae2SMauro Carvalho Chehab(without motion compensation which requires the VDIC). 3032e2eae2SMauro Carvalho Chehab 3132e2eae2SMauro Carvalho ChehabThe CSI is the backend capture unit that interfaces directly with 3232e2eae2SMauro Carvalho Chehabcamera sensors over Parallel, BT.656/1120, and MIPI CSI-2 buses. 3332e2eae2SMauro Carvalho Chehab 3432e2eae2SMauro Carvalho ChehabThe IC handles color-space conversion, resizing (downscaling and 3532e2eae2SMauro Carvalho Chehabupscaling), horizontal flip, and 90/270 degree rotation operations. 3632e2eae2SMauro Carvalho Chehab 3732e2eae2SMauro Carvalho ChehabThere are three independent "tasks" within the IC that can carry out 3832e2eae2SMauro Carvalho Chehabconversions concurrently: pre-process encoding, pre-process viewfinder, 3932e2eae2SMauro Carvalho Chehaband post-processing. Within each task, conversions are split into three 4032e2eae2SMauro Carvalho Chehabsections: downsizing section, main section (upsizing, flip, colorspace 4132e2eae2SMauro Carvalho Chehabconversion, and graphics plane combining), and rotation section. 4232e2eae2SMauro Carvalho Chehab 4332e2eae2SMauro Carvalho ChehabThe IPU time-shares the IC task operations. The time-slice granularity 4432e2eae2SMauro Carvalho Chehabis one burst of eight pixels in the downsizing section, one image line 4532e2eae2SMauro Carvalho Chehabin the main processing section, one image frame in the rotation section. 4632e2eae2SMauro Carvalho Chehab 4732e2eae2SMauro Carvalho ChehabThe SMFC is composed of four independent FIFOs that each can transfer 4832e2eae2SMauro Carvalho Chehabcaptured frames from sensors directly to memory concurrently via four 4932e2eae2SMauro Carvalho ChehabIDMAC channels. 5032e2eae2SMauro Carvalho Chehab 5132e2eae2SMauro Carvalho ChehabThe IRT carries out 90 and 270 degree image rotation operations. The 5232e2eae2SMauro Carvalho Chehabrotation operation is carried out on 8x8 pixel blocks at a time. This 5332e2eae2SMauro Carvalho Chehaboperation is supported by the IDMAC which handles the 8x8 block transfer 5432e2eae2SMauro Carvalho Chehabalong with block reordering, in coordination with vertical flip. 5532e2eae2SMauro Carvalho Chehab 5632e2eae2SMauro Carvalho ChehabThe VDIC handles the conversion of interlaced video to progressive, with 5732e2eae2SMauro Carvalho Chehabsupport for different motion compensation modes (low, medium, and high 5832e2eae2SMauro Carvalho Chehabmotion). The deinterlaced output frames from the VDIC can be sent to the 5932e2eae2SMauro Carvalho ChehabIC pre-process viewfinder task for further conversions. The VDIC also 6032e2eae2SMauro Carvalho Chehabcontains a Combiner that combines two image planes, with alpha blending 6132e2eae2SMauro Carvalho Chehaband color keying. 6232e2eae2SMauro Carvalho Chehab 6332e2eae2SMauro Carvalho ChehabIn addition to the IPU internal subunits, there are also two units 6432e2eae2SMauro Carvalho Chehaboutside the IPU that are also involved in video capture on i.MX: 6532e2eae2SMauro Carvalho Chehab 6632e2eae2SMauro Carvalho Chehab- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus 6732e2eae2SMauro Carvalho Chehab interface. This is a Synopsys DesignWare core. 6832e2eae2SMauro Carvalho Chehab- Two video multiplexers for selecting among multiple sensor inputs 6932e2eae2SMauro Carvalho Chehab to send to a CSI. 7032e2eae2SMauro Carvalho Chehab 7132e2eae2SMauro Carvalho ChehabFor more info, refer to the latest versions of the i.MX5/6 reference 7232e2eae2SMauro Carvalho Chehabmanuals [#f1]_ and [#f2]_. 7332e2eae2SMauro Carvalho Chehab 7432e2eae2SMauro Carvalho Chehab 7532e2eae2SMauro Carvalho ChehabFeatures 7632e2eae2SMauro Carvalho Chehab-------- 7732e2eae2SMauro Carvalho Chehab 7832e2eae2SMauro Carvalho ChehabSome of the features of this driver include: 7932e2eae2SMauro Carvalho Chehab 8032e2eae2SMauro Carvalho Chehab- Many different pipelines can be configured via media controller API, 8132e2eae2SMauro Carvalho Chehab that correspond to the hardware video capture pipelines supported in 8232e2eae2SMauro Carvalho Chehab the i.MX. 8332e2eae2SMauro Carvalho Chehab 8432e2eae2SMauro Carvalho Chehab- Supports parallel, BT.565, and MIPI CSI-2 interfaces. 8532e2eae2SMauro Carvalho Chehab 8632e2eae2SMauro Carvalho Chehab- Concurrent independent streams, by configuring pipelines to multiple 8732e2eae2SMauro Carvalho Chehab video capture interfaces using independent entities. 8832e2eae2SMauro Carvalho Chehab 8932e2eae2SMauro Carvalho Chehab- Scaling, color-space conversion, horizontal and vertical flip, and 9032e2eae2SMauro Carvalho Chehab image rotation via IC task subdevs. 9132e2eae2SMauro Carvalho Chehab 9232e2eae2SMauro Carvalho Chehab- Many pixel formats supported (RGB, packed and planar YUV, partial 9332e2eae2SMauro Carvalho Chehab planar YUV). 9432e2eae2SMauro Carvalho Chehab 9532e2eae2SMauro Carvalho Chehab- The VDIC subdev supports motion compensated de-interlacing, with three 9632e2eae2SMauro Carvalho Chehab motion compensation modes: low, medium, and high motion. Pipelines are 9732e2eae2SMauro Carvalho Chehab defined that allow sending frames to the VDIC subdev directly from the 9832e2eae2SMauro Carvalho Chehab CSI. There is also support in the future for sending frames to the 9932e2eae2SMauro Carvalho Chehab VDIC from memory buffers via a output/mem2mem devices. 10032e2eae2SMauro Carvalho Chehab 10132e2eae2SMauro Carvalho Chehab- Includes a Frame Interval Monitor (FIM) that can correct vertical sync 10232e2eae2SMauro Carvalho Chehab problems with the ADV718x video decoders. 10332e2eae2SMauro Carvalho Chehab 10432e2eae2SMauro Carvalho Chehab 105cdb82f82SSteve LongerbeamTopology 106cdb82f82SSteve Longerbeam-------- 107cdb82f82SSteve Longerbeam 108cdb82f82SSteve LongerbeamThe following shows the media topologies for the i.MX6Q SabreSD and 109cdb82f82SSteve Longerbeami.MX6Q SabreAuto. Refer to these diagrams in the entity descriptions 110cdb82f82SSteve Longerbeamin the next section. 111cdb82f82SSteve Longerbeam 112cdb82f82SSteve LongerbeamThe i.MX5/6 topologies can differ upstream from the IPUv3 CSI video 113cdb82f82SSteve Longerbeammultiplexers, but the internal IPUv3 topology downstream from there 114cdb82f82SSteve Longerbeamis common to all i.MX5/6 platforms. For example, the SabreSD, with the 115cdb82f82SSteve LongerbeamMIPI CSI-2 OV5640 sensor, requires the i.MX6 MIPI CSI-2 receiver. But 116cdb82f82SSteve Longerbeamthe SabreAuto has only the ADV7180 decoder on a parallel bt.656 bus, and 117cdb82f82SSteve Longerbeamtherefore does not require the MIPI CSI-2 receiver, so it is missing in 118cdb82f82SSteve Longerbeamits graph. 119cdb82f82SSteve Longerbeam 120cdb82f82SSteve Longerbeam.. _imx6q_topology_graph: 121cdb82f82SSteve Longerbeam 122cdb82f82SSteve Longerbeam.. kernel-figure:: imx6q-sabresd.dot 123cdb82f82SSteve Longerbeam :alt: Diagram of the i.MX6Q SabreSD media pipeline topology 124cdb82f82SSteve Longerbeam :align: center 125cdb82f82SSteve Longerbeam 126cdb82f82SSteve Longerbeam Media pipeline graph on i.MX6Q SabreSD 127cdb82f82SSteve Longerbeam 128cdb82f82SSteve Longerbeam.. kernel-figure:: imx6q-sabreauto.dot 129cdb82f82SSteve Longerbeam :alt: Diagram of the i.MX6Q SabreAuto media pipeline topology 130cdb82f82SSteve Longerbeam :align: center 131cdb82f82SSteve Longerbeam 132cdb82f82SSteve Longerbeam Media pipeline graph on i.MX6Q SabreAuto 133cdb82f82SSteve Longerbeam 13432e2eae2SMauro Carvalho ChehabEntities 13532e2eae2SMauro Carvalho Chehab-------- 13632e2eae2SMauro Carvalho Chehab 13732e2eae2SMauro Carvalho Chehabimx6-mipi-csi2 13832e2eae2SMauro Carvalho Chehab-------------- 13932e2eae2SMauro Carvalho Chehab 14032e2eae2SMauro Carvalho ChehabThis is the MIPI CSI-2 receiver entity. It has one sink pad to receive 14132e2eae2SMauro Carvalho Chehabthe MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has 14232e2eae2SMauro Carvalho Chehabfour source pads, corresponding to the four MIPI CSI-2 demuxed virtual 14332e2eae2SMauro Carvalho Chehabchannel outputs. Multiple source pads can be enabled to independently 14432e2eae2SMauro Carvalho Chehabstream from multiple virtual channels. 14532e2eae2SMauro Carvalho Chehab 14632e2eae2SMauro Carvalho ChehabThis entity actually consists of two sub-blocks. One is the MIPI CSI-2 14732e2eae2SMauro Carvalho Chehabcore. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block 14832e2eae2SMauro Carvalho Chehabis a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the 14932e2eae2SMauro Carvalho Chehabfour virtual channels streams, providing four separate parallel buses 15032e2eae2SMauro Carvalho Chehabcontaining each virtual channel that are routed to CSIs or video 15132e2eae2SMauro Carvalho Chehabmultiplexers as described below. 15232e2eae2SMauro Carvalho Chehab 15332e2eae2SMauro Carvalho ChehabOn i.MX6 solo/dual-lite, all four virtual channel buses are routed to 15432e2eae2SMauro Carvalho Chehabtwo video multiplexers. Both CSI0 and CSI1 can receive any virtual 15532e2eae2SMauro Carvalho Chehabchannel, as selected by the video multiplexers. 15632e2eae2SMauro Carvalho Chehab 15732e2eae2SMauro Carvalho ChehabOn i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected 15832e2eae2SMauro Carvalho Chehabby a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1 15932e2eae2SMauro Carvalho Chehaband IPU2-CSI0, respectively, and virtual channel 3 is routed to 16032e2eae2SMauro Carvalho ChehabIPU2-CSI1 (again selected by a video mux). 16132e2eae2SMauro Carvalho Chehab 16232e2eae2SMauro Carvalho ChehabipuX_csiY_mux 16332e2eae2SMauro Carvalho Chehab------------- 16432e2eae2SMauro Carvalho Chehab 16532e2eae2SMauro Carvalho ChehabThese are the video multiplexers. They have two or more sink pads to 16632e2eae2SMauro Carvalho Chehabselect from either camera sensors with a parallel interface, or from 16732e2eae2SMauro Carvalho ChehabMIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a 16832e2eae2SMauro Carvalho Chehabsingle source pad that routes to a CSI (ipuX_csiY entities). 16932e2eae2SMauro Carvalho Chehab 17032e2eae2SMauro Carvalho ChehabOn i.MX6 solo/dual-lite, there are two video mux entities. One sits 17132e2eae2SMauro Carvalho Chehabin front of IPU1-CSI0 to select between a parallel sensor and any of 17232e2eae2SMauro Carvalho Chehabthe four MIPI CSI-2 virtual channels (a total of five sink pads). The 17332e2eae2SMauro Carvalho Chehabother mux sits in front of IPU1-CSI1, and again has five sink pads to 17432e2eae2SMauro Carvalho Chehabselect between a parallel sensor and any of the four MIPI CSI-2 virtual 17532e2eae2SMauro Carvalho Chehabchannels. 17632e2eae2SMauro Carvalho Chehab 17732e2eae2SMauro Carvalho ChehabOn i.MX6 Quad, there are two video mux entities. One sits in front of 17832e2eae2SMauro Carvalho ChehabIPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual 17932e2eae2SMauro Carvalho Chehabchannel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to 18032e2eae2SMauro Carvalho Chehabselect between a parallel sensor and MIPI CSI-2 virtual channel 3 (two 18132e2eae2SMauro Carvalho Chehabsink pads). 18232e2eae2SMauro Carvalho Chehab 18332e2eae2SMauro Carvalho ChehabipuX_csiY 18432e2eae2SMauro Carvalho Chehab--------- 18532e2eae2SMauro Carvalho Chehab 18632e2eae2SMauro Carvalho ChehabThese are the CSI entities. They have a single sink pad receiving from 18732e2eae2SMauro Carvalho Chehabeither a video mux or from a MIPI CSI-2 virtual channel as described 18832e2eae2SMauro Carvalho Chehababove. 18932e2eae2SMauro Carvalho Chehab 19032e2eae2SMauro Carvalho ChehabThis entity has two source pads. The first source pad can link directly 19132e2eae2SMauro Carvalho Chehabto the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links 19232e2eae2SMauro Carvalho Chehabthat require no IDMAC memory buffer transfer. 19332e2eae2SMauro Carvalho Chehab 19432e2eae2SMauro Carvalho ChehabWhen the direct source pad is routed to the ipuX_ic_prp entity, frames 19532e2eae2SMauro Carvalho Chehabfrom the CSI can be processed by one or both of the IC pre-processing 19632e2eae2SMauro Carvalho Chehabtasks. 19732e2eae2SMauro Carvalho Chehab 19832e2eae2SMauro Carvalho ChehabWhen the direct source pad is routed to the ipuX_vdic entity, the VDIC 19932e2eae2SMauro Carvalho Chehabwill carry out motion-compensated de-interlace using "high motion" mode 20032e2eae2SMauro Carvalho Chehab(see description of ipuX_vdic entity). 20132e2eae2SMauro Carvalho Chehab 20232e2eae2SMauro Carvalho ChehabThe second source pad sends video frames directly to memory buffers 20332e2eae2SMauro Carvalho Chehabvia the SMFC and an IDMAC channel, bypassing IC pre-processing. This 20432e2eae2SMauro Carvalho Chehabsource pad is routed to a capture device node, with a node name of the 20532e2eae2SMauro Carvalho Chehabformat "ipuX_csiY capture". 20632e2eae2SMauro Carvalho Chehab 20732e2eae2SMauro Carvalho ChehabNote that since the IDMAC source pad makes use of an IDMAC channel, 20832e2eae2SMauro Carvalho Chehabpixel reordering within the same colorspace can be carried out by the 20932e2eae2SMauro Carvalho ChehabIDMAC channel. For example, if the CSI sink pad is receiving in UYVY 21032e2eae2SMauro Carvalho Chehaborder, the capture device linked to the IDMAC source pad can capture 21132e2eae2SMauro Carvalho Chehabin YUYV order. Also, if the CSI sink pad is receiving a packed YUV 21232e2eae2SMauro Carvalho Chehabformat, the capture device can capture a planar YUV format such as 21332e2eae2SMauro Carvalho ChehabYUV420. 21432e2eae2SMauro Carvalho Chehab 21532e2eae2SMauro Carvalho ChehabThe IDMAC channel at the IDMAC source pad also supports simple 21632e2eae2SMauro Carvalho Chehabinterweave without motion compensation, which is activated if the source 21732e2eae2SMauro Carvalho Chehabpad's field type is sequential top-bottom or bottom-top, and the 21832e2eae2SMauro Carvalho Chehabrequested capture interface field type is set to interlaced (t-b, b-t, 21932e2eae2SMauro Carvalho Chehabor unqualified interlaced). The capture interface will enforce the same 22032e2eae2SMauro Carvalho Chehabfield order as the source pad field order (interlaced-bt if source pad 22132e2eae2SMauro Carvalho Chehabis seq-bt, interlaced-tb if source pad is seq-tb). 22232e2eae2SMauro Carvalho Chehab 22332e2eae2SMauro Carvalho ChehabFor events produced by ipuX_csiY, see ref:`imx_api_ipuX_csiY`. 22432e2eae2SMauro Carvalho Chehab 22532e2eae2SMauro Carvalho ChehabCropping in ipuX_csiY 22632e2eae2SMauro Carvalho Chehab--------------------- 22732e2eae2SMauro Carvalho Chehab 22832e2eae2SMauro Carvalho ChehabThe CSI supports cropping the incoming raw sensor frames. This is 22932e2eae2SMauro Carvalho Chehabimplemented in the ipuX_csiY entities at the sink pad, using the 23032e2eae2SMauro Carvalho Chehabcrop selection subdev API. 23132e2eae2SMauro Carvalho Chehab 23232e2eae2SMauro Carvalho ChehabThe CSI also supports fixed divide-by-two downscaling independently in 23332e2eae2SMauro Carvalho Chehabwidth and height. This is implemented in the ipuX_csiY entities at 23432e2eae2SMauro Carvalho Chehabthe sink pad, using the compose selection subdev API. 23532e2eae2SMauro Carvalho Chehab 23632e2eae2SMauro Carvalho ChehabThe output rectangle at the ipuX_csiY source pad is the same as 23732e2eae2SMauro Carvalho Chehabthe compose rectangle at the sink pad. So the source pad rectangle 23832e2eae2SMauro Carvalho Chehabcannot be negotiated, it must be set using the compose selection 23932e2eae2SMauro Carvalho ChehabAPI at sink pad (if /2 downscale is desired, otherwise source pad 24032e2eae2SMauro Carvalho Chehabrectangle is equal to incoming rectangle). 24132e2eae2SMauro Carvalho Chehab 24232e2eae2SMauro Carvalho ChehabTo give an example of crop and /2 downscale, this will crop a 24332e2eae2SMauro Carvalho Chehab1280x960 input frame to 640x480, and then /2 downscale in both 24432e2eae2SMauro Carvalho Chehabdimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux): 24532e2eae2SMauro Carvalho Chehab 24632e2eae2SMauro Carvalho Chehab.. code-block:: none 24732e2eae2SMauro Carvalho Chehab 24832e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]" 24932e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]" 25032e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]" 25132e2eae2SMauro Carvalho Chehab 25232e2eae2SMauro Carvalho ChehabFrame Skipping in ipuX_csiY 25332e2eae2SMauro Carvalho Chehab--------------------------- 25432e2eae2SMauro Carvalho Chehab 25532e2eae2SMauro Carvalho ChehabThe CSI supports frame rate decimation, via frame skipping. Frame 25632e2eae2SMauro Carvalho Chehabrate decimation is specified by setting the frame intervals at 25732e2eae2SMauro Carvalho Chehabsink and source pads. The ipuX_csiY entity then applies the best 25832e2eae2SMauro Carvalho Chehabframe skip setting to the CSI to achieve the desired frame rate 25932e2eae2SMauro Carvalho Chehabat the source pad. 26032e2eae2SMauro Carvalho Chehab 26132e2eae2SMauro Carvalho ChehabThe following example reduces an assumed incoming 60 Hz frame 26232e2eae2SMauro Carvalho Chehabrate by half at the IDMAC output source pad: 26332e2eae2SMauro Carvalho Chehab 26432e2eae2SMauro Carvalho Chehab.. code-block:: none 26532e2eae2SMauro Carvalho Chehab 26632e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]" 26732e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]" 26832e2eae2SMauro Carvalho Chehab 26932e2eae2SMauro Carvalho ChehabFrame Interval Monitor in ipuX_csiY 27032e2eae2SMauro Carvalho Chehab----------------------------------- 27132e2eae2SMauro Carvalho Chehab 27232e2eae2SMauro Carvalho ChehabSee ref:`imx_api_FIM`. 27332e2eae2SMauro Carvalho Chehab 27432e2eae2SMauro Carvalho ChehabipuX_vdic 27532e2eae2SMauro Carvalho Chehab--------- 27632e2eae2SMauro Carvalho Chehab 27732e2eae2SMauro Carvalho ChehabThe VDIC carries out motion compensated de-interlacing, with three 27832e2eae2SMauro Carvalho Chehabmotion compensation modes: low, medium, and high motion. The mode is 27932e2eae2SMauro Carvalho Chehabspecified with the menu control V4L2_CID_DEINTERLACING_MODE. The VDIC 28032e2eae2SMauro Carvalho Chehabhas two sink pads and a single source pad. 28132e2eae2SMauro Carvalho Chehab 28232e2eae2SMauro Carvalho ChehabThe direct sink pad receives from an ipuX_csiY direct pad. With this 28332e2eae2SMauro Carvalho Chehablink the VDIC can only operate in high motion mode. 28432e2eae2SMauro Carvalho Chehab 28532e2eae2SMauro Carvalho ChehabWhen the IDMAC sink pad is activated, it receives from an output 28632e2eae2SMauro Carvalho Chehabor mem2mem device node. With this pipeline, the VDIC can also operate 28732e2eae2SMauro Carvalho Chehabin low and medium modes, because these modes require receiving 28832e2eae2SMauro Carvalho Chehabframes from memory buffers. Note that an output or mem2mem device 28932e2eae2SMauro Carvalho Chehabis not implemented yet, so this sink pad currently has no links. 29032e2eae2SMauro Carvalho Chehab 29132e2eae2SMauro Carvalho ChehabThe source pad routes to the IC pre-processing entity ipuX_ic_prp. 29232e2eae2SMauro Carvalho Chehab 29332e2eae2SMauro Carvalho ChehabipuX_ic_prp 29432e2eae2SMauro Carvalho Chehab----------- 29532e2eae2SMauro Carvalho Chehab 29632e2eae2SMauro Carvalho ChehabThis is the IC pre-processing entity. It acts as a router, routing 29732e2eae2SMauro Carvalho Chehabdata from its sink pad to one or both of its source pads. 29832e2eae2SMauro Carvalho Chehab 29932e2eae2SMauro Carvalho ChehabThis entity has a single sink pad. The sink pad can receive from the 30032e2eae2SMauro Carvalho ChehabipuX_csiY direct pad, or from ipuX_vdic. 30132e2eae2SMauro Carvalho Chehab 30232e2eae2SMauro Carvalho ChehabThis entity has two source pads. One source pad routes to the 30332e2eae2SMauro Carvalho Chehabpre-process encode task entity (ipuX_ic_prpenc), the other to the 30432e2eae2SMauro Carvalho Chehabpre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads 30532e2eae2SMauro Carvalho Chehabcan be activated at the same time if the sink pad is receiving from 30632e2eae2SMauro Carvalho ChehabipuX_csiY. Only the source pad to the pre-process viewfinder task entity 30732e2eae2SMauro Carvalho Chehabcan be activated if the sink pad is receiving from ipuX_vdic (frames 30832e2eae2SMauro Carvalho Chehabfrom the VDIC can only be processed by the pre-process viewfinder task). 30932e2eae2SMauro Carvalho Chehab 31032e2eae2SMauro Carvalho ChehabipuX_ic_prpenc 31132e2eae2SMauro Carvalho Chehab-------------- 31232e2eae2SMauro Carvalho Chehab 31332e2eae2SMauro Carvalho ChehabThis is the IC pre-processing encode entity. It has a single sink 31432e2eae2SMauro Carvalho Chehabpad from ipuX_ic_prp, and a single source pad. The source pad is 31532e2eae2SMauro Carvalho Chehabrouted to a capture device node, with a node name of the format 31632e2eae2SMauro Carvalho Chehab"ipuX_ic_prpenc capture". 31732e2eae2SMauro Carvalho Chehab 31832e2eae2SMauro Carvalho ChehabThis entity performs the IC pre-process encode task operations: 31932e2eae2SMauro Carvalho Chehabcolor-space conversion, resizing (downscaling and upscaling), 32032e2eae2SMauro Carvalho Chehabhorizontal and vertical flip, and 90/270 degree rotation. Flip 32132e2eae2SMauro Carvalho Chehaband rotation are provided via standard V4L2 controls. 32232e2eae2SMauro Carvalho Chehab 32332e2eae2SMauro Carvalho ChehabLike the ipuX_csiY IDMAC source, this entity also supports simple 32432e2eae2SMauro Carvalho Chehabde-interlace without motion compensation, and pixel reordering. 32532e2eae2SMauro Carvalho Chehab 32632e2eae2SMauro Carvalho ChehabipuX_ic_prpvf 32732e2eae2SMauro Carvalho Chehab------------- 32832e2eae2SMauro Carvalho Chehab 32932e2eae2SMauro Carvalho ChehabThis is the IC pre-processing viewfinder entity. It has a single sink 33032e2eae2SMauro Carvalho Chehabpad from ipuX_ic_prp, and a single source pad. The source pad is routed 33132e2eae2SMauro Carvalho Chehabto a capture device node, with a node name of the format 33232e2eae2SMauro Carvalho Chehab"ipuX_ic_prpvf capture". 33332e2eae2SMauro Carvalho Chehab 33432e2eae2SMauro Carvalho ChehabThis entity is identical in operation to ipuX_ic_prpenc, with the same 33532e2eae2SMauro Carvalho Chehabresizing and CSC operations and flip/rotation controls. It will receive 33632e2eae2SMauro Carvalho Chehaband process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is 33732e2eae2SMauro Carvalho Chehabreceiving from ipuX_vdic. 33832e2eae2SMauro Carvalho Chehab 33932e2eae2SMauro Carvalho ChehabLike the ipuX_csiY IDMAC source, this entity supports simple 34032e2eae2SMauro Carvalho Chehabinterweaving without motion compensation. However, note that if the 34132e2eae2SMauro Carvalho ChehabipuX_vdic is included in the pipeline (ipuX_ic_prp is receiving from 34232e2eae2SMauro Carvalho ChehabipuX_vdic), it's not possible to use interweave in ipuX_ic_prpvf, 34332e2eae2SMauro Carvalho Chehabsince the ipuX_vdic has already carried out de-interlacing (with 34432e2eae2SMauro Carvalho Chehabmotion compensation) and therefore the field type output from 34532e2eae2SMauro Carvalho ChehabipuX_vdic can only be none (progressive). 34632e2eae2SMauro Carvalho Chehab 34732e2eae2SMauro Carvalho ChehabCapture Pipelines 34832e2eae2SMauro Carvalho Chehab----------------- 34932e2eae2SMauro Carvalho Chehab 35032e2eae2SMauro Carvalho ChehabThe following describe the various use-cases supported by the pipelines. 35132e2eae2SMauro Carvalho Chehab 35232e2eae2SMauro Carvalho ChehabThe links shown do not include the backend sensor, video mux, or mipi 35332e2eae2SMauro Carvalho Chehabcsi-2 receiver links. This depends on the type of sensor interface 35432e2eae2SMauro Carvalho Chehab(parallel or mipi csi-2). So these pipelines begin with: 35532e2eae2SMauro Carvalho Chehab 35632e2eae2SMauro Carvalho Chehabsensor -> ipuX_csiY_mux -> ... 35732e2eae2SMauro Carvalho Chehab 35832e2eae2SMauro Carvalho Chehabfor parallel sensors, or: 35932e2eae2SMauro Carvalho Chehab 36032e2eae2SMauro Carvalho Chehabsensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ... 36132e2eae2SMauro Carvalho Chehab 36232e2eae2SMauro Carvalho Chehabfor mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route 36332e2eae2SMauro Carvalho Chehabto the video mux (ipuX_csiY_mux) before sending to the CSI, depending 36432e2eae2SMauro Carvalho Chehabon the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in 36532e2eae2SMauro Carvalho Chehabparenthesis. 36632e2eae2SMauro Carvalho Chehab 36732e2eae2SMauro Carvalho ChehabUnprocessed Video Capture: 36832e2eae2SMauro Carvalho Chehab-------------------------- 36932e2eae2SMauro Carvalho Chehab 37032e2eae2SMauro Carvalho ChehabSend frames directly from sensor to camera device interface node, with 37132e2eae2SMauro Carvalho Chehabno conversions, via ipuX_csiY IDMAC source pad: 37232e2eae2SMauro Carvalho Chehab 37332e2eae2SMauro Carvalho Chehab-> ipuX_csiY:2 -> ipuX_csiY capture 37432e2eae2SMauro Carvalho Chehab 37532e2eae2SMauro Carvalho ChehabIC Direct Conversions: 37632e2eae2SMauro Carvalho Chehab---------------------- 37732e2eae2SMauro Carvalho Chehab 37832e2eae2SMauro Carvalho ChehabThis pipeline uses the preprocess encode entity to route frames directly 37932e2eae2SMauro Carvalho Chehabfrom the CSI to the IC, to carry out scaling up to 1024x1024 resolution, 38032e2eae2SMauro Carvalho ChehabCSC, flipping, and image rotation: 38132e2eae2SMauro Carvalho Chehab 38232e2eae2SMauro Carvalho Chehab-> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture 38332e2eae2SMauro Carvalho Chehab 38432e2eae2SMauro Carvalho ChehabMotion Compensated De-interlace: 38532e2eae2SMauro Carvalho Chehab-------------------------------- 38632e2eae2SMauro Carvalho Chehab 38732e2eae2SMauro Carvalho ChehabThis pipeline routes frames from the CSI direct pad to the VDIC entity to 38832e2eae2SMauro Carvalho Chehabsupport motion-compensated de-interlacing (high motion mode only), 38932e2eae2SMauro Carvalho Chehabscaling up to 1024x1024, CSC, flip, and rotation: 39032e2eae2SMauro Carvalho Chehab 39132e2eae2SMauro Carvalho Chehab-> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture 39232e2eae2SMauro Carvalho Chehab 39332e2eae2SMauro Carvalho Chehab 39432e2eae2SMauro Carvalho ChehabUsage Notes 39532e2eae2SMauro Carvalho Chehab----------- 39632e2eae2SMauro Carvalho Chehab 39732e2eae2SMauro Carvalho ChehabTo aid in configuration and for backward compatibility with V4L2 39832e2eae2SMauro Carvalho Chehabapplications that access controls only from video device nodes, the 39932e2eae2SMauro Carvalho Chehabcapture device interfaces inherit controls from the active entities 40032e2eae2SMauro Carvalho Chehabin the current pipeline, so controls can be accessed either directly 40132e2eae2SMauro Carvalho Chehabfrom the subdev or from the active capture device interface. For 40232e2eae2SMauro Carvalho Chehabexample, the FIM controls are available either from the ipuX_csiY 40332e2eae2SMauro Carvalho Chehabsubdevs or from the active capture device. 40432e2eae2SMauro Carvalho Chehab 40532e2eae2SMauro Carvalho ChehabThe following are specific usage notes for the Sabre* reference 40632e2eae2SMauro Carvalho Chehabboards: 40732e2eae2SMauro Carvalho Chehab 40832e2eae2SMauro Carvalho Chehab 409cdb82f82SSteve Longerbeami.MX6Q SabreLite with OV5642 and OV5640 410cdb82f82SSteve Longerbeam--------------------------------------- 41132e2eae2SMauro Carvalho Chehab 41232e2eae2SMauro Carvalho ChehabThis platform requires the OmniVision OV5642 module with a parallel 41332e2eae2SMauro Carvalho Chehabcamera interface, and the OV5640 module with a MIPI CSI-2 41432e2eae2SMauro Carvalho Chehabinterface. Both modules are available from Boundary Devices: 41532e2eae2SMauro Carvalho Chehab 41632e2eae2SMauro Carvalho Chehab- https://boundarydevices.com/product/nit6x_5mp 41732e2eae2SMauro Carvalho Chehab- https://boundarydevices.com/product/nit6x_5mp_mipi 41832e2eae2SMauro Carvalho Chehab 41932e2eae2SMauro Carvalho ChehabNote that if only one camera module is available, the other sensor 42032e2eae2SMauro Carvalho Chehabnode can be disabled in the device tree. 42132e2eae2SMauro Carvalho Chehab 42232e2eae2SMauro Carvalho ChehabThe OV5642 module is connected to the parallel bus input on the i.MX 42332e2eae2SMauro Carvalho Chehabinternal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2. 42432e2eae2SMauro Carvalho Chehab 42532e2eae2SMauro Carvalho ChehabThe MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2 42632e2eae2SMauro Carvalho Chehabreceiver, and the four virtual channel outputs from the receiver are 42732e2eae2SMauro Carvalho Chehabrouted as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1, 42832e2eae2SMauro Carvalho Chehabvc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is 42932e2eae2SMauro Carvalho Chehabalso connected to i2c bus 2 on the SabreLite, therefore the OV5642 and 43032e2eae2SMauro Carvalho ChehabOV5640 must not share the same i2c slave address. 43132e2eae2SMauro Carvalho Chehab 43232e2eae2SMauro Carvalho ChehabThe following basic example configures unprocessed video capture 43332e2eae2SMauro Carvalho Chehabpipelines for both sensors. The OV5642 is routed to ipu1_csi0, and 43432e2eae2SMauro Carvalho Chehabthe OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is 43532e2eae2SMauro Carvalho Chehabimx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are 43632e2eae2SMauro Carvalho Chehabconfigured to output 640x480, and the OV5642 outputs YUYV2X8, the 43732e2eae2SMauro Carvalho ChehabOV5640 UYVY2X8: 43832e2eae2SMauro Carvalho Chehab 43932e2eae2SMauro Carvalho Chehab.. code-block:: none 44032e2eae2SMauro Carvalho Chehab 44132e2eae2SMauro Carvalho Chehab # Setup links for OV5642 44232e2eae2SMauro Carvalho Chehab media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]" 44332e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 44432e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" 44532e2eae2SMauro Carvalho Chehab # Setup links for OV5640 44632e2eae2SMauro Carvalho Chehab media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]" 44732e2eae2SMauro Carvalho Chehab media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" 44832e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" 44932e2eae2SMauro Carvalho Chehab # Configure pads for OV5642 pipeline 45032e2eae2SMauro Carvalho Chehab media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]" 45132e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]" 45232e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]" 45332e2eae2SMauro Carvalho Chehab # Configure pads for OV5640 pipeline 45432e2eae2SMauro Carvalho Chehab media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]" 45532e2eae2SMauro Carvalho Chehab media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]" 45632e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]" 45732e2eae2SMauro Carvalho Chehab 45832e2eae2SMauro Carvalho ChehabStreaming can then begin independently on the capture device nodes 45932e2eae2SMauro Carvalho Chehab"ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can 46032e2eae2SMauro Carvalho Chehabbe used to select any supported YUV pixelformat on the capture device 46132e2eae2SMauro Carvalho Chehabnodes, including planar. 46232e2eae2SMauro Carvalho Chehab 46332e2eae2SMauro Carvalho Chehabi.MX6Q SabreAuto with ADV7180 decoder 46432e2eae2SMauro Carvalho Chehab------------------------------------- 46532e2eae2SMauro Carvalho Chehab 46632e2eae2SMauro Carvalho ChehabOn the i.MX6Q SabreAuto, an on-board ADV7180 SD decoder is connected to the 46732e2eae2SMauro Carvalho Chehabparallel bus input on the internal video mux to IPU1 CSI0. 46832e2eae2SMauro Carvalho Chehab 46932e2eae2SMauro Carvalho ChehabThe following example configures a pipeline to capture from the ADV7180 47032e2eae2SMauro Carvalho Chehabvideo decoder, assuming NTSC 720x480 input signals, using simple 47132e2eae2SMauro Carvalho Chehabinterweave (unconverted and without motion compensation). The adv7180 47232e2eae2SMauro Carvalho Chehabmust output sequential or alternating fields (field type 'seq-bt' for 47332e2eae2SMauro Carvalho ChehabNTSC, or 'alternate'): 47432e2eae2SMauro Carvalho Chehab 47532e2eae2SMauro Carvalho Chehab.. code-block:: none 47632e2eae2SMauro Carvalho Chehab 47732e2eae2SMauro Carvalho Chehab # Setup links 47832e2eae2SMauro Carvalho Chehab media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" 47932e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 48032e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" 48132e2eae2SMauro Carvalho Chehab # Configure pads 48232e2eae2SMauro Carvalho Chehab media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" 48332e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]" 48432e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]" 48532e2eae2SMauro Carvalho Chehab # Configure "ipu1_csi0 capture" interface (assumed at /dev/video4) 48632e2eae2SMauro Carvalho Chehab v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt 48732e2eae2SMauro Carvalho Chehab 48832e2eae2SMauro Carvalho ChehabStreaming can then begin on /dev/video4. The v4l2-ctl tool can also be 48932e2eae2SMauro Carvalho Chehabused to select any supported YUV pixelformat on /dev/video4. 49032e2eae2SMauro Carvalho Chehab 49132e2eae2SMauro Carvalho ChehabThis example configures a pipeline to capture from the ADV7180 49232e2eae2SMauro Carvalho Chehabvideo decoder, assuming PAL 720x576 input signals, with Motion 49332e2eae2SMauro Carvalho ChehabCompensated de-interlacing. The adv7180 must output sequential or 49432e2eae2SMauro Carvalho Chehabalternating fields (field type 'seq-tb' for PAL, or 'alternate'). 49532e2eae2SMauro Carvalho Chehab 49632e2eae2SMauro Carvalho Chehab.. code-block:: none 49732e2eae2SMauro Carvalho Chehab 49832e2eae2SMauro Carvalho Chehab # Setup links 49932e2eae2SMauro Carvalho Chehab media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" 50032e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 50132e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" 50232e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" 50332e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" 50432e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" 50532e2eae2SMauro Carvalho Chehab # Configure pads 50632e2eae2SMauro Carvalho Chehab media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]" 50732e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x576]" 50832e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]" 50932e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]" 51032e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]" 51132e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]" 51232e2eae2SMauro Carvalho Chehab # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2) 51332e2eae2SMauro Carvalho Chehab v4l2-ctl -d2 --set-fmt-video=field=none 51432e2eae2SMauro Carvalho Chehab 51532e2eae2SMauro Carvalho ChehabStreaming can then begin on /dev/video2. The v4l2-ctl tool can also be 51632e2eae2SMauro Carvalho Chehabused to select any supported YUV pixelformat on /dev/video2. 51732e2eae2SMauro Carvalho Chehab 51832e2eae2SMauro Carvalho ChehabThis platform accepts Composite Video analog inputs to the ADV7180 on 51932e2eae2SMauro Carvalho ChehabAin1 (connector J42). 52032e2eae2SMauro Carvalho Chehab 52132e2eae2SMauro Carvalho Chehabi.MX6DL SabreAuto with ADV7180 decoder 52232e2eae2SMauro Carvalho Chehab-------------------------------------- 52332e2eae2SMauro Carvalho Chehab 52432e2eae2SMauro Carvalho ChehabOn the i.MX6DL SabreAuto, an on-board ADV7180 SD decoder is connected to the 52532e2eae2SMauro Carvalho Chehabparallel bus input on the internal video mux to IPU1 CSI0. 52632e2eae2SMauro Carvalho Chehab 52732e2eae2SMauro Carvalho ChehabThe following example configures a pipeline to capture from the ADV7180 52832e2eae2SMauro Carvalho Chehabvideo decoder, assuming NTSC 720x480 input signals, using simple 52932e2eae2SMauro Carvalho Chehabinterweave (unconverted and without motion compensation). The adv7180 53032e2eae2SMauro Carvalho Chehabmust output sequential or alternating fields (field type 'seq-bt' for 53132e2eae2SMauro Carvalho ChehabNTSC, or 'alternate'): 53232e2eae2SMauro Carvalho Chehab 53332e2eae2SMauro Carvalho Chehab.. code-block:: none 53432e2eae2SMauro Carvalho Chehab 53532e2eae2SMauro Carvalho Chehab # Setup links 53632e2eae2SMauro Carvalho Chehab media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]" 53732e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]" 53832e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" 53932e2eae2SMauro Carvalho Chehab # Configure pads 54032e2eae2SMauro Carvalho Chehab media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" 54132e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]" 54232e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]" 54332e2eae2SMauro Carvalho Chehab # Configure "ipu1_csi0 capture" interface (assumed at /dev/video0) 54432e2eae2SMauro Carvalho Chehab v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt 54532e2eae2SMauro Carvalho Chehab 54632e2eae2SMauro Carvalho ChehabStreaming can then begin on /dev/video0. The v4l2-ctl tool can also be 54732e2eae2SMauro Carvalho Chehabused to select any supported YUV pixelformat on /dev/video0. 54832e2eae2SMauro Carvalho Chehab 54932e2eae2SMauro Carvalho ChehabThis example configures a pipeline to capture from the ADV7180 55032e2eae2SMauro Carvalho Chehabvideo decoder, assuming PAL 720x576 input signals, with Motion 55132e2eae2SMauro Carvalho ChehabCompensated de-interlacing. The adv7180 must output sequential or 55232e2eae2SMauro Carvalho Chehabalternating fields (field type 'seq-tb' for PAL, or 'alternate'). 55332e2eae2SMauro Carvalho Chehab 55432e2eae2SMauro Carvalho Chehab.. code-block:: none 55532e2eae2SMauro Carvalho Chehab 55632e2eae2SMauro Carvalho Chehab # Setup links 55732e2eae2SMauro Carvalho Chehab media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]" 55832e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]" 55932e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" 56032e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" 56132e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" 56232e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" 56332e2eae2SMauro Carvalho Chehab # Configure pads 56432e2eae2SMauro Carvalho Chehab media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]" 56532e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x576]" 56632e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]" 56732e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]" 56832e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]" 56932e2eae2SMauro Carvalho Chehab media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]" 57032e2eae2SMauro Carvalho Chehab # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2) 57132e2eae2SMauro Carvalho Chehab v4l2-ctl -d2 --set-fmt-video=field=none 57232e2eae2SMauro Carvalho Chehab 57332e2eae2SMauro Carvalho ChehabStreaming can then begin on /dev/video2. The v4l2-ctl tool can also be 57432e2eae2SMauro Carvalho Chehabused to select any supported YUV pixelformat on /dev/video2. 57532e2eae2SMauro Carvalho Chehab 57632e2eae2SMauro Carvalho ChehabThis platform accepts Composite Video analog inputs to the ADV7180 on 57732e2eae2SMauro Carvalho ChehabAin1 (connector J42). 57832e2eae2SMauro Carvalho Chehab 579cdb82f82SSteve Longerbeami.MX6Q SabreSD with MIPI CSI-2 OV5640 580cdb82f82SSteve Longerbeam------------------------------------- 58132e2eae2SMauro Carvalho Chehab 582cdb82f82SSteve LongerbeamSimilarly to i.MX6Q SabreLite, the i.MX6Q SabreSD supports a parallel 583cdb82f82SSteve Longerbeaminterface OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 584cdb82f82SSteve Longerbeammodule. The OV5642 connects to i2c bus 1 and the OV5640 to i2c bus 2. 58532e2eae2SMauro Carvalho Chehab 58632e2eae2SMauro Carvalho ChehabThe device tree for SabreSD includes OF graphs for both the parallel 58732e2eae2SMauro Carvalho ChehabOV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI 58832e2eae2SMauro Carvalho ChehabCSI-2 OV5640 has been tested, so the OV5642 node is currently disabled. 5898a4676d6SFabio EstevamThe OV5640 module connects to MIPI connector J5. The NXP part number 5908a4676d6SFabio Estevamfor the OV5640 module that connects to the SabreSD board is H120729. 59132e2eae2SMauro Carvalho Chehab 59262ce7201SFabio EstevamThe following example configures unprocessed video capture pipeline to 59362ce7201SFabio Estevamcapture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0: 59462ce7201SFabio Estevam 59562ce7201SFabio Estevam.. code-block:: none 59662ce7201SFabio Estevam 59762ce7201SFabio Estevam # Setup links 59862ce7201SFabio Estevam media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" 59962ce7201SFabio Estevam media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]" 60062ce7201SFabio Estevam media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 60162ce7201SFabio Estevam media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" 60262ce7201SFabio Estevam # Configure pads 60362ce7201SFabio Estevam media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]" 60462ce7201SFabio Estevam media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]" 60562ce7201SFabio Estevam media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]" 60662ce7201SFabio Estevam media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]" 60762ce7201SFabio Estevam 60862ce7201SFabio EstevamStreaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl 60962ce7201SFabio Estevamtool can be used to select any supported pixelformat on the capture 61062ce7201SFabio Estevamdevice node. 61162ce7201SFabio Estevam 612*d00c9761SFabio EstevamTo determine what is the /dev/video node correspondent to 613*d00c9761SFabio Estevam"ipu1_csi0 capture": 614*d00c9761SFabio Estevam 615*d00c9761SFabio Estevam.. code-block:: none 616*d00c9761SFabio Estevam 617*d00c9761SFabio Estevam media-ctl -e "ipu1_csi0 capture" 618*d00c9761SFabio Estevam /dev/video0 619*d00c9761SFabio Estevam 620*d00c9761SFabio Estevam/dev/video0 is the streaming element in this case. 621*d00c9761SFabio Estevam 622*d00c9761SFabio EstevamStarting the streaming via v4l2-ctl: 623*d00c9761SFabio Estevam 624*d00c9761SFabio Estevam.. code-block:: none 625*d00c9761SFabio Estevam 626*d00c9761SFabio Estevam v4l2-ctl --stream-mmap -d /dev/video0 627*d00c9761SFabio Estevam 628*d00c9761SFabio EstevamStarting the streaming via Gstreamer and sending the content to the display: 629*d00c9761SFabio Estevam 630*d00c9761SFabio Estevam.. code-block:: none 631*d00c9761SFabio Estevam 632*d00c9761SFabio Estevam gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink 633*d00c9761SFabio Estevam 63432e2eae2SMauro Carvalho ChehabThe following example configures a direct conversion pipeline to capture 6359429e707SFabio Estevamfrom the OV5640, transmitting on MIPI CSI-2 virtual channel 0. It also 6369429e707SFabio Estevamshows colorspace conversion and scaling at IC output. 63732e2eae2SMauro Carvalho Chehab 63832e2eae2SMauro Carvalho Chehab.. code-block:: none 63932e2eae2SMauro Carvalho Chehab 64032e2eae2SMauro Carvalho Chehab # Setup links 64132e2eae2SMauro Carvalho Chehab media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" 6429429e707SFabio Estevam media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]" 6439429e707SFabio Estevam media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 6449429e707SFabio Estevam media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]" 64532e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" 64632e2eae2SMauro Carvalho Chehab media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" 64732e2eae2SMauro Carvalho Chehab # Configure pads 6489429e707SFabio Estevam media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]" 6499429e707SFabio Estevam media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]" 6509429e707SFabio Estevam media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]" 6519429e707SFabio Estevam media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]" 6529429e707SFabio Estevam media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]" 6539429e707SFabio Estevam media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]" 6549429e707SFabio Estevam # Set a format at the capture interface 6559429e707SFabio Estevam v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3 65632e2eae2SMauro Carvalho Chehab 6579429e707SFabio EstevamStreaming can then begin on "ipu1_ic_prpenc capture" node. 65832e2eae2SMauro Carvalho Chehab 659*d00c9761SFabio EstevamTo determine what is the /dev/video node correspondent to 660*d00c9761SFabio Estevam"ipu1_ic_prpenc capture": 661*d00c9761SFabio Estevam 662*d00c9761SFabio Estevam.. code-block:: none 663*d00c9761SFabio Estevam 664*d00c9761SFabio Estevam media-ctl -e "ipu1_ic_prpenc capture" 665*d00c9761SFabio Estevam /dev/video1 666*d00c9761SFabio Estevam 667*d00c9761SFabio Estevam 668*d00c9761SFabio Estevam/dev/video1 is the streaming element in this case. 669*d00c9761SFabio Estevam 670*d00c9761SFabio EstevamStarting the streaming via v4l2-ctl: 671*d00c9761SFabio Estevam 672*d00c9761SFabio Estevam.. code-block:: none 673*d00c9761SFabio Estevam 674*d00c9761SFabio Estevam v4l2-ctl --stream-mmap -d /dev/video1 675*d00c9761SFabio Estevam 676*d00c9761SFabio EstevamStarting the streaming via Gstreamer and sending the content to the display: 677*d00c9761SFabio Estevam 678*d00c9761SFabio Estevam.. code-block:: none 679*d00c9761SFabio Estevam 680*d00c9761SFabio Estevam gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink 681*d00c9761SFabio Estevam 68232e2eae2SMauro Carvalho ChehabKnown Issues 68332e2eae2SMauro Carvalho Chehab------------ 68432e2eae2SMauro Carvalho Chehab 68532e2eae2SMauro Carvalho Chehab1. When using 90 or 270 degree rotation control at capture resolutions 68632e2eae2SMauro Carvalho Chehab near the IC resizer limit of 1024x1024, and combined with planar 68732e2eae2SMauro Carvalho Chehab pixel formats (YUV420, YUV422p), frame capture will often fail with 68832e2eae2SMauro Carvalho Chehab no end-of-frame interrupts from the IDMAC channel. To work around 68932e2eae2SMauro Carvalho Chehab this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) 69032e2eae2SMauro Carvalho Chehab when 90 or 270 rotations are needed. 69132e2eae2SMauro Carvalho Chehab 69232e2eae2SMauro Carvalho Chehab 69332e2eae2SMauro Carvalho ChehabFile list 69432e2eae2SMauro Carvalho Chehab--------- 69532e2eae2SMauro Carvalho Chehab 69632e2eae2SMauro Carvalho Chehabdrivers/staging/media/imx/ 69732e2eae2SMauro Carvalho Chehabinclude/media/imx.h 69832e2eae2SMauro Carvalho Chehabinclude/linux/imx-media.h 69932e2eae2SMauro Carvalho Chehab 70032e2eae2SMauro Carvalho ChehabReferences 70132e2eae2SMauro Carvalho Chehab---------- 70232e2eae2SMauro Carvalho Chehab 70332e2eae2SMauro Carvalho Chehab.. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf 70432e2eae2SMauro Carvalho Chehab.. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf 70532e2eae2SMauro Carvalho Chehab 70632e2eae2SMauro Carvalho Chehab 70732e2eae2SMauro Carvalho ChehabAuthors 70832e2eae2SMauro Carvalho Chehab------- 70932e2eae2SMauro Carvalho Chehab 71032e2eae2SMauro Carvalho Chehab- Steve Longerbeam <steve_longerbeam@mentor.com> 71132e2eae2SMauro Carvalho Chehab- Philipp Zabel <kernel@pengutronix.de> 71232e2eae2SMauro Carvalho Chehab- Russell King <linux@armlinux.org.uk> 71332e2eae2SMauro Carvalho Chehab 71432e2eae2SMauro Carvalho ChehabCopyright (C) 2012-2017 Mentor Graphics Inc. 715