About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / DocBook / media / v4l / dev-overlay.xml

Custom Search

Based on kernel version 4.7.2. Page generated on 2016-08-22 22:44 EST.

1	  <title>Video Overlay Interface</title>
2	  <subtitle>Also known as Framebuffer Overlay or Previewing</subtitle>
4	  <para>Video overlay devices have the ability to genlock (TV-)video
5	into the (VGA-)video signal of a graphics card, or to store captured
6	images directly in video memory of a graphics card, typically with
7	clipping. This can be considerable more efficient than capturing
8	images and displaying them by other means. In the old days when only
9	nuclear power plants needed cooling towers this used to be the only
10	way to put live video into a window.</para>
12	  <para>Video overlay devices are accessed through the same character
13	special files as <link linkend="capture">video capture</link> devices.
14	Note the default function of a <filename>/dev/video</filename> device
15	is video capturing. The overlay function is only available after
16	calling the &VIDIOC-S-FMT; ioctl.</para>
18	    <para>The driver may support simultaneous overlay and capturing
19	using the read/write and streaming I/O methods. If so, operation at
20	the nominal frame rate of the video standard is not guaranteed. Frames
21	may be directed away from overlay to capture, or one field may be used
22	for overlay and the other for capture if the capture parameters permit
23	this.</para>
25	  <para>Applications should use different file descriptors for
26	capturing and overlay. This must be supported by all drivers capable
27	of simultaneous capturing and overlay. Optionally these drivers may
28	also permit capturing and overlay with a single file descriptor for
29	compatibility with V4L and earlier versions of V4L2.<footnote>
30		<para>A common application of two file descriptors is the
31	XFree86 <link linkend="xvideo">Xv/V4L</link> interface driver and
32	a V4L2 application. While the X server controls video overlay, the
33	application can take advantage of memory mapping and DMA.</para>
34		<para>In the opinion of the designers of this API, no driver
35	writer taking the efforts to support simultaneous capturing and
36	overlay will restrict this ability by requiring a single file
37	descriptor, as in V4L and earlier versions of V4L2. Making this
38	optional means applications depending on two file descriptors need
39	backup routines to be compatible with all drivers, which is
40	considerable more work than using two fds in applications which do
41	not. Also two fd's fit the general concept of one file descriptor for
42	each logical stream. Hence as a complexity trade-off drivers
43	<emphasis>must</emphasis> support two file descriptors and
44	<emphasis>may</emphasis> support single fd operation.</para>
45	      </footnote></para>
47	  <section>
48	    <title>Querying Capabilities</title>
50	    <para>Devices supporting the video overlay interface set the
51	<constant>V4L2_CAP_VIDEO_OVERLAY</constant> flag in the
52	<structfield>capabilities</structfield> field of &v4l2-capability;
53	returned by the &VIDIOC-QUERYCAP; ioctl. The overlay I/O method specified
54	below must be supported. Tuners and audio inputs are optional.</para>
55	  </section>
57	  <section>
58	    <title>Supplemental Functions</title>
60	    <para>Video overlay devices shall support <link
61	linkend="audio">audio input</link>, <link
62	linkend="tuner">tuner</link>, <link linkend="control">controls</link>,
63	<link linkend="crop">cropping and scaling</link> and <link
64	linkend="streaming-par">streaming parameter</link> ioctls as needed.
65	The <link linkend="video">video input</link> and <link
66	linkend="standard">video standard</link> ioctls must be supported by
67	all video overlay devices.</para>
68	  </section>
70	  <section>
71	    <title>Setup</title>
73	    <para>Before overlay can commence applications must program the
74	driver with frame buffer parameters, namely the address and size of
75	the frame buffer and the image format, for example RGB 5:6:5. The
76	&VIDIOC-G-FBUF; and &VIDIOC-S-FBUF; ioctls are available to get
77	and set these parameters, respectively. The
78	<constant>VIDIOC_S_FBUF</constant> ioctl is privileged because it
79	allows to set up DMA into physical memory, bypassing the memory
80	protection mechanisms of the kernel. Only the superuser can change the
81	frame buffer address and size. Users are not supposed to run TV
82	applications as root or with SUID bit set. A small helper application
83	with suitable privileges should query the graphics system and program
84	the V4L2 driver at the appropriate time.</para>
86	    <para>Some devices add the video overlay to the output signal
87	of the graphics card. In this case the frame buffer is not modified by
88	the video device, and the frame buffer address and pixel format are
89	not needed by the driver. The <constant>VIDIOC_S_FBUF</constant> ioctl
90	is not privileged. An application can check for this type of device by
91	calling the <constant>VIDIOC_G_FBUF</constant> ioctl.</para>
93	    <para>A driver may support any (or none) of five clipping/blending
94	methods:<orderedlist>
95		<listitem>
96		  <para>Chroma-keying displays the overlaid image only where
97	pixels in the primary graphics surface assume a certain color.</para>
98		</listitem>
99		<listitem>
100		  <para>A bitmap can be specified where each bit corresponds
101	to a pixel in the overlaid image. When the bit is set, the
102	corresponding video pixel is displayed, otherwise a pixel of the
103	graphics surface.</para>
104		</listitem>
105		<listitem>
106		  <para>A list of clipping rectangles can be specified. In
107	these regions <emphasis>no</emphasis> video is displayed, so the
108	graphics surface can be seen here.</para>
109		</listitem>
110		<listitem>
111		  <para>The framebuffer has an alpha channel that can be used
112	to clip or blend the framebuffer with the video.</para>
113		</listitem>
114		<listitem>
115		  <para>A global alpha value can be specified to blend the
116	framebuffer contents with video images.</para>
117		</listitem>
118	      </orderedlist></para>
120	    <para>When simultaneous capturing and overlay is supported and
121	the hardware prohibits different image and frame buffer formats, the
122	format requested first takes precedence. The attempt to capture
123	(&VIDIOC-S-FMT;) or overlay (&VIDIOC-S-FBUF;) may fail with an
124	&EBUSY; or return accordingly modified parameters..</para>
125	  </section>
127	  <section>
128	    <title>Overlay Window</title>
130	    <para>The overlaid image is determined by cropping and overlay
131	window parameters. The former select an area of the video picture to
132	capture, the latter how images are overlaid and clipped. Cropping
133	initialization at minimum requires to reset the parameters to
134	defaults. An example is given in <xref linkend="crop" />.</para>
136	    <para>The overlay window is described by a &v4l2-window;. It
137	defines the size of the image, its position over the graphics surface
138	and the clipping to be applied. To get the current parameters
139	applications set the <structfield>type</structfield> field of a
140	&v4l2-format; to <constant>V4L2_BUF_TYPE_VIDEO_OVERLAY</constant> and
141	call the &VIDIOC-G-FMT; ioctl. The driver fills the
142	<structname>v4l2_window</structname> substructure named
143	<structfield>win</structfield>. It is not possible to retrieve a
144	previously programmed clipping list or bitmap.</para>
146	    <para>To program the overlay window applications set the
147	<structfield>type</structfield> field of a &v4l2-format; to
148	<constant>V4L2_BUF_TYPE_VIDEO_OVERLAY</constant>, initialize the
149	<structfield>win</structfield> substructure and call the
150	&VIDIOC-S-FMT; ioctl. The driver adjusts the parameters against
151	hardware limits and returns the actual parameters as
152	<constant>VIDIOC_G_FMT</constant> does. Like
153	<constant>VIDIOC_S_FMT</constant>, the &VIDIOC-TRY-FMT; ioctl can be
154	used to learn about driver capabilities without actually changing
155	driver state. Unlike <constant>VIDIOC_S_FMT</constant> this also works
156	after the overlay has been enabled.</para>
158	    <para>The scaling factor of the overlaid image is implied by the
159	width and height given in &v4l2-window; and the size of the cropping
160	rectangle. For more information see <xref linkend="crop" />.</para>
162	    <para>When simultaneous capturing and overlay is supported and
163	the hardware prohibits different image and window sizes, the size
164	requested first takes precedence. The attempt to capture or overlay as
165	well (&VIDIOC-S-FMT;) may fail with an &EBUSY; or return accordingly
166	modified parameters.</para>
168	    <table pgwide="1" frame="none" id="v4l2-window">
169	      <title>struct <structname>v4l2_window</structname></title>
170	      <tgroup cols="3">
171		&cs-str;
172		<tbody valign="top">
173		  <row>
174		    <entry>&v4l2-rect;</entry>
175		    <entry><structfield>w</structfield></entry>
176		    <entry>Size and position of the window relative to the
177	top, left corner of the frame buffer defined with &VIDIOC-S-FBUF;. The
178	window can extend the frame buffer width and height, the
179	<structfield>x</structfield> and <structfield>y</structfield>
180	coordinates can be negative, and it can lie completely outside the
181	frame buffer. The driver clips the window accordingly, or if that is
182	not possible, modifies its size and/or position.</entry>
183		  </row>
184		  <row>
185		    <entry>&v4l2-field;</entry>
186		    <entry><structfield>field</structfield></entry>
187		    <entry>Applications set this field to determine which
188	video field shall be overlaid, typically one of
189	<constant>V4L2_FIELD_ANY</constant> (0),
190	<constant>V4L2_FIELD_TOP</constant>,
191	<constant>V4L2_FIELD_BOTTOM</constant> or
192	<constant>V4L2_FIELD_INTERLACED</constant>. Drivers may have to choose
193	a different field order and return the actual setting here.</entry>
194		    </row>
195		  <row>
196		    <entry>__u32</entry>
197		    <entry><structfield>chromakey</structfield></entry>
198		    <entry>When chroma-keying has been negotiated with
199	&VIDIOC-S-FBUF; applications set this field to the desired pixel value
200	for the chroma key. The format is the same as the pixel format of the
201	framebuffer (&v4l2-framebuffer;
202	<structfield>fmt.pixelformat</structfield> field), with bytes in host
203	order. E.&nbsp;g. for <link
204	linkend="V4L2-PIX-FMT-BGR32"><constant>V4L2_PIX_FMT_BGR24</constant></link>
205	the value should be 0xRRGGBB on a little endian, 0xBBGGRR on a big
206	endian host.</entry>
207		  </row>
208		  <row>
209		    <entry>&v4l2-clip; *</entry>
210		    <entry><structfield>clips</structfield></entry>
211		    <entry>When chroma-keying has <emphasis>not</emphasis>
212	been negotiated and &VIDIOC-G-FBUF; indicated this capability,
213	applications can set this field to point to an array of
214	clipping rectangles.</entry>
215		  </row>
216		  <row>
217		    <entry></entry>
218		    <entry></entry>
219		    <entry>Like the window coordinates
220	<structfield>w</structfield>, clipping rectangles are defined relative
221	to the top, left corner of the frame buffer. However clipping
222	rectangles must not extend the frame buffer width and height, and they
223	must not overlap. If possible applications should merge adjacent
224	rectangles. Whether this must create x-y or y-x bands, or the order of
225	rectangles, is not defined. When clip lists are not supported the
226	driver ignores this field. Its contents after calling &VIDIOC-S-FMT;
227	are undefined.</entry>
228		  </row>
229		  <row>
230		    <entry>__u32</entry>
231		    <entry><structfield>clipcount</structfield></entry>
232		    <entry>When the application set the
233	<structfield>clips</structfield> field, this field must contain the
234	number of clipping rectangles in the list. When clip lists are not
235	supported the driver ignores this field, its contents after calling
236	<constant>VIDIOC_S_FMT</constant> are undefined. When clip lists are
237	supported but no clipping is desired this field must be set to
238	zero.</entry>
239		  </row>
240		  <row>
241		    <entry>void *</entry>
242		    <entry><structfield>bitmap</structfield></entry>
243		    <entry>When chroma-keying has
244	<emphasis>not</emphasis> been negotiated and &VIDIOC-G-FBUF; indicated
245	this capability, applications can set this field to point to a
246	clipping bit mask.</entry>
247		  </row>
248		  <row>
249		    <entry spanname="hspan"><para>It must be of the same size
250	as the window, <structfield>w.width</structfield> and
251	<structfield>w.height</structfield>. Each bit corresponds to a pixel
252	in the overlaid image, which is displayed only when the bit is
253	<emphasis>set</emphasis>. Pixel coordinates translate to bits like:
254	<programlisting>
255	((__u8 *) <structfield>bitmap</structfield>)[<structfield>w.width</structfield> * y + x / 8] &amp; (1 &lt;&lt; (x &amp; 7))</programlisting></para><para>where <structfield>0</structfield> &le; x &lt;
256	<structfield>w.width</structfield> and <structfield>0</structfield> &le;
257	y &lt;<structfield>w.height</structfield>.<footnote>
258			  <para>Should we require
259		      <structfield>w.width</structfield> to be a multiple of
260		      eight?</para>
261			</footnote></para><para>When a clipping
262	bit mask is not supported the driver ignores this field, its contents
263	after calling &VIDIOC-S-FMT; are undefined. When a bit mask is supported
264	but no clipping is desired this field must be set to
265	<constant>NULL</constant>.</para><para>Applications need not create a
266	clip list or bit mask. When they pass both, or despite negotiating
267	chroma-keying, the results are undefined. Regardless of the chosen
268	method, the clipping abilities of the hardware may be limited in
269	quantity or quality. The results when these limits are exceeded are
270	undefined.<footnote>
271			  <para>When the image is written into frame buffer
272	memory it will be undesirable if the driver clips out less pixels
273	than expected, because the application and graphics system are not
274	aware these regions need to be refreshed. The driver should clip out
275	more pixels or not write the image at all.</para>
276			</footnote></para></entry>
277		  </row>
278		  <row>
279		    <entry>__u8</entry>
280		    <entry><structfield>global_alpha</structfield></entry>
281		    <entry>The global alpha value used to blend the
282	framebuffer with video images, if global alpha blending has been
283	negotiated (<constant>V4L2_FBUF_FLAG_GLOBAL_ALPHA</constant>, see
284	&VIDIOC-S-FBUF;, <xref linkend="framebuffer-flags" />).</entry>
285		  </row>
286		  <row>
287		    <entry></entry>
288		    <entry></entry>
289		    <entry>Note this field was added in Linux 2.6.23, extending the structure. However
290	the <link linkend="vidioc-g-fmt">VIDIOC_G/S/TRY_FMT</link> ioctls,
291	which take a pointer to a <link
292	linkend="v4l2-format">v4l2_format</link> parent structure with padding
293	bytes at the end, are not affected.</entry>
294		  </row>
295		</tbody>
296	      </tgroup>
297	    </table>
299	    <table pgwide="1" frame="none" id="v4l2-clip">
300	      <title>struct <structname>v4l2_clip</structname><footnote>
301		  <para>The X Window system defines "regions" which are
302	vectors of struct BoxRec { short x1, y1, x2, y2; } with width = x2 -
303	x1 and height = y2 - y1, so one cannot pass X11 clip lists
304	directly.</para>
305		</footnote></title>
306	      <tgroup cols="3">
307		&cs-str;
308		<tbody valign="top">
309		  <row>
310		    <entry>&v4l2-rect;</entry>
311		    <entry><structfield>c</structfield></entry>
312		    <entry>Coordinates of the clipping rectangle, relative to
313	the top, left corner of the frame buffer. Only window pixels
314	<emphasis>outside</emphasis> all clipping rectangles are
315	displayed.</entry>
316		  </row>
317		  <row>
318		    <entry>&v4l2-clip; *</entry>
319		    <entry><structfield>next</structfield></entry>
320		    <entry>Pointer to the next clipping rectangle, NULL when
321	this is the last rectangle. Drivers ignore this field, it cannot be
322	used to pass a linked list of clipping rectangles.</entry>
323		  </row>
324		</tbody>
325	      </tgroup>
326	    </table>
328	    <!-- NB for easier reading this table is duplicated
329	    in the vidioc-cropcap chapter.-->
331	    <table pgwide="1" frame="none" id="v4l2-rect">
332	      <title>struct <structname>v4l2_rect</structname></title>
333	      <tgroup cols="3">
334		&cs-str;
335		<tbody valign="top">
336		  <row>
337		    <entry>__s32</entry>
338		    <entry><structfield>left</structfield></entry>
339		    <entry>Horizontal offset of the top, left corner of the
340	rectangle, in pixels.</entry>
341		  </row>
342		  <row>
343		    <entry>__s32</entry>
344		    <entry><structfield>top</structfield></entry>
345		    <entry>Vertical offset of the top, left corner of the
346	rectangle, in pixels. Offsets increase to the right and down.</entry>
347		  </row>
348		  <row>
349		    <entry>__u32</entry>
350		    <entry><structfield>width</structfield></entry>
351		    <entry>Width of the rectangle, in pixels.</entry>
352		  </row>
353		  <row>
354		    <entry>__u32</entry>
355		    <entry><structfield>height</structfield></entry>
356		    <entry>Height of the rectangle, in pixels.</entry>
357		  </row>
358		</tbody>
359	      </tgroup>
360	    </table>
361	  </section>
363	  <section>
364	    <title>Enabling Overlay</title>
366	    <para>To start or stop the frame buffer overlay applications call
367	the &VIDIOC-OVERLAY; ioctl.</para>
368	  </section>
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.