Based on kernel version 4.7.2. Page generated on 2016-08-22 22:48 EST.
1 Overview of the V4L2 driver framework 2 ===================================== 3 4 This text documents the various structures provided by the V4L2 framework and 5 their relationships. 6 7 8 Introduction 9 ------------ 10 11 The V4L2 drivers tend to be very complex due to the complexity of the 12 hardware: most devices have multiple ICs, export multiple device nodes in 13 /dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input 14 (IR) devices. 15 16 Especially the fact that V4L2 drivers have to setup supporting ICs to 17 do audio/video muxing/encoding/decoding makes it more complex than most. 18 Usually these ICs are connected to the main bridge driver through one or 19 more I2C busses, but other busses can also be used. Such devices are 20 called 'sub-devices'. 21 22 For a long time the framework was limited to the video_device struct for 23 creating V4L device nodes and video_buf for handling the video buffers 24 (note that this document does not discuss the video_buf framework). 25 26 This meant that all drivers had to do the setup of device instances and 27 connecting to sub-devices themselves. Some of this is quite complicated 28 to do right and many drivers never did do it correctly. 29 30 There is also a lot of common code that could never be refactored due to 31 the lack of a framework. 32 33 So this framework sets up the basic building blocks that all drivers 34 need and this same framework should make it much easier to refactor 35 common code into utility functions shared by all drivers. 36 37 A good example to look at as a reference is the v4l2-pci-skeleton.c 38 source that is available in samples/v4l/. It is a skeleton driver for 39 a PCI capture card, and demonstrates how to use the V4L2 driver 40 framework. It can be used as a template for real PCI video capture driver. 41 42 Structure of a driver 43 --------------------- 44 45 All drivers have the following structure: 46 47 1) A struct for each device instance containing the device state. 48 49 2) A way of initializing and commanding sub-devices (if any). 50 51 3) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX and /dev/radioX) 52 and keeping track of device-node specific data. 53 54 4) Filehandle-specific structs containing per-filehandle data; 55 56 5) video buffer handling. 57 58 This is a rough schematic of how it all relates: 59 60 device instances 61 | 62 +-sub-device instances 63 | 64 \-V4L2 device nodes 65 | 66 \-filehandle instances 67 68 69 Structure of the framework 70 -------------------------- 71 72 The framework closely resembles the driver structure: it has a v4l2_device 73 struct for the device instance data, a v4l2_subdev struct to refer to 74 sub-device instances, the video_device struct stores V4L2 device node data 75 and the v4l2_fh struct keeps track of filehandle instances. 76 77 The V4L2 framework also optionally integrates with the media framework. If a 78 driver sets the struct v4l2_device mdev field, sub-devices and video nodes 79 will automatically appear in the media framework as entities. 80 81 82 struct v4l2_device 83 ------------------ 84 85 Each device instance is represented by a struct v4l2_device (v4l2-device.h). 86 Very simple devices can just allocate this struct, but most of the time you 87 would embed this struct inside a larger struct. 88 89 You must register the device instance: 90 91 v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev); 92 93 Registration will initialize the v4l2_device struct. If the dev->driver_data 94 field is NULL, it will be linked to v4l2_dev. 95 96 Drivers that want integration with the media device framework need to set 97 dev->driver_data manually to point to the driver-specific device structure 98 that embed the struct v4l2_device instance. This is achieved by a 99 dev_set_drvdata() call before registering the V4L2 device instance. They must 100 also set the struct v4l2_device mdev field to point to a properly initialized 101 and registered media_device instance. 102 103 If v4l2_dev->name is empty then it will be set to a value derived from dev 104 (driver name followed by the bus_id, to be precise). If you set it up before 105 calling v4l2_device_register then it will be untouched. If dev is NULL, then 106 you *must* setup v4l2_dev->name before calling v4l2_device_register. 107 108 You can use v4l2_device_set_name() to set the name based on a driver name and 109 a driver-global atomic_t instance. This will generate names like ivtv0, ivtv1, 110 etc. If the name ends with a digit, then it will insert a dash: cx18-0, 111 cx18-1, etc. This function returns the instance number. 112 113 The first 'dev' argument is normally the struct device pointer of a pci_dev, 114 usb_interface or platform_device. It is rare for dev to be NULL, but it happens 115 with ISA devices or when one device creates multiple PCI devices, thus making 116 it impossible to associate v4l2_dev with a particular parent. 117 118 You can also supply a notify() callback that can be called by sub-devices to 119 notify you of events. Whether you need to set this depends on the sub-device. 120 Any notifications a sub-device supports must be defined in a header in 121 include/media/<subdevice>.h. 122 123 You unregister with: 124 125 v4l2_device_unregister(struct v4l2_device *v4l2_dev); 126 127 If the dev->driver_data field points to v4l2_dev, it will be reset to NULL. 128 Unregistering will also automatically unregister all subdevs from the device. 129 130 If you have a hotpluggable device (e.g. a USB device), then when a disconnect 131 happens the parent device becomes invalid. Since v4l2_device has a pointer to 132 that parent device it has to be cleared as well to mark that the parent is 133 gone. To do this call: 134 135 v4l2_device_disconnect(struct v4l2_device *v4l2_dev); 136 137 This does *not* unregister the subdevs, so you still need to call the 138 v4l2_device_unregister() function for that. If your driver is not hotpluggable, 139 then there is no need to call v4l2_device_disconnect(). 140 141 Sometimes you need to iterate over all devices registered by a specific 142 driver. This is usually the case if multiple device drivers use the same 143 hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv 144 hardware. The same is true for alsa drivers for example. 145 146 You can iterate over all registered devices as follows: 147 148 static int callback(struct device *dev, void *p) 149 { 150 struct v4l2_device *v4l2_dev = dev_get_drvdata(dev); 151 152 /* test if this device was inited */ 153 if (v4l2_dev == NULL) 154 return 0; 155 ... 156 return 0; 157 } 158 159 int iterate(void *p) 160 { 161 struct device_driver *drv; 162 int err; 163 164 /* Find driver 'ivtv' on the PCI bus. 165 pci_bus_type is a global. For USB busses use usb_bus_type. */ 166 drv = driver_find("ivtv", &pci_bus_type); 167 /* iterate over all ivtv device instances */ 168 err = driver_for_each_device(drv, NULL, p, callback); 169 put_driver(drv); 170 return err; 171 } 172 173 Sometimes you need to keep a running counter of the device instance. This is 174 commonly used to map a device instance to an index of a module option array. 175 176 The recommended approach is as follows: 177 178 static atomic_t drv_instance = ATOMIC_INIT(0); 179 180 static int drv_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id) 181 { 182 ... 183 state->instance = atomic_inc_return(&drv_instance) - 1; 184 } 185 186 If you have multiple device nodes then it can be difficult to know when it is 187 safe to unregister v4l2_device for hotpluggable devices. For this purpose 188 v4l2_device has refcounting support. The refcount is increased whenever 189 video_register_device is called and it is decreased whenever that device node 190 is released. When the refcount reaches zero, then the v4l2_device release() 191 callback is called. You can do your final cleanup there. 192 193 If other device nodes (e.g. ALSA) are created, then you can increase and 194 decrease the refcount manually as well by calling: 195 196 void v4l2_device_get(struct v4l2_device *v4l2_dev); 197 198 or: 199 200 int v4l2_device_put(struct v4l2_device *v4l2_dev); 201 202 Since the initial refcount is 1 you also need to call v4l2_device_put in the 203 disconnect() callback (for USB devices) or in the remove() callback (for e.g. 204 PCI devices), otherwise the refcount will never reach 0. 205 206 struct v4l2_subdev 207 ------------------ 208 209 Many drivers need to communicate with sub-devices. These devices can do all 210 sort of tasks, but most commonly they handle audio and/or video muxing, 211 encoding or decoding. For webcams common sub-devices are sensors and camera 212 controllers. 213 214 Usually these are I2C devices, but not necessarily. In order to provide the 215 driver with a consistent interface to these sub-devices the v4l2_subdev struct 216 (v4l2-subdev.h) was created. 217 218 Each sub-device driver must have a v4l2_subdev struct. This struct can be 219 stand-alone for simple sub-devices or it might be embedded in a larger struct 220 if more state information needs to be stored. Usually there is a low-level 221 device struct (e.g. i2c_client) that contains the device data as setup 222 by the kernel. It is recommended to store that pointer in the private 223 data of v4l2_subdev using v4l2_set_subdevdata(). That makes it easy to go 224 from a v4l2_subdev to the actual low-level bus-specific device data. 225 226 You also need a way to go from the low-level struct to v4l2_subdev. For the 227 common i2c_client struct the i2c_set_clientdata() call is used to store a 228 v4l2_subdev pointer, for other busses you may have to use other methods. 229 230 Bridges might also need to store per-subdev private data, such as a pointer to 231 bridge-specific per-subdev private data. The v4l2_subdev structure provides 232 host private data for that purpose that can be accessed with 233 v4l2_get_subdev_hostdata() and v4l2_set_subdev_hostdata(). 234 235 From the bridge driver perspective you load the sub-device module and somehow 236 obtain the v4l2_subdev pointer. For i2c devices this is easy: you call 237 i2c_get_clientdata(). For other busses something similar needs to be done. 238 Helper functions exists for sub-devices on an I2C bus that do most of this 239 tricky work for you. 240 241 Each v4l2_subdev contains function pointers that sub-device drivers can 242 implement (or leave NULL if it is not applicable). Since sub-devices can do 243 so many different things and you do not want to end up with a huge ops struct 244 of which only a handful of ops are commonly implemented, the function pointers 245 are sorted according to category and each category has its own ops struct. 246 247 The top-level ops struct contains pointers to the category ops structs, which 248 may be NULL if the subdev driver does not support anything from that category. 249 250 It looks like this: 251 252 struct v4l2_subdev_core_ops { 253 int (*log_status)(struct v4l2_subdev *sd); 254 int (*init)(struct v4l2_subdev *sd, u32 val); 255 ... 256 }; 257 258 struct v4l2_subdev_tuner_ops { 259 ... 260 }; 261 262 struct v4l2_subdev_audio_ops { 263 ... 264 }; 265 266 struct v4l2_subdev_video_ops { 267 ... 268 }; 269 270 struct v4l2_subdev_pad_ops { 271 ... 272 }; 273 274 struct v4l2_subdev_ops { 275 const struct v4l2_subdev_core_ops *core; 276 const struct v4l2_subdev_tuner_ops *tuner; 277 const struct v4l2_subdev_audio_ops *audio; 278 const struct v4l2_subdev_video_ops *video; 279 const struct v4l2_subdev_pad_ops *video; 280 }; 281 282 The core ops are common to all subdevs, the other categories are implemented 283 depending on the sub-device. E.g. a video device is unlikely to support the 284 audio ops and vice versa. 285 286 This setup limits the number of function pointers while still making it easy 287 to add new ops and categories. 288 289 A sub-device driver initializes the v4l2_subdev struct using: 290 291 v4l2_subdev_init(sd, &ops); 292 293 Afterwards you need to initialize subdev->name with a unique name and set the 294 module owner. This is done for you if you use the i2c helper functions. 295 296 If integration with the media framework is needed, you must initialize the 297 media_entity struct embedded in the v4l2_subdev struct (entity field) by 298 calling media_entity_pads_init(), if the entity has pads: 299 300 struct media_pad *pads = &my_sd->pads; 301 int err; 302 303 err = media_entity_pads_init(&sd->entity, npads, pads); 304 305 The pads array must have been previously initialized. There is no need to 306 manually set the struct media_entity function and name fields, but the 307 revision field must be initialized if needed. 308 309 A reference to the entity will be automatically acquired/released when the 310 subdev device node (if any) is opened/closed. 311 312 Don't forget to cleanup the media entity before the sub-device is destroyed: 313 314 media_entity_cleanup(&sd->entity); 315 316 If the subdev driver intends to process video and integrate with the media 317 framework, it must implement format related functionality using 318 v4l2_subdev_pad_ops instead of v4l2_subdev_video_ops. 319 320 In that case, the subdev driver may set the link_validate field to provide 321 its own link validation function. The link validation function is called for 322 every link in the pipeline where both of the ends of the links are V4L2 323 sub-devices. The driver is still responsible for validating the correctness 324 of the format configuration between sub-devices and video nodes. 325 326 If link_validate op is not set, the default function 327 v4l2_subdev_link_validate_default() is used instead. This function ensures 328 that width, height and the media bus pixel code are equal on both source and 329 sink of the link. Subdev drivers are also free to use this function to 330 perform the checks mentioned above in addition to their own checks. 331 332 There are currently two ways to register subdevices with the V4L2 core. The 333 first (traditional) possibility is to have subdevices registered by bridge 334 drivers. This can be done when the bridge driver has the complete information 335 about subdevices connected to it and knows exactly when to register them. This 336 is typically the case for internal subdevices, like video data processing units 337 within SoCs or complex PCI(e) boards, camera sensors in USB cameras or connected 338 to SoCs, which pass information about them to bridge drivers, usually in their 339 platform data. 340 341 There are however also situations where subdevices have to be registered 342 asynchronously to bridge devices. An example of such a configuration is a Device 343 Tree based system where information about subdevices is made available to the 344 system independently from the bridge devices, e.g. when subdevices are defined 345 in DT as I2C device nodes. The API used in this second case is described further 346 below. 347 348 Using one or the other registration method only affects the probing process, the 349 run-time bridge-subdevice interaction is in both cases the same. 350 351 In the synchronous case a device (bridge) driver needs to register the 352 v4l2_subdev with the v4l2_device: 353 354 int err = v4l2_device_register_subdev(v4l2_dev, sd); 355 356 This can fail if the subdev module disappeared before it could be registered. 357 After this function was called successfully the subdev->dev field points to 358 the v4l2_device. 359 360 If the v4l2_device parent device has a non-NULL mdev field, the sub-device 361 entity will be automatically registered with the media device. 362 363 You can unregister a sub-device using: 364 365 v4l2_device_unregister_subdev(sd); 366 367 Afterwards the subdev module can be unloaded and sd->dev == NULL. 368 369 You can call an ops function either directly: 370 371 err = sd->ops->core->g_std(sd, &norm); 372 373 but it is better and easier to use this macro: 374 375 err = v4l2_subdev_call(sd, core, g_std, &norm); 376 377 The macro will to the right NULL pointer checks and returns -ENODEV if subdev 378 is NULL, -ENOIOCTLCMD if either subdev->core or subdev->core->g_std is 379 NULL, or the actual result of the subdev->ops->core->g_std ops. 380 381 It is also possible to call all or a subset of the sub-devices: 382 383 v4l2_device_call_all(v4l2_dev, 0, core, g_std, &norm); 384 385 Any subdev that does not support this ops is skipped and error results are 386 ignored. If you want to check for errors use this: 387 388 err = v4l2_device_call_until_err(v4l2_dev, 0, core, g_std, &norm); 389 390 Any error except -ENOIOCTLCMD will exit the loop with that error. If no 391 errors (except -ENOIOCTLCMD) occurred, then 0 is returned. 392 393 The second argument to both calls is a group ID. If 0, then all subdevs are 394 called. If non-zero, then only those whose group ID match that value will 395 be called. Before a bridge driver registers a subdev it can set sd->grp_id 396 to whatever value it wants (it's 0 by default). This value is owned by the 397 bridge driver and the sub-device driver will never modify or use it. 398 399 The group ID gives the bridge driver more control how callbacks are called. 400 For example, there may be multiple audio chips on a board, each capable of 401 changing the volume. But usually only one will actually be used when the 402 user want to change the volume. You can set the group ID for that subdev to 403 e.g. AUDIO_CONTROLLER and specify that as the group ID value when calling 404 v4l2_device_call_all(). That ensures that it will only go to the subdev 405 that needs it. 406 407 If the sub-device needs to notify its v4l2_device parent of an event, then 408 it can call v4l2_subdev_notify(sd, notification, arg). This macro checks 409 whether there is a notify() callback defined and returns -ENODEV if not. 410 Otherwise the result of the notify() call is returned. 411 412 The advantage of using v4l2_subdev is that it is a generic struct and does 413 not contain any knowledge about the underlying hardware. So a driver might 414 contain several subdevs that use an I2C bus, but also a subdev that is 415 controlled through GPIO pins. This distinction is only relevant when setting 416 up the device, but once the subdev is registered it is completely transparent. 417 418 419 In the asynchronous case subdevice probing can be invoked independently of the 420 bridge driver availability. The subdevice driver then has to verify whether all 421 the requirements for a successful probing are satisfied. This can include a 422 check for a master clock availability. If any of the conditions aren't satisfied 423 the driver might decide to return -EPROBE_DEFER to request further reprobing 424 attempts. Once all conditions are met the subdevice shall be registered using 425 the v4l2_async_register_subdev() function. Unregistration is performed using 426 the v4l2_async_unregister_subdev() call. Subdevices registered this way are 427 stored in a global list of subdevices, ready to be picked up by bridge drivers. 428 429 Bridge drivers in turn have to register a notifier object with an array of 430 subdevice descriptors that the bridge device needs for its operation. This is 431 performed using the v4l2_async_notifier_register() call. To unregister the 432 notifier the driver has to call v4l2_async_notifier_unregister(). The former of 433 the two functions takes two arguments: a pointer to struct v4l2_device and a 434 pointer to struct v4l2_async_notifier. The latter contains a pointer to an array 435 of pointers to subdevice descriptors of type struct v4l2_async_subdev type. The 436 V4L2 core will then use these descriptors to match asynchronously registered 437 subdevices to them. If a match is detected the .bound() notifier callback is 438 called. After all subdevices have been located the .complete() callback is 439 called. When a subdevice is removed from the system the .unbind() method is 440 called. All three callbacks are optional. 441 442 443 V4L2 sub-device userspace API 444 ----------------------------- 445 446 Beside exposing a kernel API through the v4l2_subdev_ops structure, V4L2 447 sub-devices can also be controlled directly by userspace applications. 448 449 Device nodes named v4l-subdevX can be created in /dev to access sub-devices 450 directly. If a sub-device supports direct userspace configuration it must set 451 the V4L2_SUBDEV_FL_HAS_DEVNODE flag before being registered. 452 453 After registering sub-devices, the v4l2_device driver can create device nodes 454 for all registered sub-devices marked with V4L2_SUBDEV_FL_HAS_DEVNODE by calling 455 v4l2_device_register_subdev_nodes(). Those device nodes will be automatically 456 removed when sub-devices are unregistered. 457 458 The device node handles a subset of the V4L2 API. 459 460 VIDIOC_QUERYCTRL 461 VIDIOC_QUERYMENU 462 VIDIOC_G_CTRL 463 VIDIOC_S_CTRL 464 VIDIOC_G_EXT_CTRLS 465 VIDIOC_S_EXT_CTRLS 466 VIDIOC_TRY_EXT_CTRLS 467 468 The controls ioctls are identical to the ones defined in V4L2. They 469 behave identically, with the only exception that they deal only with 470 controls implemented in the sub-device. Depending on the driver, those 471 controls can be also be accessed through one (or several) V4L2 device 472 nodes. 473 474 VIDIOC_DQEVENT 475 VIDIOC_SUBSCRIBE_EVENT 476 VIDIOC_UNSUBSCRIBE_EVENT 477 478 The events ioctls are identical to the ones defined in V4L2. They 479 behave identically, with the only exception that they deal only with 480 events generated by the sub-device. Depending on the driver, those 481 events can also be reported by one (or several) V4L2 device nodes. 482 483 Sub-device drivers that want to use events need to set the 484 V4L2_SUBDEV_USES_EVENTS v4l2_subdev::flags and initialize 485 v4l2_subdev::nevents to events queue depth before registering the 486 sub-device. After registration events can be queued as usual on the 487 v4l2_subdev::devnode device node. 488 489 To properly support events, the poll() file operation is also 490 implemented. 491 492 Private ioctls 493 494 All ioctls not in the above list are passed directly to the sub-device 495 driver through the core::ioctl operation. 496 497 498 I2C sub-device drivers 499 ---------------------- 500 501 Since these drivers are so common, special helper functions are available to 502 ease the use of these drivers (v4l2-common.h). 503 504 The recommended method of adding v4l2_subdev support to an I2C driver is to 505 embed the v4l2_subdev struct into the state struct that is created for each 506 I2C device instance. Very simple devices have no state struct and in that case 507 you can just create a v4l2_subdev directly. 508 509 A typical state struct would look like this (where 'chipname' is replaced by 510 the name of the chip): 511 512 struct chipname_state { 513 struct v4l2_subdev sd; 514 ... /* additional state fields */ 515 }; 516 517 Initialize the v4l2_subdev struct as follows: 518 519 v4l2_i2c_subdev_init(&state->sd, client, subdev_ops); 520 521 This function will fill in all the fields of v4l2_subdev and ensure that the 522 v4l2_subdev and i2c_client both point to one another. 523 524 You should also add a helper inline function to go from a v4l2_subdev pointer 525 to a chipname_state struct: 526 527 static inline struct chipname_state *to_state(struct v4l2_subdev *sd) 528 { 529 return container_of(sd, struct chipname_state, sd); 530 } 531 532 Use this to go from the v4l2_subdev struct to the i2c_client struct: 533 534 struct i2c_client *client = v4l2_get_subdevdata(sd); 535 536 And this to go from an i2c_client to a v4l2_subdev struct: 537 538 struct v4l2_subdev *sd = i2c_get_clientdata(client); 539 540 Make sure to call v4l2_device_unregister_subdev(sd) when the remove() callback 541 is called. This will unregister the sub-device from the bridge driver. It is 542 safe to call this even if the sub-device was never registered. 543 544 You need to do this because when the bridge driver destroys the i2c adapter 545 the remove() callbacks are called of the i2c devices on that adapter. 546 After that the corresponding v4l2_subdev structures are invalid, so they 547 have to be unregistered first. Calling v4l2_device_unregister_subdev(sd) 548 from the remove() callback ensures that this is always done correctly. 549 550 551 The bridge driver also has some helper functions it can use: 552 553 struct v4l2_subdev *sd = v4l2_i2c_new_subdev(v4l2_dev, adapter, 554 "module_foo", "chipid", 0x36, NULL); 555 556 This loads the given module (can be NULL if no module needs to be loaded) and 557 calls i2c_new_device() with the given i2c_adapter and chip/address arguments. 558 If all goes well, then it registers the subdev with the v4l2_device. 559 560 You can also use the last argument of v4l2_i2c_new_subdev() to pass an array 561 of possible I2C addresses that it should probe. These probe addresses are 562 only used if the previous argument is 0. A non-zero argument means that you 563 know the exact i2c address so in that case no probing will take place. 564 565 Both functions return NULL if something went wrong. 566 567 Note that the chipid you pass to v4l2_i2c_new_subdev() is usually 568 the same as the module name. It allows you to specify a chip variant, e.g. 569 "saa7114" or "saa7115". In general though the i2c driver autodetects this. 570 The use of chipid is something that needs to be looked at more closely at a 571 later date. It differs between i2c drivers and as such can be confusing. 572 To see which chip variants are supported you can look in the i2c driver code 573 for the i2c_device_id table. This lists all the possibilities. 574 575 There are two more helper functions: 576 577 v4l2_i2c_new_subdev_cfg: this function adds new irq and platform_data 578 arguments and has both 'addr' and 'probed_addrs' arguments: if addr is not 579 0 then that will be used (non-probing variant), otherwise the probed_addrs 580 are probed. 581 582 For example: this will probe for address 0x10: 583 584 struct v4l2_subdev *sd = v4l2_i2c_new_subdev_cfg(v4l2_dev, adapter, 585 "module_foo", "chipid", 0, NULL, 0, I2C_ADDRS(0x10)); 586 587 v4l2_i2c_new_subdev_board uses an i2c_board_info struct which is passed 588 to the i2c driver and replaces the irq, platform_data and addr arguments. 589 590 If the subdev supports the s_config core ops, then that op is called with 591 the irq and platform_data arguments after the subdev was setup. The older 592 v4l2_i2c_new_(probed_)subdev functions will call s_config as well, but with 593 irq set to 0 and platform_data set to NULL. 594 595 struct video_device 596 ------------------- 597 598 The actual device nodes in the /dev directory are created using the 599 video_device struct (v4l2-dev.h). This struct can either be allocated 600 dynamically or embedded in a larger struct. 601 602 To allocate it dynamically use: 603 604 struct video_device *vdev = video_device_alloc(); 605 606 if (vdev == NULL) 607 return -ENOMEM; 608 609 vdev->release = video_device_release; 610 611 If you embed it in a larger struct, then you must set the release() 612 callback to your own function: 613 614 struct video_device *vdev = &my_vdev->vdev; 615 616 vdev->release = my_vdev_release; 617 618 The release callback must be set and it is called when the last user 619 of the video device exits. 620 621 The default video_device_release() callback just calls kfree to free the 622 allocated memory. 623 624 There is also a video_device_release_empty() function that does nothing 625 (is empty) and can be used if the struct is embedded and there is nothing 626 to do when it is released. 627 628 You should also set these fields: 629 630 - v4l2_dev: must be set to the v4l2_device parent device. 631 632 - name: set to something descriptive and unique. 633 634 - vfl_dir: set this to VFL_DIR_RX for capture devices (VFL_DIR_RX has value 0, 635 so this is normally already the default), set to VFL_DIR_TX for output 636 devices and VFL_DIR_M2M for mem2mem (codec) devices. 637 638 - fops: set to the v4l2_file_operations struct. 639 640 - ioctl_ops: if you use the v4l2_ioctl_ops to simplify ioctl maintenance 641 (highly recommended to use this and it might become compulsory in the 642 future!), then set this to your v4l2_ioctl_ops struct. The vfl_type and 643 vfl_dir fields are used to disable ops that do not match the type/dir 644 combination. E.g. VBI ops are disabled for non-VBI nodes, and output ops 645 are disabled for a capture device. This makes it possible to provide 646 just one v4l2_ioctl_ops struct for both vbi and video nodes. 647 648 - lock: leave to NULL if you want to do all the locking in the driver. 649 Otherwise you give it a pointer to a struct mutex_lock and before the 650 unlocked_ioctl file operation is called this lock will be taken by the 651 core and released afterwards. See the next section for more details. 652 653 - queue: a pointer to the struct vb2_queue associated with this device node. 654 If queue is non-NULL, and queue->lock is non-NULL, then queue->lock is 655 used for the queuing ioctls (VIDIOC_REQBUFS, CREATE_BUFS, QBUF, DQBUF, 656 QUERYBUF, PREPARE_BUF, STREAMON and STREAMOFF) instead of the lock above. 657 That way the vb2 queuing framework does not have to wait for other ioctls. 658 This queue pointer is also used by the vb2 helper functions to check for 659 queuing ownership (i.e. is the filehandle calling it allowed to do the 660 operation). 661 662 - prio: keeps track of the priorities. Used to implement VIDIOC_G/S_PRIORITY. 663 If left to NULL, then it will use the struct v4l2_prio_state in v4l2_device. 664 If you want to have a separate priority state per (group of) device node(s), 665 then you can point it to your own struct v4l2_prio_state. 666 667 - dev_parent: you only set this if v4l2_device was registered with NULL as 668 the parent device struct. This only happens in cases where one hardware 669 device has multiple PCI devices that all share the same v4l2_device core. 670 671 The cx88 driver is an example of this: one core v4l2_device struct, but 672 it is used by both a raw video PCI device (cx8800) and a MPEG PCI device 673 (cx8802). Since the v4l2_device cannot be associated with two PCI devices 674 at the same time it is setup without a parent device. But when the struct 675 video_device is initialized you *do* know which parent PCI device to use and 676 so you set dev_device to the correct PCI device. 677 678 If you use v4l2_ioctl_ops, then you should set .unlocked_ioctl to video_ioctl2 679 in your v4l2_file_operations struct. 680 681 Do not use .ioctl! This is deprecated and will go away in the future. 682 683 In some cases you want to tell the core that a function you had specified in 684 your v4l2_ioctl_ops should be ignored. You can mark such ioctls by calling this 685 function before video_device_register is called: 686 687 void v4l2_disable_ioctl(struct video_device *vdev, unsigned int cmd); 688 689 This tends to be needed if based on external factors (e.g. which card is 690 being used) you want to turns off certain features in v4l2_ioctl_ops without 691 having to make a new struct. 692 693 The v4l2_file_operations struct is a subset of file_operations. The main 694 difference is that the inode argument is omitted since it is never used. 695 696 If integration with the media framework is needed, you must initialize the 697 media_entity struct embedded in the video_device struct (entity field) by 698 calling media_entity_pads_init(): 699 700 struct media_pad *pad = &my_vdev->pad; 701 int err; 702 703 err = media_entity_pads_init(&vdev->entity, 1, pad); 704 705 The pads array must have been previously initialized. There is no need to 706 manually set the struct media_entity type and name fields. 707 708 A reference to the entity will be automatically acquired/released when the 709 video device is opened/closed. 710 711 ioctls and locking 712 ------------------ 713 714 The V4L core provides optional locking services. The main service is the 715 lock field in struct video_device, which is a pointer to a mutex. If you set 716 this pointer, then that will be used by unlocked_ioctl to serialize all ioctls. 717 718 If you are using the videobuf2 framework, then there is a second lock that you 719 can set: video_device->queue->lock. If set, then this lock will be used instead 720 of video_device->lock to serialize all queuing ioctls (see the previous section 721 for the full list of those ioctls). 722 723 The advantage of using a different lock for the queuing ioctls is that for some 724 drivers (particularly USB drivers) certain commands such as setting controls 725 can take a long time, so you want to use a separate lock for the buffer queuing 726 ioctls. That way your VIDIOC_DQBUF doesn't stall because the driver is busy 727 changing the e.g. exposure of the webcam. 728 729 Of course, you can always do all the locking yourself by leaving both lock 730 pointers at NULL. 731 732 If you use the old videobuf then you must pass the video_device lock to the 733 videobuf queue initialize function: if videobuf has to wait for a frame to 734 arrive, then it will temporarily unlock the lock and relock it afterwards. If 735 your driver also waits in the code, then you should do the same to allow other 736 processes to access the device node while the first process is waiting for 737 something. 738 739 In the case of videobuf2 you will need to implement the wait_prepare and 740 wait_finish callbacks to unlock/lock if applicable. If you use the queue->lock 741 pointer, then you can use the helper functions vb2_ops_wait_prepare/finish. 742 743 The implementation of a hotplug disconnect should also take the lock from 744 video_device before calling v4l2_device_disconnect. If you are also using 745 video_device->queue->lock, then you have to first lock video_device->queue->lock 746 followed by video_device->lock. That way you can be sure no ioctl is running 747 when you call v4l2_device_disconnect. 748 749 video_device registration 750 ------------------------- 751 752 Next you register the video device: this will create the character device 753 for you. 754 755 err = video_register_device(vdev, VFL_TYPE_GRABBER, -1); 756 if (err) { 757 video_device_release(vdev); /* or kfree(my_vdev); */ 758 return err; 759 } 760 761 If the v4l2_device parent device has a non-NULL mdev field, the video device 762 entity will be automatically registered with the media device. 763 764 Which device is registered depends on the type argument. The following 765 types exist: 766 767 VFL_TYPE_GRABBER: videoX for video input/output devices 768 VFL_TYPE_VBI: vbiX for vertical blank data (i.e. closed captions, teletext) 769 VFL_TYPE_RADIO: radioX for radio tuners 770 VFL_TYPE_SDR: swradioX for Software Defined Radio tuners 771 772 The last argument gives you a certain amount of control over the device 773 device node number used (i.e. the X in videoX). Normally you will pass -1 774 to let the v4l2 framework pick the first free number. But sometimes users 775 want to select a specific node number. It is common that drivers allow 776 the user to select a specific device node number through a driver module 777 option. That number is then passed to this function and video_register_device 778 will attempt to select that device node number. If that number was already 779 in use, then the next free device node number will be selected and it 780 will send a warning to the kernel log. 781 782 Another use-case is if a driver creates many devices. In that case it can 783 be useful to place different video devices in separate ranges. For example, 784 video capture devices start at 0, video output devices start at 16. 785 So you can use the last argument to specify a minimum device node number 786 and the v4l2 framework will try to pick the first free number that is equal 787 or higher to what you passed. If that fails, then it will just pick the 788 first free number. 789 790 Since in this case you do not care about a warning about not being able 791 to select the specified device node number, you can call the function 792 video_register_device_no_warn() instead. 793 794 Whenever a device node is created some attributes are also created for you. 795 If you look in /sys/class/video4linux you see the devices. Go into e.g. 796 video0 and you will see 'name', 'dev_debug' and 'index' attributes. The 'name' 797 attribute is the 'name' field of the video_device struct. The 'dev_debug' attribute 798 can be used to enable core debugging. See the next section for more detailed 799 information on this. 800 801 The 'index' attribute is the index of the device node: for each call to 802 video_register_device() the index is just increased by 1. The first video 803 device node you register always starts with index 0. 804 805 Users can setup udev rules that utilize the index attribute to make fancy 806 device names (e.g. 'mpegX' for MPEG video capture device nodes). 807 808 After the device was successfully registered, then you can use these fields: 809 810 - vfl_type: the device type passed to video_register_device. 811 - minor: the assigned device minor number. 812 - num: the device node number (i.e. the X in videoX). 813 - index: the device index number. 814 815 If the registration failed, then you need to call video_device_release() 816 to free the allocated video_device struct, or free your own struct if the 817 video_device was embedded in it. The vdev->release() callback will never 818 be called if the registration failed, nor should you ever attempt to 819 unregister the device if the registration failed. 820 821 video device debugging 822 ---------------------- 823 824 The 'dev_debug' attribute that is created for each video, vbi, radio or swradio 825 device in /sys/class/video4linux/<devX>/ allows you to enable logging of 826 file operations. 827 828 It is a bitmask and the following bits can be set: 829 830 0x01: Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are only logged 831 if bit 0x08 is also set. 832 0x02: Log the ioctl name arguments and error code. VIDIOC_(D)QBUF ioctls are 833 only logged if bit 0x08 is also set. 834 0x04: Log the file operations open, release, read, write, mmap and 835 get_unmapped_area. The read and write operations are only logged if 836 bit 0x08 is also set. 837 0x08: Log the read and write file operations and the VIDIOC_QBUF and 838 VIDIOC_DQBUF ioctls. 839 0x10: Log the poll file operation. 840 841 video_device cleanup 842 -------------------- 843 844 When the video device nodes have to be removed, either during the unload 845 of the driver or because the USB device was disconnected, then you should 846 unregister them: 847 848 video_unregister_device(vdev); 849 850 This will remove the device nodes from sysfs (causing udev to remove them 851 from /dev). 852 853 After video_unregister_device() returns no new opens can be done. However, 854 in the case of USB devices some application might still have one of these 855 device nodes open. So after the unregister all file operations (except 856 release, of course) will return an error as well. 857 858 When the last user of the video device node exits, then the vdev->release() 859 callback is called and you can do the final cleanup there. 860 861 Don't forget to cleanup the media entity associated with the video device if 862 it has been initialized: 863 864 media_entity_cleanup(&vdev->entity); 865 866 This can be done from the release callback. 867 868 869 video_device helper functions 870 ----------------------------- 871 872 There are a few useful helper functions: 873 874 - file/video_device private data 875 876 You can set/get driver private data in the video_device struct using: 877 878 void *video_get_drvdata(struct video_device *vdev); 879 void video_set_drvdata(struct video_device *vdev, void *data); 880 881 Note that you can safely call video_set_drvdata() before calling 882 video_register_device(). 883 884 And this function: 885 886 struct video_device *video_devdata(struct file *file); 887 888 returns the video_device belonging to the file struct. 889 890 The video_drvdata function combines video_get_drvdata with video_devdata: 891 892 void *video_drvdata(struct file *file); 893 894 You can go from a video_device struct to the v4l2_device struct using: 895 896 struct v4l2_device *v4l2_dev = vdev->v4l2_dev; 897 898 - Device node name 899 900 The video_device node kernel name can be retrieved using 901 902 const char *video_device_node_name(struct video_device *vdev); 903 904 The name is used as a hint by userspace tools such as udev. The function 905 should be used where possible instead of accessing the video_device::num and 906 video_device::minor fields. 907 908 909 video buffer helper functions 910 ----------------------------- 911 912 The v4l2 core API provides a set of standard methods (called "videobuf") 913 for dealing with video buffers. Those methods allow a driver to implement 914 read(), mmap() and overlay() in a consistent way. There are currently 915 methods for using video buffers on devices that supports DMA with 916 scatter/gather method (videobuf-dma-sg), DMA with linear access 917 (videobuf-dma-contig), and vmalloced buffers, mostly used on USB drivers 918 (videobuf-vmalloc). 919 920 Please see Documentation/video4linux/videobuf for more information on how 921 to use the videobuf layer. 922 923 struct v4l2_fh 924 -------------- 925 926 struct v4l2_fh provides a way to easily keep file handle specific data 927 that is used by the V4L2 framework. New drivers must use struct v4l2_fh 928 since it is also used to implement priority handling (VIDIOC_G/S_PRIORITY). 929 930 The users of v4l2_fh (in the V4L2 framework, not the driver) know 931 whether a driver uses v4l2_fh as its file->private_data pointer by 932 testing the V4L2_FL_USES_V4L2_FH bit in video_device->flags. This bit is 933 set whenever v4l2_fh_init() is called. 934 935 struct v4l2_fh is allocated as a part of the driver's own file handle 936 structure and file->private_data is set to it in the driver's open 937 function by the driver. 938 939 In many cases the struct v4l2_fh will be embedded in a larger structure. 940 In that case you should call v4l2_fh_init+v4l2_fh_add in open() and 941 v4l2_fh_del+v4l2_fh_exit in release(). 942 943 Drivers can extract their own file handle structure by using the container_of 944 macro. Example: 945 946 struct my_fh { 947 int blah; 948 struct v4l2_fh fh; 949 }; 950 951 ... 952 953 int my_open(struct file *file) 954 { 955 struct my_fh *my_fh; 956 struct video_device *vfd; 957 int ret; 958 959 ... 960 961 my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL); 962 963 ... 964 965 v4l2_fh_init(&my_fh->fh, vfd); 966 967 ... 968 969 file->private_data = &my_fh->fh; 970 v4l2_fh_add(&my_fh->fh); 971 return 0; 972 } 973 974 int my_release(struct file *file) 975 { 976 struct v4l2_fh *fh = file->private_data; 977 struct my_fh *my_fh = container_of(fh, struct my_fh, fh); 978 979 ... 980 v4l2_fh_del(&my_fh->fh); 981 v4l2_fh_exit(&my_fh->fh); 982 kfree(my_fh); 983 return 0; 984 } 985 986 Below is a short description of the v4l2_fh functions used: 987 988 void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev) 989 990 Initialise the file handle. This *MUST* be performed in the driver's 991 v4l2_file_operations->open() handler. 992 993 void v4l2_fh_add(struct v4l2_fh *fh) 994 995 Add a v4l2_fh to video_device file handle list. Must be called once the 996 file handle is completely initialized. 997 998 void v4l2_fh_del(struct v4l2_fh *fh) 999 1000 Unassociate the file handle from video_device(). The file handle 1001 exit function may now be called. 1002 1003 void v4l2_fh_exit(struct v4l2_fh *fh) 1004 1005 Uninitialise the file handle. After uninitialisation the v4l2_fh 1006 memory can be freed. 1007 1008 1009 If struct v4l2_fh is not embedded, then you can use these helper functions: 1010 1011 int v4l2_fh_open(struct file *filp) 1012 1013 This allocates a struct v4l2_fh, initializes it and adds it to the struct 1014 video_device associated with the file struct. 1015 1016 int v4l2_fh_release(struct file *filp) 1017 1018 This deletes it from the struct video_device associated with the file 1019 struct, uninitialised the v4l2_fh and frees it. 1020 1021 These two functions can be plugged into the v4l2_file_operation's open() and 1022 release() ops. 1023 1024 1025 Several drivers need to do something when the first file handle is opened and 1026 when the last file handle closes. Two helper functions were added to check 1027 whether the v4l2_fh struct is the only open filehandle of the associated 1028 device node: 1029 1030 int v4l2_fh_is_singular(struct v4l2_fh *fh) 1031 1032 Returns 1 if the file handle is the only open file handle, else 0. 1033 1034 int v4l2_fh_is_singular_file(struct file *filp) 1035 1036 Same, but it calls v4l2_fh_is_singular with filp->private_data. 1037 1038 1039 V4L2 events 1040 ----------- 1041 1042 The V4L2 events provide a generic way to pass events to user space. 1043 The driver must use v4l2_fh to be able to support V4L2 events. 1044 1045 Events are defined by a type and an optional ID. The ID may refer to a V4L2 1046 object such as a control ID. If unused, then the ID is 0. 1047 1048 When the user subscribes to an event the driver will allocate a number of 1049 kevent structs for that event. So every (type, ID) event tuple will have 1050 its own set of kevent structs. This guarantees that if a driver is generating 1051 lots of events of one type in a short time, then that will not overwrite 1052 events of another type. 1053 1054 But if you get more events of one type than the number of kevents that were 1055 reserved, then the oldest event will be dropped and the new one added. 1056 1057 Furthermore, the internal struct v4l2_subscribed_event has merge() and 1058 replace() callbacks which drivers can set. These callbacks are called when 1059 a new event is raised and there is no more room. The replace() callback 1060 allows you to replace the payload of the old event with that of the new event, 1061 merging any relevant data from the old payload into the new payload that 1062 replaces it. It is called when this event type has only one kevent struct 1063 allocated. The merge() callback allows you to merge the oldest event payload 1064 into that of the second-oldest event payload. It is called when there are two 1065 or more kevent structs allocated. 1066 1067 This way no status information is lost, just the intermediate steps leading 1068 up to that state. 1069 1070 A good example of these replace/merge callbacks is in v4l2-event.c: 1071 ctrls_replace() and ctrls_merge() callbacks for the control event. 1072 1073 Note: these callbacks can be called from interrupt context, so they must be 1074 fast. 1075 1076 Useful functions: 1077 1078 void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev) 1079 1080 Queue events to video device. The driver's only responsibility is to fill 1081 in the type and the data fields. The other fields will be filled in by 1082 V4L2. 1083 1084 int v4l2_event_subscribe(struct v4l2_fh *fh, 1085 struct v4l2_event_subscription *sub, unsigned elems, 1086 const struct v4l2_subscribed_event_ops *ops) 1087 1088 The video_device->ioctl_ops->vidioc_subscribe_event must check the driver 1089 is able to produce events with specified event id. Then it calls 1090 v4l2_event_subscribe() to subscribe the event. 1091 1092 The elems argument is the size of the event queue for this event. If it is 0, 1093 then the framework will fill in a default value (this depends on the event 1094 type). 1095 1096 The ops argument allows the driver to specify a number of callbacks: 1097 * add: called when a new listener gets added (subscribing to the same 1098 event twice will only cause this callback to get called once) 1099 * del: called when a listener stops listening 1100 * replace: replace event 'old' with event 'new'. 1101 * merge: merge event 'old' into event 'new'. 1102 All 4 callbacks are optional, if you don't want to specify any callbacks 1103 the ops argument itself maybe NULL. 1104 1105 int v4l2_event_unsubscribe(struct v4l2_fh *fh, 1106 struct v4l2_event_subscription *sub) 1107 1108 vidioc_unsubscribe_event in struct v4l2_ioctl_ops. A driver may use 1109 v4l2_event_unsubscribe() directly unless it wants to be involved in 1110 unsubscription process. 1111 1112 The special type V4L2_EVENT_ALL may be used to unsubscribe all events. The 1113 drivers may want to handle this in a special way. 1114 1115 int v4l2_event_pending(struct v4l2_fh *fh) 1116 1117 Returns the number of pending events. Useful when implementing poll. 1118 1119 Events are delivered to user space through the poll system call. The driver 1120 can use v4l2_fh->wait (a wait_queue_head_t) as the argument for poll_wait(). 1121 1122 There are standard and private events. New standard events must use the 1123 smallest available event type. The drivers must allocate their events from 1124 their own class starting from class base. Class base is 1125 V4L2_EVENT_PRIVATE_START + n * 1000 where n is the lowest available number. 1126 The first event type in the class is reserved for future use, so the first 1127 available event type is 'class base + 1'. 1128 1129 An example on how the V4L2 events may be used can be found in the OMAP 1130 3 ISP driver (drivers/media/platform/omap3isp). 1131 1132 A subdev can directly send an event to the v4l2_device notify function with 1133 V4L2_DEVICE_NOTIFY_EVENT. This allows the bridge to map the subdev that sends 1134 the event to the video node(s) associated with the subdev that need to be 1135 informed about such an event. 1136 1137 V4L2 clocks 1138 ----------- 1139 1140 Many subdevices, like camera sensors, TV decoders and encoders, need a clock 1141 signal to be supplied by the system. Often this clock is supplied by the 1142 respective bridge device. The Linux kernel provides a Common Clock Framework for 1143 this purpose. However, it is not (yet) available on all architectures. Besides, 1144 the nature of the multi-functional (clock, data + synchronisation, I2C control) 1145 connection of subdevices to the system might impose special requirements on the 1146 clock API usage. E.g. V4L2 has to support clock provider driver unregistration 1147 while a subdevice driver is holding a reference to the clock. For these reasons 1148 a V4L2 clock helper API has been developed and is provided to bridge and 1149 subdevice drivers. 1150 1151 The API consists of two parts: two functions to register and unregister a V4L2 1152 clock source: v4l2_clk_register() and v4l2_clk_unregister() and calls to control 1153 a clock object, similar to the respective generic clock API calls: 1154 v4l2_clk_get(), v4l2_clk_put(), v4l2_clk_enable(), v4l2_clk_disable(), 1155 v4l2_clk_get_rate(), and v4l2_clk_set_rate(). Clock suppliers have to provide 1156 clock operations that will be called when clock users invoke respective API 1157 methods. 1158 1159 It is expected that once the CCF becomes available on all relevant 1160 architectures this API will be removed.