Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.
1 Linux for S/390 and zSeries 2 3 Common Device Support (CDS) 4 Device Driver I/O Support Routines 5 6 Authors : Ingo Adlung 7 Cornelia Huck 8 9 Copyright, IBM Corp. 1999-2002 10 11 Introduction 12 13 This document describes the common device support routines for Linux/390. 14 Different than other hardware architectures, ESA/390 has defined a unified 15 I/O access method. This gives relief to the device drivers as they don't 16 have to deal with different bus types, polling versus interrupt 17 processing, shared versus non-shared interrupt processing, DMA versus port 18 I/O (PIO), and other hardware features more. However, this implies that 19 either every single device driver needs to implement the hardware I/O 20 attachment functionality itself, or the operating system provides for a 21 unified method to access the hardware, providing all the functionality that 22 every single device driver would have to provide itself. 23 24 The document does not intend to explain the ESA/390 hardware architecture in 25 every detail.This information can be obtained from the ESA/390 Principles of 26 Operation manual (IBM Form. No. SA22-7201). 27 28 In order to build common device support for ESA/390 I/O interfaces, a 29 functional layer was introduced that provides generic I/O access methods to 30 the hardware. 31 32 The common device support layer comprises the I/O support routines defined 33 below. Some of them implement common Linux device driver interfaces, while 34 some of them are ESA/390 platform specific. 35 36 Note: 37 In order to write a driver for S/390, you also need to look into the interface 38 described in Documentation/s390/driver-model.txt. 39 40 Note for porting drivers from 2.4: 41 The major changes are: 42 * The functions use a ccw_device instead of an irq (subchannel). 43 * All drivers must define a ccw_driver (see driver-model.txt) and the associated 44 functions. 45 * request_irq() and free_irq() are no longer done by the driver. 46 * The oper_handler is (kindof) replaced by the probe() and set_online() functions 47 of the ccw_driver. 48 * The not_oper_handler is (kindof) replaced by the remove() and set_offline() 49 functions of the ccw_driver. 50 * The channel device layer is gone. 51 * The interrupt handlers must be adapted to use a ccw_device as argument. 52 Moreover, they don't return a devstat, but an irb. 53 * Before initiating an io, the options must be set via ccw_device_set_options(). 54 * Instead of calling read_dev_chars()/read_conf_data(), the driver issues 55 the channel program and handles the interrupt itself. 56 57 ccw_device_get_ciw() 58 get commands from extended sense data. 59 60 ccw_device_start() 61 ccw_device_start_timeout() 62 ccw_device_start_key() 63 ccw_device_start_key_timeout() 64 initiate an I/O request. 65 66 ccw_device_resume() 67 resume channel program execution. 68 69 ccw_device_halt() 70 terminate the current I/O request processed on the device. 71 72 do_IRQ() 73 generic interrupt routine. This function is called by the interrupt entry 74 routine whenever an I/O interrupt is presented to the system. The do_IRQ() 75 routine determines the interrupt status and calls the device specific 76 interrupt handler according to the rules (flags) defined during I/O request 77 initiation with do_IO(). 78 79 The next chapters describe the functions other than do_IRQ() in more details. 80 The do_IRQ() interface is not described, as it is called from the Linux/390 81 first level interrupt handler only and does not comprise a device driver 82 callable interface. Instead, the functional description of do_IO() also 83 describes the input to the device specific interrupt handler. 84 85 Note: All explanations apply also to the 64 bit architecture s390x. 86 87 88 Common Device Support (CDS) for Linux/390 Device Drivers 89 90 General Information 91 92 The following chapters describe the I/O related interface routines the 93 Linux/390 common device support (CDS) provides to allow for device specific 94 driver implementations on the IBM ESA/390 hardware platform. Those interfaces 95 intend to provide the functionality required by every device driver 96 implementation to allow to drive a specific hardware device on the ESA/390 97 platform. Some of the interface routines are specific to Linux/390 and some 98 of them can be found on other Linux platforms implementations too. 99 Miscellaneous function prototypes, data declarations, and macro definitions 100 can be found in the architecture specific C header file 101 linux/arch/s390/include/asm/irq.h. 102 103 Overview of CDS interface concepts 104 105 Different to other hardware platforms, the ESA/390 architecture doesn't define 106 interrupt lines managed by a specific interrupt controller and bus systems 107 that may or may not allow for shared interrupts, DMA processing, etc.. Instead, 108 the ESA/390 architecture has implemented a so called channel subsystem, that 109 provides a unified view of the devices physically attached to the systems. 110 Though the ESA/390 hardware platform knows about a huge variety of different 111 peripheral attachments like disk devices (aka. DASDs), tapes, communication 112 controllers, etc. they can all be accessed by a well defined access method and 113 they are presenting I/O completion a unified way : I/O interruptions. Every 114 single device is uniquely identified to the system by a so called subchannel, 115 where the ESA/390 architecture allows for 64k devices be attached. 116 117 Linux, however, was first built on the Intel PC architecture, with its two 118 cascaded 8259 programmable interrupt controllers (PICs), that allow for a 119 maximum of 15 different interrupt lines. All devices attached to such a system 120 share those 15 interrupt levels. Devices attached to the ISA bus system must 121 not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered 122 interrupts. MCA, EISA, PCI and other bus systems base on level triggered 123 interrupts, and therewith allow for shared IRQs. However, if multiple devices 124 present their hardware status by the same (shared) IRQ, the operating system 125 has to call every single device driver registered on this IRQ in order to 126 determine the device driver owning the device that raised the interrupt. 127 128 Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel). 129 For internal use of the common I/O layer, these are still there. However, 130 device drivers should use the new calling interface via the ccw_device only. 131 132 During its startup the Linux/390 system checks for peripheral devices. Each 133 of those devices is uniquely defined by a so called subchannel by the ESA/390 134 channel subsystem. While the subchannel numbers are system generated, each 135 subchannel also takes a user defined attribute, the so called device number. 136 Both subchannel number and device number cannot exceed 65535. During sysfs 137 initialisation, the information about control unit type and device types that 138 imply specific I/O commands (channel command words - CCWs) in order to operate 139 the device are gathered. Device drivers can retrieve this set of hardware 140 information during their initialization step to recognize the devices they 141 support using the information saved in the struct ccw_device given to them. 142 This methods implies that Linux/390 doesn't require to probe for free (not 143 armed) interrupt request lines (IRQs) to drive its devices with. Where 144 applicable, the device drivers can use issue the READ DEVICE CHARACTERISTICS 145 ccw to retrieve device characteristics in its online routine. 146 147 In order to allow for easy I/O initiation the CDS layer provides a 148 ccw_device_start() interface that takes a device specific channel program (one 149 or more CCWs) as input sets up the required architecture specific control blocks 150 and initiates an I/O request on behalf of the device driver. The 151 ccw_device_start() routine allows to specify whether it expects the CDS layer 152 to notify the device driver for every interrupt it observes, or with final status 153 only. See ccw_device_start() for more details. A device driver must never issue 154 ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead. 155 156 For long running I/O request to be canceled, the CDS layer provides the 157 ccw_device_halt() function. Some devices require to initially issue a HALT 158 SUBCHANNEL (HSCH) command without having pending I/O requests. This function is 159 also covered by ccw_device_halt(). 160 161 162 get_ciw() - get command information word 163 164 This call enables a device driver to get information about supported commands 165 from the extended SenseID data. 166 167 struct ciw * 168 ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd); 169 170 cdev - The ccw_device for which the command is to be retrieved. 171 cmd - The command type to be retrieved. 172 173 ccw_device_get_ciw() returns: 174 NULL - No extended data available, invalid device or command not found. 175 !NULL - The command requested. 176 177 178 ccw_device_start() - Initiate I/O Request 179 180 The ccw_device_start() routines is the I/O request front-end processor. All 181 device driver I/O requests must be issued using this routine. A device driver 182 must not issue ESA/390 I/O commands itself. Instead the ccw_device_start() 183 routine provides all interfaces required to drive arbitrary devices. 184 185 This description also covers the status information passed to the device 186 driver's interrupt handler as this is related to the rules (flags) defined 187 with the associated I/O request when calling ccw_device_start(). 188 189 int ccw_device_start(struct ccw_device *cdev, 190 struct ccw1 *cpa, 191 unsigned long intparm, 192 __u8 lpm, 193 unsigned long flags); 194 int ccw_device_start_timeout(struct ccw_device *cdev, 195 struct ccw1 *cpa, 196 unsigned long intparm, 197 __u8 lpm, 198 unsigned long flags, 199 int expires); 200 int ccw_device_start_key(struct ccw_device *cdev, 201 struct ccw1 *cpa, 202 unsigned long intparm, 203 __u8 lpm, 204 __u8 key, 205 unsigned long flags); 206 int ccw_device_start_key_timeout(struct ccw_device *cdev, 207 struct ccw1 *cpa, 208 unsigned long intparm, 209 __u8 lpm, 210 __u8 key, 211 unsigned long flags, 212 int expires); 213 214 cdev : ccw_device the I/O is destined for 215 cpa : logical start address of channel program 216 user_intparm : user specific interrupt information; will be presented 217 back to the device driver's interrupt handler. Allows a 218 device driver to associate the interrupt with a 219 particular I/O request. 220 lpm : defines the channel path to be used for a specific I/O 221 request. A value of 0 will make cio use the opm. 222 key : the storage key to use for the I/O (useful for operating on a 223 storage with a storage key != default key) 224 flag : defines the action to be performed for I/O processing 225 expires : timeout value in jiffies. The common I/O layer will terminate 226 the running program after this and call the interrupt handler 227 with ERR_PTR(-ETIMEDOUT) as irb. 228 229 Possible flag values are : 230 231 DOIO_ALLOW_SUSPEND - channel program may become suspended 232 DOIO_DENY_PREFETCH - don't allow for CCW prefetch; usually 233 this implies the channel program might 234 become modified 235 DOIO_SUPPRESS_INTER - don't call the handler on intermediate status 236 237 The cpa parameter points to the first format 1 CCW of a channel program : 238 239 struct ccw1 { 240 __u8 cmd_code;/* command code */ 241 __u8 flags; /* flags, like IDA addressing, etc. */ 242 __u16 count; /* byte count */ 243 __u32 cda; /* data address */ 244 } __attribute__ ((packed,aligned(8))); 245 246 with the following CCW flags values defined : 247 248 CCW_FLAG_DC - data chaining 249 CCW_FLAG_CC - command chaining 250 CCW_FLAG_SLI - suppress incorrect length 251 CCW_FLAG_SKIP - skip 252 CCW_FLAG_PCI - PCI 253 CCW_FLAG_IDA - indirect addressing 254 CCW_FLAG_SUSPEND - suspend 255 256 257 Via ccw_device_set_options(), the device driver may specify the following 258 options for the device: 259 260 DOIO_EARLY_NOTIFICATION - allow for early interrupt notification 261 DOIO_REPORT_ALL - report all interrupt conditions 262 263 264 The ccw_device_start() function returns : 265 266 0 - successful completion or request successfully initiated 267 -EBUSY - The device is currently processing a previous I/O request, or there is 268 a status pending at the device. 269 -ENODEV - cdev is invalid, the device is not operational or the ccw_device is 270 not online. 271 272 When the I/O request completes, the CDS first level interrupt handler will 273 accumulate the status in a struct irb and then call the device interrupt handler. 274 The intparm field will contain the value the device driver has associated with a 275 particular I/O request. If a pending device status was recognized, 276 intparm will be set to 0 (zero). This may happen during I/O initiation or delayed 277 by an alert status notification. In any case this status is not related to the 278 current (last) I/O request. In case of a delayed status notification no special 279 interrupt will be presented to indicate I/O completion as the I/O request was 280 never started, even though ccw_device_start() returned with successful completion. 281 282 The irb may contain an error value, and the device driver should check for this 283 first: 284 285 -ETIMEDOUT: the common I/O layer terminated the request after the specified 286 timeout value 287 -EIO: the common I/O layer terminated the request due to an error state 288 289 If the concurrent sense flag in the extended status word (esw) in the irb is 290 set, the field erw.scnt in the esw describes the number of device specific 291 sense bytes available in the extended control word irb->scsw.ecw[]. No device 292 sensing by the device driver itself is required. 293 294 The device interrupt handler can use the following definitions to investigate 295 the primary unit check source coded in sense byte 0 : 296 297 SNS0_CMD_REJECT 0x80 298 SNS0_INTERVENTION_REQ 0x40 299 SNS0_BUS_OUT_CHECK 0x20 300 SNS0_EQUIPMENT_CHECK 0x10 301 SNS0_DATA_CHECK 0x08 302 SNS0_OVERRUN 0x04 303 SNS0_INCOMPL_DOMAIN 0x01 304 305 Depending on the device status, multiple of those values may be set together. 306 Please refer to the device specific documentation for details. 307 308 The irb->scsw.cstat field provides the (accumulated) subchannel status : 309 310 SCHN_STAT_PCI - program controlled interrupt 311 SCHN_STAT_INCORR_LEN - incorrect length 312 SCHN_STAT_PROG_CHECK - program check 313 SCHN_STAT_PROT_CHECK - protection check 314 SCHN_STAT_CHN_DATA_CHK - channel data check 315 SCHN_STAT_CHN_CTRL_CHK - channel control check 316 SCHN_STAT_INTF_CTRL_CHK - interface control check 317 SCHN_STAT_CHAIN_CHECK - chaining check 318 319 The irb->scsw.dstat field provides the (accumulated) device status : 320 321 DEV_STAT_ATTENTION - attention 322 DEV_STAT_STAT_MOD - status modifier 323 DEV_STAT_CU_END - control unit end 324 DEV_STAT_BUSY - busy 325 DEV_STAT_CHN_END - channel end 326 DEV_STAT_DEV_END - device end 327 DEV_STAT_UNIT_CHECK - unit check 328 DEV_STAT_UNIT_EXCEP - unit exception 329 330 Please see the ESA/390 Principles of Operation manual for details on the 331 individual flag meanings. 332 333 Usage Notes : 334 335 ccw_device_start() must be called disabled and with the ccw device lock held. 336 337 The device driver is allowed to issue the next ccw_device_start() call from 338 within its interrupt handler already. It is not required to schedule a 339 bottom-half, unless a non deterministically long running error recovery procedure 340 or similar needs to be scheduled. During I/O processing the Linux/390 generic 341 I/O device driver support has already obtained the IRQ lock, i.e. the handler 342 must not try to obtain it again when calling ccw_device_start() or we end in a 343 deadlock situation! 344 345 If a device driver relies on an I/O request to be completed prior to start the 346 next it can reduce I/O processing overhead by chaining a NoOp I/O command 347 CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End 348 and Device-End status to be presented together, with a single interrupt. 349 However, this should be used with care as it implies the channel will remain 350 busy, not being able to process I/O requests for other devices on the same 351 channel. Therefore e.g. read commands should never use this technique, as the 352 result will be presented by a single interrupt anyway. 353 354 In order to minimize I/O overhead, a device driver should use the 355 DOIO_REPORT_ALL only if the device can report intermediate interrupt 356 information prior to device-end the device driver urgently relies on. In this 357 case all I/O interruptions are presented to the device driver until final 358 status is recognized. 359 360 If a device is able to recover from asynchronously presented I/O errors, it can 361 perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some 362 devices always report channel-end and device-end together, with a single 363 interrupt, others present primary status (channel-end) when the channel is 364 ready for the next I/O request and secondary status (device-end) when the data 365 transmission has been completed at the device. 366 367 Above flag allows to exploit this feature, e.g. for communication devices that 368 can handle lost data on the network to allow for enhanced I/O processing. 369 370 Unless the channel subsystem at any time presents a secondary status interrupt, 371 exploiting this feature will cause only primary status interrupts to be 372 presented to the device driver while overlapping I/O is performed. When a 373 secondary status without error (alert status) is presented, this indicates 374 successful completion for all overlapping ccw_device_start() requests that have 375 been issued since the last secondary (final) status. 376 377 Channel programs that intend to set the suspend flag on a channel command word 378 (CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the 379 suspend flag will cause a channel program check. At the time the channel program 380 becomes suspended an intermediate interrupt will be generated by the channel 381 subsystem. 382 383 ccw_device_resume() - Resume Channel Program Execution 384 385 If a device driver chooses to suspend the current channel program execution by 386 setting the CCW suspend flag on a particular CCW, the channel program execution 387 is suspended. In order to resume channel program execution the CIO layer 388 provides the ccw_device_resume() routine. 389 390 int ccw_device_resume(struct ccw_device *cdev); 391 392 cdev - ccw_device the resume operation is requested for 393 394 The ccw_device_resume() function returns: 395 396 0 - suspended channel program is resumed 397 -EBUSY - status pending 398 -ENODEV - cdev invalid or not-operational subchannel 399 -EINVAL - resume function not applicable 400 -ENOTCONN - there is no I/O request pending for completion 401 402 Usage Notes: 403 Please have a look at the ccw_device_start() usage notes for more details on 404 suspended channel programs. 405 406 ccw_device_halt() - Halt I/O Request Processing 407 408 Sometimes a device driver might need a possibility to stop the processing of 409 a long-running channel program or the device might require to initially issue 410 a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt() 411 command is provided. 412 413 ccw_device_halt() must be called disabled and with the ccw device lock held. 414 415 int ccw_device_halt(struct ccw_device *cdev, 416 unsigned long intparm); 417 418 cdev : ccw_device the halt operation is requested for 419 intparm : interruption parameter; value is only used if no I/O 420 is outstanding, otherwise the intparm associated with 421 the I/O request is returned 422 423 The ccw_device_halt() function returns : 424 425 0 - request successfully initiated 426 -EBUSY - the device is currently busy, or status pending. 427 -ENODEV - cdev invalid. 428 -EINVAL - The device is not operational or the ccw device is not online. 429 430 Usage Notes : 431 432 A device driver may write a never-ending channel program by writing a channel 433 program that at its end loops back to its beginning by means of a transfer in 434 channel (TIC) command (CCW_CMD_TIC). Usually this is performed by network 435 device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is 436 executed a program controlled interrupt (PCI) is generated. The device driver 437 can then perform an appropriate action. Prior to interrupt of an outstanding 438 read to a network device (with or without PCI flag) a ccw_device_halt() 439 is required to end the pending operation. 440 441 ccw_device_clear() - Terminage I/O Request Processing 442 443 In order to terminate all I/O processing at the subchannel, the clear subchannel 444 (CSCH) command is used. It can be issued via ccw_device_clear(). 445 446 ccw_device_clear() must be called disabled and with the ccw device lock held. 447 448 int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm); 449 450 cdev: ccw_device the clear operation is requested for 451 intparm: interruption parameter (see ccw_device_halt()) 452 453 The ccw_device_clear() function returns: 454 455 0 - request successfully initiated 456 -ENODEV - cdev invalid 457 -EINVAL - The device is not operational or the ccw device is not online. 458 459 Miscellaneous Support Routines 460 461 This chapter describes various routines to be used in a Linux/390 device 462 driver programming environment. 463 464 get_ccwdev_lock() 465 466 Get the address of the device specific lock. This is then used in 467 spin_lock() / spin_unlock() calls. 468 469 470 __u8 ccw_device_get_path_mask(struct ccw_device *cdev); 471 472 Get the mask of the path currently available for cdev.