Documentation/PCI/pci-error-recovery.rst

1 .. SPDX-License-Identifier: GPL-2.0
8 :Authors: - Linas Vepstas <linasvepstas@gmail.com>
9           - Richard Lary <rlary@us.ibm.com>
10           - Mike Mason <mmlnx@us.ibm.com>
14 PCI errors on the bus, such as parity errors on the data and address
16 chipsets are able to deal with these errors; these include PCI-E chipsets,
17 and the PCI-host bridges found on IBM Power4, Power5 and Power6-based
22 offered, so that the affected PCI device(s) are reset and put back
23 into working condition. The reset phase requires coordination
32 including multiple instances of a device driver on multi-function
34 waiting for some i/o-space register to change, when it never will.
39 is forced by the need to handle multi-function devices, that is,
42 of reset it desires, the choices being a simple re-enabling of I/O
43 or requesting a slot reset.
45 If any driver requests a slot reset, that is what will be done.
47 After a reset and/or a re-enabling of I/O, all drivers are
50 "resume normal operations" event is sent out.
52 The biggest reason for choosing a kernel-based implementation rather
53 than a user-space implementation was the need to deal with bus
56 file system is disconnected, a user-space mechanism would have to go
62 for example, the SCSI-generic layer already provides significant
69 Design and implementation details below, based on a chain of
74 pci_driver. A driver that fails to provide the structure is "non-aware",
85 		void (*resume)(struct pci_dev *dev);
100 		PCI_ERS_RESULT_CAN_RECOVER, /* Device driver can recover without slot reset */
101 		PCI_ERS_RESULT_NEED_RESET,  /* Device driver wants slot to be reset. */
109 For example, if mmio_enabled() and resume() aren't there, then it
111 a slot reset.  Typically a driver will want to know about
115 event will be platform-dependent, but will follow the general
119 -------------------
120 A PCI bus error is detected by the PCI hardware.  On powerpc, the slot
126 --------------------
127 Platform calls the error_detected() callback on every instance of
130 At this point, the device might not be accessible anymore, depending on
131 the platform (the slot will be isolated on powerpc). The driver may
143   - PCI_ERS_RESULT_CAN_RECOVER
148   - PCI_ERS_RESULT_NEED_RESET
150       slot reset.
151   - PCI_ERS_RESULT_DISCONNECT
154 The next step taken will depend on the result codes returned by the
157 If all drivers on the segment/slot return PCI_ERS_RESULT_CAN_RECOVER,
158 then the platform should re-enable IOs on the slot (or do nothing in
162 If any driver requested a slot reset (by returning PCI_ERS_RESULT_NEED_RESET),
163 then recovery proceeds to STEP 4 (Slot Reset).
174    Doing better requires complex multi-threaded logic in the error
181    a value of 0xff on read, and writes will be dropped. If more than
188 --------------------
189 The platform re-enables MMIO to the device (but typically not the
190 DMA), and then calls the mmio_enabled() callback on all affected
197 reset or some such, but not restart operations. This callback is made if
198 all drivers on a segment agree that they can try to recover and if no automatic
199 link reset was performed by the HW. If the platform can't just re-enable IOs
200 without a slot reset or a link reset, it will not call this callback, and
201 instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
210    such an error might cause IOs to be re-blocked for the whole
212    on the same segment might have done, forcing the whole segment
213    into one of the next states, that is, link reset or slot reset.
216   - PCI_ERS_RESULT_RECOVERED
221       allowed to proceed, as another driver on the
223       slot reset on platforms that support it.
225   - PCI_ERS_RESULT_NEED_RESET
227       recoverable in its current state and it needs a slot
228       reset to proceed.
230   - PCI_ERS_RESULT_DISCONNECT
232       reset driver dead. (To be defined more precisely)
234 The next step taken depends on the results returned by the drivers.
236 proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations).
239 proceeds to STEP 4 (Slot Reset)
241 STEP 3: Link Reset
242 ------------------
243 The platform resets the link.  This is a PCI-Express specific step
247 STEP 4: Slot Reset
248 ------------------
251 platform will perform a slot reset on the requesting PCI device(s).
252 The actual steps taken by a platform to perform a slot reset
253 will be platform-dependent. Upon completion of slot reset, the
256 Powerpc platforms implement two levels of slot reset:
257 soft reset(default) and fundamental(optional) reset.
259 Powerpc soft reset consists of asserting the adapter #RST line and then
262 power-on followed by power-on BIOS/system firmware initialization.
263 Soft reset is also known as hot-reset.
265 Powerpc fundamental reset is supported by PCI Express cards only
269 For most PCI devices, a soft reset will be sufficient for recovery.
270 Optional fundamental reset is provided to support a limited number
271 of PCI Express devices for which a soft reset is not sufficient
274 If the platform supports PCI hotplug, then the reset might be
275 performed by toggling the slot electrical power off/on.
279 a slot reset, the device driver will almost always use its standard
283 This call gives drivers the chance to re-initialize the hardware
284 (re-download firmware, etc.).  At this point, the driver may assume
287 memory mapped I/O space and DMA. Interrupts (Legacy, MSI, or MSI-X)
291 at this point.  If all device drivers report success on this
292 callback, the platform will call resume() to complete the sequence,
296 it can't get the device operational after reset.  If the platform
297 previously tried a soft reset, it might now try a hard reset (power
303 Drivers for multi-function cards will need to coordinate among
304 themselves as to which driver instance will perform any "one-shot"
308 	+       if (PCI_FUNC(pdev->devfn) == 0)
312 	- PCI_ERS_RESULT_DISCONNECT
315 Drivers for PCI Express cards that require a fundamental reset must
320 	+	/* Set EEH reset type to fundamental if required by hba  */
322 	+		pdev->needs_freset = 1;
325 Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent
330    The current powerpc implementation does not try a power-cycle
331    reset if the driver returned PCI_ERS_RESULT_DISCONNECT.
335 STEP 5: Resume Operations
336 -------------------------
337 The platform will call the resume() callback on all affected device
338 drivers if all drivers on the segment have returned
348 -------------------------
354 cancel all pending I/O, refuse all new I/O, returning -EIO to
360 permanent failure in some way.  If the device is hotplug-capable,
363 caused by over-heating, some by a poorly seated card. Many
366 errors. See the discussion in powerpc/eeh-pci-error-recovery.txt
367 for additional detail on real-life experience of the causes of
372 ---------------------------
374 no slot reset capability may want to just "ignore" drivers that can't
375 recover (disconnect them) and try to let other cards on the same segment
384  - There is no guarantee that interrupt delivery can proceed from any
385    device on the segment starting from the error detection and until the
389  - There is no guarantee that interrupt delivery is stopped, that is,
396    interrupts are routed to error-management capable slots and can deal
406    the file Documentation/powerpc/eeh-pci-error-recovery.rst
412    - drivers/scsi/ipr
413    - drivers/scsi/sym53c8xx_2
414    - drivers/scsi/qla2xxx
415    - drivers/scsi/lpfc
416    - drivers/next/bnx2.c
417    - drivers/next/e100.c
418    - drivers/net/e1000
419    - drivers/net/e1000e
420    - drivers/net/ixgb
421    - drivers/net/ixgbe
422    - drivers/net/cxgb3
423    - drivers/net/s2io.c
426 -------