1.. SPDX-License-Identifier: GPL-2.0 2 3==================== 4APEI Error INJection 5==================== 6 7EINJ provides a hardware error injection mechanism. It is very useful 8for debugging and testing APEI and RAS features in general. 9 10You need to check whether your BIOS supports EINJ first. For that, look 11for early boot messages similar to this one:: 12 13 ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL 00000001 INTL 00000001) 14 15which shows that the BIOS is exposing an EINJ table - it is the 16mechanism through which the injection is done. 17 18Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file, 19which is a different representation of the same thing. 20 21It doesn't necessarily mean that EINJ is not supported if those above 22don't exist: before you give up, go into BIOS setup to see if the BIOS 23has an option to enable error injection. Look for something called WHEA 24or similar. Often, you need to enable an ACPI5 support option prior, in 25order to see the APEI,EINJ,... functionality supported and exposed by 26the BIOS menu. 27 28To use EINJ, make sure the following are options enabled in your kernel 29configuration:: 30 31 CONFIG_DEBUG_FS 32 CONFIG_ACPI_APEI 33 CONFIG_ACPI_APEI_EINJ 34 35...and to (optionally) enable CXL protocol error injection set:: 36 37 CONFIG_ACPI_APEI_EINJ_CXL 38 39The EINJ user interface is in <debugfs mount point>/apei/einj. 40 41The following files belong to it: 42 43- available_error_type 44 45 This file shows which error types are supported: 46 47 ================ =================================== 48 Error Type Value Error Description 49 ================ =================================== 50 0x00000001 Processor Correctable 51 0x00000002 Processor Uncorrectable non-fatal 52 0x00000004 Processor Uncorrectable fatal 53 0x00000008 Memory Correctable 54 0x00000010 Memory Uncorrectable non-fatal 55 0x00000020 Memory Uncorrectable fatal 56 0x00000040 PCI Express Correctable 57 0x00000080 PCI Express Uncorrectable non-fatal 58 0x00000100 PCI Express Uncorrectable fatal 59 0x00000200 Platform Correctable 60 0x00000400 Platform Uncorrectable non-fatal 61 0x00000800 Platform Uncorrectable fatal 62 V2_0x00000001 EINJV2 Processor Error 63 V2_0x00000002 EINJV2 Memory Error 64 V2_0x00000004 EINJV2 PCI Express Error 65 ================ =================================== 66 67 The format of the file contents are as above, except present are only 68 the available error types. 69 70- error_type 71 72 Set the value of the error type being injected. Possible error types 73 are defined in the file available_error_type above. 74 75- error_inject 76 77 Write any integer to this file to trigger the error injection. Make 78 sure you have specified all necessary error parameters, i.e. this 79 write should be the last step when injecting errors. 80 81- flags 82 83 Present for kernel versions 3.13 and above. Used to specify which 84 of param{1..4} are valid and should be used by the firmware during 85 injection. Value is a bitmask as specified in ACPI5.0 spec for the 86 SET_ERROR_TYPE_WITH_ADDRESS data structure: 87 88 Bit 0 89 Processor APIC field valid (see param3 below). 90 Bit 1 91 Memory address and mask valid (param1 and param2). 92 Bit 2 93 PCIe (seg,bus,dev,fn) valid (see param4 below). 94 Bit 3 95 EINJv2 extension structure is valid 96 97 If set to zero, legacy behavior is mimicked where the type of 98 injection specifies just one bit set, and param1 is multiplexed. 99 100- param1 101 102 This file is used to set the first error parameter value. Its effect 103 depends on the error type specified in error_type. For example, if 104 error type is memory related type, the param1 should be a valid 105 physical memory address. [Unless "flag" is set - see above] 106 107- param2 108 109 Same use as param1 above. For example, if error type is of memory 110 related type, then param2 should be a physical memory address mask. 111 Linux requires page or narrower granularity, say, 0xfffffffffffff000. 112 113- param3 114 115 Used when the 0x1 bit is set in "flags" to specify the APIC id 116 117- param4 118 Used when the 0x4 bit is set in "flags" to specify target PCIe device 119 120- notrigger 121 122 The error injection mechanism is a two-step process. First inject the 123 error, then perform some actions to trigger it. Setting "notrigger" 124 to 1 skips the trigger phase, which *may* allow the user to cause the 125 error in some other context by a simple access to the CPU, memory 126 location, or device that is the target of the error injection. Whether 127 this actually works depends on what operations the BIOS actually 128 includes in the trigger phase. 129 130- component_id0 .. component_idN, component_syndrome0 .. component_syndromeN 131 132 These files are used to set the "Component Array" field 133 of the EINJv2 Extension Structure. Each holds a 128-bit 134 hex value. Writing just a newline to any of these files 135 sets an invalid (all-ones) value. 136 137CXL error types are supported from ACPI 6.5 onwards (given a CXL port 138is present). The EINJ user interface for CXL error types is at 139<debugfs mount point>/cxl. The following files belong to it: 140 141- einj_types: 142 143 Provides the same functionality as available_error_types above, but 144 for CXL error types 145 146- $dport_dev/einj_inject: 147 148 Injects a CXL error type into the CXL port represented by $dport_dev, 149 where $dport_dev is the name of the CXL port (usually a PCIe device name). 150 Error injections targeting a CXL 2.0+ port can use the legacy interface 151 under <debugfs mount point>/apei/einj, while CXL 1.1/1.0 port injections 152 must use this file. 153 154 155BIOS versions based on the ACPI 4.0 specification have limited options 156in controlling where the errors are injected. Your BIOS may support an 157extension (enabled with the param_extension=1 module parameter, or boot 158command line einj.param_extension=1). This allows the address and mask 159for memory injections to be specified by the param1 and param2 files in 160apei/einj. 161 162BIOS versions based on the ACPI 5.0 specification have more control over 163the target of the injection. For processor-related errors (type 0x1, 0x2 164and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and 165param2 for bit 1) so that you have more information added to the error 166signature being injected. The actual data passed is this:: 167 168 memory_address = param1; 169 memory_address_range = param2; 170 apicid = param3; 171 pcie_sbdf = param4; 172 173For memory errors (type 0x8, 0x10 and 0x20) the address is set using 174param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI 175express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and 176function are specified using param1:: 177 178 31 24 23 16 15 11 10 8 7 0 179 +-------------------------------------------------+ 180 | segment | bus | device | function | reserved | 181 +-------------------------------------------------+ 182 183Anyway, you get the idea, if there's doubt just take a look at the code 184in drivers/acpi/apei/einj.c. 185 186An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected. 187In this case a file named vendor will contain identifying information 188from the BIOS that hopefully will allow an application wishing to use 189the vendor-specific extension to tell that they are running on a BIOS 190that supports it. All vendor extensions have the 0x80000000 bit set in 191error_type. A file vendor_flags controls the interpretation of param1 192and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor 193documentation for details (and expect changes to this API if vendors 194creativity in using this feature expands beyond our expectations). 195 196 197An error injection example:: 198 199 # cd /sys/kernel/debug/apei/einj 200 # cat available_error_type # See which errors can be injected 201 0x00000002 Processor Uncorrectable non-fatal 202 0x00000008 Memory Correctable 203 0x00000010 Memory Uncorrectable non-fatal 204 # echo 0x12345000 > param1 # Set memory address for injection 205 # echo 0xfffffffffffff000 > param2 # Mask - anywhere in this page 206 # echo 0x8 > error_type # Choose correctable memory error 207 # echo 1 > error_inject # Inject now 208 209An EINJv2 error injection example:: 210 211 # cd /sys/kernel/debug/apei/einj 212 # cat available_error_type # See which errors can be injected 213 0x00000002 Processor Uncorrectable non-fatal 214 0x00000008 Memory Correctable 215 0x00000010 Memory Uncorrectable non-fatal 216 V2_0x00000001 EINJV2 Processor Error 217 V2_0x00000002 EINJV2 Memory Error 218 219 # echo 0x12345000 > param1 # Set memory address for injection 220 # echo 0xfffffffffffff000 > param2 # Range - anywhere in this page 221 # echo 0x1 > component_id0 # First device ID 222 # echo 0x4 > component_syndrome0 # First error syndrome 223 # echo 0x2 > component_id1 # Second device ID 224 # echo 0x4 > component_syndrome1 # Second error syndrome 225 # echo '' > component_id2 # Mark id2 invalid to terminate list 226 # echo V2_0x2 > error_type # Choose EINJv2 memory error 227 # echo 0xa > flags # set flags to indicate EINJv2 228 # echo 1 > error_inject # Inject now 229 230You should see something like this in dmesg:: 231 232 [22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR 233 [22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090 234 [22715.834759] EDAC sbridge MC3: TSC 0 235 [22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86 236 [22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0 237 [22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0) 238 239A CXL error injection example with $dport_dev=0000:e0:01.1:: 240 241 # cd /sys/kernel/debug/cxl/ 242 # ls 243 0000:e0:01.1 0000:0c:00.0 244 # cat einj_types # See which errors can be injected 245 0x00008000 CXL.mem Protocol Correctable 246 0x00010000 CXL.mem Protocol Uncorrectable non-fatal 247 0x00020000 CXL.mem Protocol Uncorrectable fatal 248 # cd 0000:e0:01.1 # Navigate to dport to inject into 249 # echo 0x8000 > einj_inject # Inject error 250 251Special notes for injection into SGX enclaves: 252 253There may be a separate BIOS setup option to enable SGX injection. 254 255The injection process consists of setting some special memory controller 256trigger that will inject the error on the next write to the target 257address. But the h/w prevents any software outside of an SGX enclave 258from accessing enclave pages (even BIOS SMM mode). 259 260The following sequence can be used: 261 1) Determine physical address of enclave page 262 2) Use "notrigger=1" mode to inject (this will setup 263 the injection address, but will not actually inject) 264 3) Enter the enclave 265 4) Store data to the virtual address matching physical address from step 1 266 5) Execute CLFLUSH for that virtual address 267 6) Spin delay for 250ms 268 7) Read from the virtual address. This will trigger the error 269 270For more information about EINJ, please refer to ACPI specification 271version 4.0, section 17.5 and ACPI 5.0, section 18.6. 272