userspace-api/fwctl/fwctl.rst

1 .. SPDX-License-Identifier: GPL-2.0
12 Modern devices contain extensive amounts of FW, and in many cases, are largely
13 software-defined pieces of hardware. The evolution of this approach is largely a
17 respin. Problems in the HW design can be counteracted in device FW. This is
21 The FW layer in devices has grown to incredible size and devices frequently
26 The availability of such a flexible layer has created quite a variety in the
27 industry where single pieces of silicon are now configurable software-defined
28 devices and can operate in substantially different ways depending on the need.
29 Further, we often see cases where specific sites wish to operate devices in ways
33 Further, devices have become multi-functional and integrated to the point they
34 no longer fit neatly into the kernel's division of subsystems. Modern
35 multi-functional devices have drivers, such as bnxt/ice/mlx5/pds, that span many
40 have an expansive FW environment that needs robust device-specific debugging
41 support, and FW-driven functionality that is not well suited to “generic”
43 user space in the areas of debuggability, management, and first-boot/nth-boot
48 In this case the driver will typically have some layer to deliver RPC messages
49 and collect RPC responses from device FW. The in-kernel subsystem drivers that
56 In the past when devices were more single function, individual subsystems would
59 provisioning, all have various unique interfaces across the kernel.
70 ---------------
73 It is not an avenue to access random kernel internals, or other operating system
76 fwctl instances must operate on a well-defined device function, and the device
77 should have a well-defined security model for what scope within the physical
79 device today may broadly have several function-level scopes:
81  1. A privileged function with full access to the on-device global state and
91 quite common in the VFIO world that the hypervisor environment has a complex
100     transparent or non-disruptive to any driver or VM.
102  2. Read-only access to function debug information that may report on FW objects
103     in the function & child, including FW objects owned by other kernel
107     the principles of kernel lockdown and kernel integrity protection. Triggers
108     a kernel Taint.
110  4. Full debug device access. Triggers a kernel Taint, requires CAP_SYS_RAWIO.
112 User space will provide a scope label on each RPC and the kernel must enforce the
113 above CAPs and taints based on that scope. A combination of kernel and FW can
114 enforce that RPCs are placed in the correct scope by user space.
117 ---------------
120 Taint or CAP), broadly derived from the principles of kernel lockdown. Some
127  2. Provide an abnormal “back door” to kernel drivers. No manipulation of kernel
128     objects owned by kernel drivers.
130  3. Directly configure or otherwise control kernel drivers. A subsystem kernel
134  4. Operate the HW in a way that overlaps with the core purpose of another
135     primary kernel subsystem, such as read/write to LBAs, send/receive of
141 Operations exposed through fwctl's non-taining interfaces should be fully
143 fwctl should never prevent a kernel subsystem from also concurrently using that
144 same RPC or hardware unit down the road. In such cases fwctl will be less
145 important than proper kernel subsystems that eventually emerge. Mistakes in this
146 area resulting in clashes will be resolved in favour of a kernel implementation.
151 .. kernel-doc:: include/uapi/fwctl/fwctl.h
152 .. kernel-doc:: include/uapi/fwctl/mlx5.h
153 .. kernel-doc:: include/uapi/fwctl/pds.h
156 -----------
162 fwctl devices can be related to driver components in other subsystems through
175 --------------------
177 Drawing inspiration from nvme-cli, participating in the kernel side must come
178 with a user space in a common TBD git tree, at a minimum to usefully operate the
179 kernel driver. Providing such an implementation is a pre-condition to merging a
180 kernel driver.
186  - Device in-field debugging
188  - HW provisioning
190  - VFIO child device profiling before VM boot
192  - Confidential Compute topics (attestation, secure provisioning)
194 that stretch across all subsystems in the kernel. fwupd is a great example of
195 how an excellent user space experience can emerge out of kernel-side diversity.
197 fwctl Kernel API
200 .. kernel-doc:: drivers/fwctl/main.c
202 .. kernel-doc:: include/linux/fwctl.h
205 -------------------
207 In many cases a fwctl driver is going to be part of a larger cross-subsystem
208 device possibly using the auxiliary_device mechanism. In that case several
210 device design must already provide for isolation and cooperation between kernel
216 done in the driver then the validation can read a 'command effects' report from
217 the device, or hardwire the enforcement. If the validation is done in the FW,
228 meet an expectation that userspace tools that are deployed and in significant
229 use don't needlessly break. FW upgrade and kernel upgrade should keep widely
241 The kernel remains the gatekeeper for this interface. If violations of the
243 devices fix them with a FW update, push a kernel patch to parse and block RPC
244 commands or push a kernel patch to block entire firmware versions/devices.
246 While the kernel can always directly parse and restrict RPCs, it is expected
247 that the existing kernel pattern of allowing drivers to delegate validation to
253 The approach described in this document is not a new idea. Direct, or near
254 direct device access has been offered by the kernel in different areas for
261  - HW RAID controllers. This includes RPCs to do things like compose drives into
264  - Baseboard managers. RPCs for configuring settings in the device and more
266  - NVMe vendor command capsules. nvme-cli provides access to some monitoring
269  - CXL also has a NVMe-like vendor command system.
271  - DRM allows user space drivers to send commands to the device via kernel
274  - RDMA allows user space drivers to directly push commands to the device
275    without kernel involvement
277  - Various “raw” APIs, raw HID (SDL2), raw USB, NVMe Generic Interface, etc.
281 of a kernel subsystem.
284 common user space project to use as a pre-condition for obtaining a kernel
285 driver. Developing good community around useful software in user space is key to