1.. SPDX-License-Identifier: GPL-2.0-or-later
2
3====================
4Kexec Handover Usage
5====================
6
7Kexec HandOver (KHO) is a mechanism that allows Linux to preserve memory
8regions, which could contain serialized system states, across kexec.
9
10This document expects that you are familiar with the base KHO
11:ref:`concepts <kho-concepts>`. If you have not read
12them yet, please do so now.
13
14Prerequisites
15=============
16
17KHO is available when the kernel is compiled with ``CONFIG_KEXEC_HANDOVER``
18set to y. Every KHO producer may have its own config option that you
19need to enable if you would like to preserve their respective state across
20kexec.
21
22To use KHO, please boot the kernel with the ``kho=on`` command line
23parameter. You may use ``kho_scratch`` parameter to define size of the
24scratch regions. For example ``kho_scratch=16M,512M,256M`` will reserve a
2516 MiB low memory scratch area, a 512 MiB global scratch region, and 256 MiB
26per NUMA node scratch regions on boot.
27
28Perform a KHO kexec
29===================
30
31First, before you perform a KHO kexec, you need to move the system into
32the :ref:`KHO finalization phase <kho-finalization-phase>` ::
33
34  $ echo 1 > /sys/kernel/debug/kho/out/finalize
35
36After this command, the KHO FDT is available in
37``/sys/kernel/debug/kho/out/fdt``. Other subsystems may also register
38their own preserved sub FDTs under
39``/sys/kernel/debug/kho/out/sub_fdts/``.
40
41Next, load the target payload and kexec into it. It is important that you
42use the ``-s`` parameter to use the in-kernel kexec file loader, as user
43space kexec tooling currently has no support for KHO with the user space
44based file loader ::
45
46  # kexec -l /path/to/bzImage --initrd /path/to/initrd -s
47  # kexec -e
48
49The new kernel will boot up and contain some of the previous kernel's state.
50
51For example, if you used ``reserve_mem`` command line parameter to create
52an early memory reservation, the new kernel will have that memory at the
53same physical address as the old kernel.
54
55Abort a KHO exec
56================
57
58You can move the system out of KHO finalization phase again by calling ::
59
60  $ echo 0 > /sys/kernel/debug/kho/out/active
61
62After this command, the KHO FDT is no longer available in
63``/sys/kernel/debug/kho/out/fdt``.
64
65debugfs Interfaces
66==================
67
68Currently KHO creates the following debugfs interfaces. Notice that these
69interfaces may change in the future. They will be moved to sysfs once KHO is
70stabilized.
71
72``/sys/kernel/debug/kho/out/finalize``
73    Kexec HandOver (KHO) allows Linux to transition the state of
74    compatible drivers into the next kexec'ed kernel. To do so,
75    device drivers will instruct KHO to preserve memory regions,
76    which could contain serialized kernel state.
77    While the state is serialized, they are unable to perform
78    any modifications to state that was serialized, such as
79    handed over memory allocations.
80
81    When this file contains "1", the system is in the transition
82    state. When contains "0", it is not. To switch between the
83    two states, echo the respective number into this file.
84
85``/sys/kernel/debug/kho/out/fdt``
86    When KHO state tree is finalized, the kernel exposes the
87    flattened device tree blob that carries its current KHO
88    state in this file. Kexec user space tooling can use this
89    as input file for the KHO payload image.
90
91``/sys/kernel/debug/kho/out/scratch_len``
92    Lengths of KHO scratch regions, which are physically contiguous
93    memory regions that will always stay available for future kexec
94    allocations. Kexec user space tools can use this file to determine
95    where it should place its payload images.
96
97``/sys/kernel/debug/kho/out/scratch_phys``
98    Physical locations of KHO scratch regions. Kexec user space tools
99    can use this file in conjunction to scratch_phys to determine where
100    it should place its payload images.
101
102``/sys/kernel/debug/kho/out/sub_fdts/``
103    In the KHO finalization phase, KHO producers register their own
104    FDT blob under this directory.
105
106``/sys/kernel/debug/kho/in/fdt``
107    When the kernel was booted with Kexec HandOver (KHO),
108    the state tree that carries metadata about the previous
109    kernel's state is in this file in the format of flattened
110    device tree. This file may disappear when all consumers of
111    it finished to interpret their metadata.
112
113``/sys/kernel/debug/kho/in/sub_fdts/``
114    Similar to ``kho/out/sub_fdts/``, but contains sub FDT blobs
115    of KHO producers passed from the old kernel.
116