1.. SPDX-License-Identifier: GPL-2.0 2 3Running driver tests 4==================== 5 6Networking driver tests are executed within kselftest framework like any 7other tests. They support testing both real device drivers and emulated / 8software drivers (latter mostly to test the core parts of the stack). 9 10SW mode 11~~~~~~~ 12 13By default, when no extra parameters are set or exported, tests execute 14against software drivers such as netdevsim. No extra preparation is required 15the software devices are created and destroyed as part of the test. 16In this mode the tests are indistinguishable from other selftests and 17(for example) can be run under ``virtme-ng`` like the core networking selftests. 18 19HW mode 20~~~~~~~ 21 22Executing tests against a real device requires external preparation. 23The netdevice against which tests will be run must exist, be running 24(in UP state) and be configured with an IP address. 25 26Refer to list of :ref:`Variables` later in this file to set up running 27the tests against a real device. 28 29The current support for bash tests restricts the use of the same interface name 30on the local system and the remote one and will bail if this case is 31encountered. 32 33Both modes required 34~~~~~~~~~~~~~~~~~~~ 35 36All tests in drivers/net must support running both against a software device 37and a real device. SW-only tests should instead be placed in net/ or 38drivers/net/netdevsim, HW-only tests in drivers/net/hw. 39 40Variables 41========= 42 43The variables can be set in the environment or by creating a net.config 44file in the same directory as this README file. Example:: 45 46 $ NETIF=eth0 ./some_test.sh 47 48or:: 49 50 $ cat tools/testing/selftests/drivers/net/net.config 51 # Variable set in a file 52 NETIF=eth0 53 54Please note that the config parser is very simple, if there are 55any non-alphanumeric characters in the value it needs to be in 56double quotes. 57 58Local test (which don't require endpoint for sending / receiving traffic) 59need only the ``NETIF`` variable. Remaining variables define the endpoint 60and communication method. 61 62NETIF 63~~~~~ 64 65Name of the netdevice against which the test should be executed. 66When empty or not set software devices will be used. 67 68LOCAL_V4, LOCAL_V6, REMOTE_V4, REMOTE_V6 69~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 70 71Local and remote endpoint IP addresses. 72 73LOCAL_PREFIX_V6 74~~~~~~~~~~~~~~~ 75 76Local IP prefix/subnet which can be used to allocate extra IP addresses (for 77network name spaces behind macvlan, veth, netkit devices). DUT must be 78reachable using these addresses from the endpoint. 79 80LOCAL_PREFIX_V6 must NOT match LOCAL_V6. 81 82Example: 83 NETIF = "eth0" 84 LOCAL_V6 = "2001:db8:1::1" 85 REMOTE_V6 = "2001:db8:1::2" 86 LOCAL_PREFIX_V6 = "2001:db8:2::0/64" 87 88 +-----------------------------+ +------------------------------+ 89 dst | INIT NS | | TEST NS | 90 2001: | +---------------+ | | | 91 db8:2::2| | NETIF | | bpf | | 92 +---|>| 2001:db8:1::1 | |redirect| +-------------------------+ | 93 | | | |-----------|--------|>| Netkit | | 94 | | +---------------+ | _peer | | nk_guest | | 95 | | +-------------+ Netkit pair | | | fe80::2/64 | | 96 | | | Netkit |.............|........|>| 2001:db8:2::2/64 | | 97 | | | nk_host | | | +-------------------------+ | 98 | | | fe80::1/64 | | | | 99 | | +-------------+ | | route: | 100 | | | | default | 101 | | route: | | via fe80::1 dev nk_guest | 102 | | 2001:db8:2::2/128 | +------------------------------+ 103 | | via fe80::2 dev nk_host | 104 | +-----------------------------+ 105 | 106 | +---------------+ 107 | | REMOTE | 108 +---| 2001:db8:1::2 | 109 +---------------+ 110 111REMOTE_TYPE 112~~~~~~~~~~~ 113 114Communication method used to run commands on the remote endpoint. 115Test framework has built-in support for ``netns`` and ``ssh`` channels. 116``netns`` assumes the "remote" interface is part of the same 117host, just moved to the specified netns. 118``ssh`` communicates with remote endpoint over ``ssh`` and ``scp``. 119Using persistent SSH connections is strongly encouraged to avoid 120the latency of SSH connection setup on every command. 121 122Communication methods are defined by classes in ``lib/py/remote_{name}.py``. 123It should be possible to add a new method without modifying any of 124the framework, by simply adding an appropriately named file to ``lib/py``. 125 126REMOTE_ARGS 127~~~~~~~~~~~ 128 129Arguments used to construct the communication channel. 130Communication channel dependent:: 131 132 for netns - name of the "remote" namespace 133 for ssh - name/address of the remote host 134 135Example 136======= 137 138Build the selftests:: 139 140 # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" 141 142"Install" the tests and copy them over to the target machine:: 143 144 # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" \ 145 install INSTALL_PATH=/tmp/ksft-net-drv 146 147 # rsync -ra --delete /tmp/ksft-net-drv root@192.168.1.1:/root/ 148 149On the target machine, running the tests will use netdevsim by default:: 150 151 [/root] # ./ksft-net-drv/run_kselftest.sh -t drivers/net:ping.py 152 TAP version 13 153 1..1 154 # timeout set to 45 155 # selftests: drivers/net: ping.py 156 # KTAP version 1 157 # 1..3 158 # ok 1 ping.test_v4 159 # ok 2 ping.test_v6 160 # ok 3 ping.test_tcp 161 # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0 162 ok 1 selftests: drivers/net: ping.py 163 164Create a config with remote info:: 165 166 [/root] # cat > ./ksft-net-drv/drivers/net/net.config <<EOF 167 NETIF=eth0 168 LOCAL_V4=192.168.1.1 169 REMOTE_V4=192.168.1.2 170 REMOTE_TYPE=ssh 171 REMOTE_ARGS=root@192.168.1.2 172 EOF 173 174Run the test:: 175 176 [/root] # ./ksft-net-drv/drivers/net/ping.py 177 KTAP version 1 178 1..3 179 ok 1 ping.test_v4 180 ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity 181 ok 3 ping.test_tcp 182 # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:1 error:0 183 184Dependencies 185~~~~~~~~~~~~ 186 187The tests have a handful of dependencies. For Fedora / CentOS:: 188 189 dnf -y install netsniff-ng python-yaml socat iperf3 190 191Guidance for test authors 192========================= 193 194This section mostly applies to Python tests but some of the guidance 195may be more broadly applicable. 196 197Kernel config 198~~~~~~~~~~~~~ 199 200Each test directory has a ``config`` file listing which kernel 201configuration options the tests depend on. This file must be kept 202up to date, the CIs build minimal kernels for each test group. 203 204Adding checks inside the tests to validate that the necessary kernel 205configs are enabled is discouraged. The test author may include such 206checks, but standalone patches to make tests compatible e.g. with 207distro kernel configs are unlikely to be accepted. 208 209Avoid libraries and frameworks 210~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 211 212Test files should be relatively self contained. The libraries should 213only include very core or non-trivial code. 214It may be tempting to "factor out" the common code, but fight that urge. 215Library code increases the barrier of entry, and complexity in general. 216 217Avoid mixing test code and boilerplate 218~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 219 220In Python, try to avoid adding code in the ``main()`` function which 221instantiates ``NetDrvEnv()`` and calls ``ksft_run()``. It's okay to 222set up global resources (e.g. open an RtNetlink socket used by multiple 223tests), but any complex logic, test-specific environment configuration 224and validation should be done in the tests (even if it means it has to 225be repeated). 226 227Local host is the DUT 228~~~~~~~~~~~~~~~~~~~~~ 229 230Dual-host tests (tests with an endpoint) should be written from the DUT 231perspective. IOW the local machine should be the one tested, remote is 232just for traffic generation. 233 234Avoid modifying remote 235~~~~~~~~~~~~~~~~~~~~~~ 236 237Avoid making configuration changes to the remote system as much as possible. 238Remote system may be used concurrently by multiple DUTs. 239 240defer() 241~~~~~~~ 242 243The env must be clean after test exits. Register a ``defer()`` for any 244action that needs an "undo" as soon as possible. If you need to run 245the cancel action as part of the test - ``defer()`` returns an object 246you can ``.exec()``-ute. 247 248ksft_pr() 249~~~~~~~~~ 250 251Use ``ksft_pr()`` instead of ``print()`` to avoid breaking TAP format. 252 253ksft_disruptive 254~~~~~~~~~~~~~~~ 255 256By default the tests are expected to be able to run on 257single-interface systems. All tests which may disconnect ``NETIF`` 258must be annotated with ``@ksft_disruptive``. 259 260ksft_variants 261~~~~~~~~~~~~~ 262 263Use the ``@ksft_variants`` decorator to run a test with multiple sets 264of inputs as separate test cases. This avoids duplicating test functions 265that only differ in parameters. 266 267Parameters can be a single value, a tuple, or a ``KsftNamedVariant`` 268(which gives an explicit name to the sub-case). The argument to the 269decorator can be a list or a generator. 270 271Example:: 272 273 @ksft_variants([ 274 KsftNamedVariant("main", False), 275 KsftNamedVariant("ctx", True), 276 ]) 277 def resize_periodic(cfg, create_context): 278 # test body receives (cfg, create_context) where create_context 279 # is False for the "main" variant and True for "ctx" 280 pass 281 282or:: 283 284 def _gro_variants(): 285 for mode in ["sw", "hw"]: 286 for protocol in ["tcp4", "tcp6"]: 287 yield (mode, protocol) 288 289 @ksft_variants(_gro_variants()) 290 def test(cfg, mode, protocol): 291 pass 292 293Running tests CI-style 294====================== 295 296See https://github.com/linux-netdev/nipa/wiki for instructions on how 297to easily run the tests using ``virtme-ng``. 298