atf_python/vnet: Make it possible to set the FIB of vnet interfaces

Reviewed by: zlei, ngie
MFC after: 1 week
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://revie

atf_python/vnet: Make it possible to set the FIB of vnet interfaces

Reviewed by: zlei, ngie
MFC after: 1 week
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D54972

show more ...

atf_python: Run vnet handlers in $HOME

When kyua runs a test, it creates a temp directory and sets $HOME to
point to it. Tests are run with the cwd set to that temp directory.

When a process attac

atf_python: Run vnet handlers in $HOME

When kyua runs a test, it creates a temp directory and sets $HOME to
point to it. Tests are run with the cwd set to that temp directory.

When a process attaches to a jail, its cwd is set to the root of the
jail. Modify atf_python to cd to $HOME instead, so that it's easier for
tests to share files.

Reviewed by: zlei, ngie
MFC after: 2 weeks
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D54971

show more ...

atf_python: support setting interface mtu

Teach the vnet support code to set interface MTU. Some tests make use of
this, so have the framework handle it.

Adapt a few pf tests to use this.

Reviewed

atf_python: support setting interface mtu

Teach the vnet support code to set interface MTU. Some tests make use of
this, so have the framework handle it.

Adapt a few pf tests to use this.

Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D54333

show more ...

atf_python: allow test scripts to pass jail options

Test scripts based on atf_python can now pass jail command options via the
'opts' key in the 'vnetX' key of TOPOLOGY.

Reviewed by: melifaro
MFC a

atf_python: allow test scripts to pass jail options

Test scripts based on atf_python can now pass jail command options via the
'opts' key in the 'vnetX' key of TOPOLOGY.

Reviewed by: melifaro
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D52761

show more ...

tests: disable transmit checksum offloading for epair interfaces

When transmit checksum offloading is enabled, if_epair does not
compute checksums, it just marks packets that this computation still

tests: disable transmit checksum offloading for epair interfaces

When transmit checksum offloading is enabled, if_epair does not
compute checksums, it just marks packets that this computation still
needs to be done. However, some test cases verify the checksum.
Therefore disable this for IPv4 and IPv6.

Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D52379

show more ...

tests: Get the MAC from the epairs.

This removes knowledge of the implementation of if_epair.
Makes it easier to modify if_epair in future commits.

Reviewed by: kp
Differential Revision: https://re

tests: Get the MAC from the epairs.

This removes knowledge of the implementation of if_epair.
Makes it easier to modify if_epair in future commits.

Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D51205

show more ...

aft_python: fix incorrect ndp use

If we want to clear a flag we have to include '--' before the flag we clear,
otherwise ndp will misparse our command (the getopt call will interpret it as
another o

aft_python: fix incorrect ndp use

If we want to clear a flag we have to include '--' before the flag we clear,
otherwise ndp will misparse our command (the getopt call will interpret it as
another option, and reject it).

This caused spurious errors at the end of every atf_python test case such as:

============================== 1 passed in 7.26s ===============================
usage: ndp [-nt] hostname
ndp [-nt] -a | -c | -p | -r | -H | -P | -R
ndp [-nt] -A wait
ndp [-nt] -d hostname
ndp [-nt] -f filename
ndp [-nt] -i interface [flags...]
ndp [-nt] -I [interface|delete]
ndp [-nt] -s nodename etheraddr [temp] [proxy]

Sponsored by: Rubicon Communications, LLC ("Netgate")

show more ...

vnet tests: verify that we can load if_epair and if_bridge

We're going to start running many of the vnet tests in nested jails (so they
can run in parallel). That means the tests won't be able to lo

vnet tests: verify that we can load if_epair and if_bridge

We're going to start running many of the vnet tests in nested jails (so they
can run in parallel). That means the tests won't be able to load kernel modules,
which we commonly do for if_epair and if_bridge.

Just assume that all vnet tests need this, because so many of them do that we
don't want to manually annotate all of them.
This is essentially a no-op on non-nested tests.

Do the same for the python test framework.

While here also have pflog_init actually call pft_init. While having pflog
loaded implies we have pf too pft_init also checks for vimage support, and now
for if_epair.

Reviewed by: markj
MFC after: 1 month
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D46039

show more ...

atf_python: Standardize custom sections

To be replaced with pytest's section/add_report_section.

Reviewed by: markj
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/865

atf_python: Set verbosity for commands in a vnet

Reviewed by: markj
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/865

atf_python: vnet: Use absolute paths within helpers

Usually tests are run in sterile environments; however, there is a
slight chance that the PATH overrides the utilities used for testing.

Pedantic

atf_python: vnet: Use absolute paths within helpers

Usually tests are run in sterile environments; however, there is a
slight chance that the PATH overrides the utilities used for testing.

Pedantically use absolute paths, even inside VNETs, to avoid ambiguity.

Chiefly, jexec -> /usr/sbin/jexec, and ifconfig -> /sbin/ifconfig.

Reviewed by: markj
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/865

show more ...

atf_python: Actually implement verbosity for run_cmd

Reviewed by: markj
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/865

atf_python: Do not attempt to automatically delete lo0

Else `ifconfig lo0 destroy` will throw an:

ifconfig: SIOCIFDESTROY: Invalid argument

Reviewed by: markj
MFC after: 1 week
Pull Request: h

atf_python: Do not attempt to automatically delete lo0

Else `ifconfig lo0 destroy` will throw an:

ifconfig: SIOCIFDESTROY: Invalid argument

Reviewed by: markj
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/865

show more ...

testing: improve vnet support in pytest
* Allow vnet object to be directly referenced
(self.vnet1 vs self.vnet_map["vnet1"])
* Allow iface object to be directly reference
(vnet.bridge vs vnet.iface

testing: improve vnet support in pytest
* Allow vnet object to be directly referenced
(self.vnet1 vs self.vnet_map["vnet1"])
* Allow iface object to be directly reference
(vnet.bridge vs vnet.iface_alias_map["bridge"])
* Allow arbitrary interface alias names insted of ifX
* Add wait_objects_any() method for waiting object from
multiple vnets
* Add wait() method for indefinite sleep on vnet handlers

MFC after: 2 weeks

show more ...

testing: add support for using custom interfaces in pytest framework.

MFC after: 2 weeks

testing: handling non-root users with VNETs in pytest-based tests.

Currently isolation and resource requirements are handled directly
by the kyua runner, based on the requirements specified by the

testing: handling non-root users with VNETs in pytest-based tests.

Currently isolation and resource requirements are handled directly
by the kyua runner, based on the requirements specified by the test.
It works well for simple tests, but may cause discrepancy with tests
doing complex pre-setups. For example, all tests that perform
VNET setups require root access to properly function.

This change adds additional handling of the "require_user" property
within the python testing framework. Specifically, it requests
root access if the test class signals its root requirements and
drops privileges to the desired user after performing the pre-setup.

Differential Revision: https://reviews.freebsd.org/D37923
MFC after: 2 weeks

show more ...

pytest: delete interfaces from inside the jail.

This change follows the approach used in 80fc25025ffc, to
minimise the impact of the delayed interface migration.

MFC after: 2 weeks

tests: Add an IPv4 loopback address of 127.0.0.1/8 to the lo0
interface by default when creating VNETSs using pytest.

Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D38021

testing: improve python vnet wrapper.

* Derive jail name from class name and method name, instead of just
method name. This change reduces the chances of different tests
clashing.
Old: 'jail_test_o

testing: improve python vnet wrapper.

* Derive jail name from class name and method name, instead of just
method name. This change reduces the chances of different tests
clashing.
Old: 'jail_test_one'. New: 'pytest:TestExampleSimplest:test_one'
* Simplify vnetX_handler() method signature by skipping obj_map (unused)
and pipe. The latter can be accessed as the vnet property.
* Add `send_object()` method as a pair to the `wait_object` inside the
VnetTestTemplate class.
* Add `test_id` property to the BaseTest method. Previously it was
provided only for the VnetTestTemplate class. This change makes
the identifier easily accessible for all users.

MFC after: 2 weeks

show more ...

netlink: connect netlink tests to the build

Reviewed By: ngie
Differential Revision: https://reviews.freebsd.org/D37708

tests: add routing tests for switching between same prefixes

Differential Revision: https://reviews.freebsd.org/D36055
MFC after: 2 weeks

testing: add ability to specify multi-vnet topologies in the pytest framework.

Notable amount of tests related to the packet IO require two VNET jails
for proper testing and avoiding side effects f

testing: add ability to specify multi-vnet topologies in the pytest framework.

Notable amount of tests related to the packet IO require two VNET jails
for proper testing and avoiding side effects for the host system.
Additionally, it is often required to run actions in the jails seme-sequentially
- waiting for the listener initialisation can be an example of such
dependency.

This change extends pytest vnet framework to allow defining multi-vnet
multi-epair topologies in declarative style, without any need to bother
about jail or repair names. All jail creation/teardown, interface
creation/teardown and address assignments are handled automatically.

Example:

TOPOLOGY = {
"vnet1": {"ifaces": ["if1", "if2", "if3"]},
"vnet2": {"ifaces": ["if1", "if2", "if3"]},
"if1": {"prefixes6": [("2001:db8:a::1/64", "2001:db8:a::2/64")]},
"if2": {"prefixes6": [("2001:db8:b::1/64", "2001:db8:b::2/64")]},
"if3": {"prefixes6": [("2001:db8:c::1/64", "2001:db8:c::2/64")]},
}
def vnet2_handler(self, vnet, obj_map, pipe):
ss = VerboseSocketServer("::", self.DEFAULT_PORT)
pipe.send("READY")

def test_output6_base(self):
self.wait_object(second_vnet.pipe)

The definitions above will create 2 vnets ("jail_test_output6_base",
"jail_test_output6_base_2"), 3 epairs, attached to both first and
second jails, set up the IP addresses for each epair, spawn another
process for vnet2_handler and pass control to vnet2_handler and
test_output6_base. Both processes can pass objects between each
other using pre-created pipes.

Differential Revision: https://reviews.freebsd.org/D35708

show more ...

testing: Add basic atf support to pytest.

Implementation consists of the pytest plugin implementing ATF format and
a simple C++ wrapper, which reorders the provided arguments from ATF format
to the

testing: Add basic atf support to pytest.

Implementation consists of the pytest plugin implementing ATF format and
a simple C++ wrapper, which reorders the provided arguments from ATF format
to the format understandable by pytest. Each test has this wrapper specified
after the shebang. When kyua executes the test, wrapper calls pytest, which
loads atf plugin, does the work and returns the result. Additionally, a
separate python "package", `/usr/tests/atf_python` has been added to collect
code that may be useful across different tests.

Current limitations:
* Opaque metadata passing via X-Name properties. Require some fixtures to write
* `-s srcdir` parameter passed by the runner is ignored.
* No `atf-c-api(3)` or similar - relying on pytest framework & existing python libraries
* No support for `atf_tc_<get|has>_config_var()` & `atf_tc_set_md_var()`.
Can be probably implemented with env variables & autoload fixtures

Differential Revision: https://reviews.freebsd.org/D31084
Reviewed by: kp, ngie

show more ...

atf_python: support setting interface mtu

Teach the vnet support code to set interface MTU. Some tests make use of
this, so have the framework handle it.

Adapt a few pf tests to use this.

Reviewed

atf_python: support setting interface mtu

Teach the vnet support code to set interface MTU. Some tests make use of
this, so have the framework handle it.

Adapt a few pf tests to use this.

Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D54333

show more ...

atf_python: allow test scripts to pass jail options

Test scripts based on atf_python can now pass jail command options via the
'opts' key in the 'vnetX' key of TOPOLOGY.

Reviewed by: melifaro
MFC a

atf_python: allow test scripts to pass jail options

Test scripts based on atf_python can now pass jail command options via the
'opts' key in the 'vnetX' key of TOPOLOGY.

Reviewed by: melifaro
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D52761

show more ...