1.. SPDX-License-Identifier: GPL-2.0
2.. _xfrm_device:
3
4===============================================
5XFRM device - offloading the IPsec computations
6===============================================
7
8Shannon Nelson <shannon.nelson@oracle.com>
9Leon Romanovsky <leonro@nvidia.com>
10
11
12Overview
13========
14
15IPsec is a useful feature for securing network traffic, but the
16computational cost is high: a 10Gbps link can easily be brought down
17to under 1Gbps, depending on the traffic and link configuration.
18Luckily, there are NICs that offer a hardware based IPsec offload which
19can radically increase throughput and decrease CPU utilization.  The XFRM
20Device interface allows NIC drivers to offer to the stack access to the
21hardware offload.
22
23Right now, there are two types of hardware offload that kernel supports.
24 * IPsec crypto offload:
25   * NIC performs encrypt/decrypt
26   * Kernel does everything else
27 * IPsec packet offload:
28   * NIC performs encrypt/decrypt
29   * NIC does encapsulation
30   * Kernel and NIC have SA and policy in-sync
31   * NIC handles the SA and policies states
32   * The Kernel talks to the keymanager
33
34Userland access to the offload is typically through a system such as
35libreswan or KAME/raccoon, but the iproute2 'ip xfrm' command set can
36be handy when experimenting.  An example command might look something
37like this for crypto offload:
38
39  ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
40     reqid 0x07 replay-window 32 \
41     aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
42     sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
43     offload dev eth4 dir in
44
45and for packet offload
46
47  ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
48     reqid 0x07 replay-window 32 \
49     aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
50     sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
51     offload packet dev eth4 dir in
52
53  ip x p add src 14.0.0.70 dst 14.0.0.52 offload packet dev eth4 dir in
54  tmpl src 14.0.0.70 dst 14.0.0.52 proto esp reqid 10000 mode transport
55
56Yes, that's ugly, but that's what shell scripts and/or libreswan are for.
57
58
59
60Callbacks to implement
61======================
62
63::
64
65  /* from include/linux/netdevice.h */
66  struct xfrmdev_ops {
67        /* Crypto and Packet offload callbacks */
68	int	(*xdo_dev_state_add)(struct net_device *dev,
69                                     struct xfrm_state *x,
70                                     struct netlink_ext_ack *extack);
71	void	(*xdo_dev_state_delete)(struct net_device *dev,
72                                        struct xfrm_state *x);
73	void	(*xdo_dev_state_free)(struct net_device *dev,
74                                      struct xfrm_state *x);
75	bool	(*xdo_dev_offload_ok) (struct sk_buff *skb,
76				       struct xfrm_state *x);
77	void    (*xdo_dev_state_advance_esn) (struct xfrm_state *x);
78	void    (*xdo_dev_state_update_stats) (struct xfrm_state *x);
79
80        /* Solely packet offload callbacks */
81	int	(*xdo_dev_policy_add) (struct xfrm_policy *x, struct netlink_ext_ack *extack);
82	void	(*xdo_dev_policy_delete) (struct xfrm_policy *x);
83	void	(*xdo_dev_policy_free) (struct xfrm_policy *x);
84  };
85
86The NIC driver offering ipsec offload will need to implement callbacks
87relevant to supported offload to make the offload available to the network
88stack's XFRM subsystem. Additionally, the feature bits NETIF_F_HW_ESP and
89NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload.
90
91
92
93Flow
94====
95
96At probe time and before the call to register_netdev(), the driver should
97set up local data structures and XFRM callbacks, and set the feature bits.
98The XFRM code's listener will finish the setup on NETDEV_REGISTER.
99
100::
101
102		adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
103		adapter->netdev->features |= NETIF_F_HW_ESP;
104		adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;
105
106When new SAs are set up with a request for "offload" feature, the
107driver's xdo_dev_state_add() will be given the new SA to be offloaded
108and an indication of whether it is for Rx or Tx.  The driver should
109
110	- verify the algorithm is supported for offloads
111	- store the SA information (key, salt, target-ip, protocol, etc)
112	- enable the HW offload of the SA
113	- return status value:
114
115		===========   ===================================
116		0             success
117		-EOPNETSUPP   offload not supported, try SW IPsec,
118                              not applicable for packet offload mode
119		other         fail the request
120		===========   ===================================
121
122The driver can also set an offload_handle in the SA, an opaque void pointer
123that can be used to convey context into the fast-path offload requests::
124
125		xs->xso.offload_handle = context;
126
127
128When the network stack is preparing an IPsec packet for an SA that has
129been setup for offload, it first calls into xdo_dev_offload_ok() with
130the skb and the intended offload state to ask the driver if the offload
131will serviceable.  This can check the packet information to be sure the
132offload can be supported (e.g. IPv4 or IPv6, no IPv4 options, etc) and
133return true or false to signify its support. In case driver doesn't implement
134this callback, the stack provides reasonable defaults.
135
136Crypto offload mode:
137When ready to send, the driver needs to inspect the Tx packet for the
138offload information, including the opaque context, and set up the packet
139send accordingly::
140
141		xs = xfrm_input_state(skb);
142		context = xs->xso.offload_handle;
143		set up HW for send
144
145The stack has already inserted the appropriate IPsec headers in the
146packet data, the offload just needs to do the encryption and fix up the
147header values.
148
149
150When a packet is received and the HW has indicated that it offloaded a
151decryption, the driver needs to add a reference to the decoded SA into
152the packet's skb.  At this point the data should be decrypted but the
153IPsec headers are still in the packet data; they are removed later up
154the stack in xfrm_input().
155
156	find and hold the SA that was used to the Rx skb::
157
158		get spi, protocol, and destination IP from packet headers
159		xs = find xs from (spi, protocol, dest_IP)
160		xfrm_state_hold(xs);
161
162	store the state information into the skb::
163
164		sp = secpath_set(skb);
165		if (!sp) return;
166		sp->xvec[sp->len++] = xs;
167		sp->olen++;
168
169	indicate the success and/or error status of the offload::
170
171		xo = xfrm_offload(skb);
172		xo->flags = CRYPTO_DONE;
173		xo->status = crypto_status;
174
175	hand the packet to napi_gro_receive() as usual
176
177In ESN mode, xdo_dev_state_advance_esn() is called from
178xfrm_replay_advance_esn() for RX, and xfrm_replay_overflow_offload_esn for TX.
179Driver will check packet seq number and update HW ESN state machine if needed.
180
181Packet offload mode:
182HW adds and deletes XFRM headers. So in RX path, XFRM stack is bypassed if HW
183reported success. In TX path, the packet lefts kernel without extra header
184and not encrypted, the HW is responsible to perform it.
185
186When the SA is removed by the user, the driver's xdo_dev_state_delete()
187and xdo_dev_policy_delete() are asked to disable the offload.  Later,
188xdo_dev_state_free() and xdo_dev_policy_free() are called from a garbage
189collection routine after all reference counts to the state and policy
190have been removed and any remaining resources can be cleared for the
191offload state.  How these are used by the driver will depend on specific
192hardware needs.
193
194As a netdev is set to DOWN the XFRM stack's netdev listener will call
195xdo_dev_state_delete(), xdo_dev_policy_delete(), xdo_dev_state_free() and
196xdo_dev_policy_free() on any remaining offloaded states.
197
198Outcome of HW handling packets, the XFRM core can't count hard, soft limits.
199The HW/driver are responsible to perform it and provide accurate data when
200xdo_dev_state_update_stats() is called. In case of one of these limits
201occuried, the driver needs to call to xfrm_state_check_expire() to make sure
202that XFRM performs rekeying sequence.
203