ipsec offload: make hw-decrypted plain text packet like sw decrypted.

Mark hw-decrypted mbufs with M_DECRYPTED in the CHECK_POLICY() hook,
when the flag is owned by IPSEC.

Convert PACKET_TAG_IPSEC_

ipsec offload: make hw-decrypted plain text packet like sw decrypted.

Mark hw-decrypted mbufs with M_DECRYPTED in the CHECK_POLICY() hook,
when the flag is owned by IPSEC.

Convert PACKET_TAG_IPSEC_ACCEL_IN to PACKET_TAG_IPSEC_IN_DONE to
provide the xform history for ipsec transform history check.

The hw-decrypted packets are then subject to exactly the same checks at
CHECK_POLICY() hooks as the sw-decrypted packet. This includes the
policy checking, and updating the corresponding policy' lastused field,
needed for IKE daemons to track association lifetime.

Reviewed by: Ariel Ehrenberg <aehrenberg@nvidia.com>, slavash
Sponsored by: Nvidia networking

show more ...

netinet: allow per protocol random IP id control, single out IPSEC

A globally enabled random IP id generation maybe useful in most IP
contexts, but it may be unnecessary in the case of IPsec encapsu

netinet: allow per protocol random IP id control, single out IPSEC

A globally enabled random IP id generation maybe useful in most IP
contexts, but it may be unnecessary in the case of IPsec encapsulated
packets because IPsec can be configured to use anti-replay windows.

This commit adds a new net.inet.ipsec.random_id sysctl to control whether
or not IPsec packets should use random IP id generation.

Rest of the protocols/modules are still controlled by the global
net.inet.ip.random_id, but can be easily augmented with a knob.

Reviewed by: glebius
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D49164

show more ...

ipsec: allow it to work with unmapped mbufs

Only map mbuf when a policy is looked up and indicates that IPSEC needs
to transform the packet. If IPSEC is inline offloaded, it is up to the
interface

ipsec: allow it to work with unmapped mbufs

Only map mbuf when a policy is looked up and indicates that IPSEC needs
to transform the packet. If IPSEC is inline offloaded, it is up to the
interface driver to request remap if needed.

Fetch the IP header using m_copydata() instead of using mtod() to select
policy/SA.

Reviewed by: markj
Sponsored by: NVidia networking
Differential revision: https://reviews.freebsd.org/D48265

show more ...

netipsec: move declaration of the sysctl net.inet{,6}.ipsec nodes to header

Reviewed by: kp
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D46045

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content i

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content is not touched by CPU during TX
operations, kernel only does the required policy and security
association lookups to find out that given flow is offloaded, and then
packet is transmitted as plain text to the card. For driver convenience,
a metadata is attached to the packet identifying SA which must process
the packet. Card does encryption of the payload, padding, calculates
authentication, and does the reformat according to the policy.

Similarly, on receive, card does the decapsulation, decryption, and
authentification. Kernel receives the identifier of SA that was
used to process the packet, together with the plain-text packet.

Overall, payload octets are only read or written by card DMA engine,
removing a lot of memory subsystem overhead, and saving CPU time because
IPSEC algos calculations are avoided.

If driver declares support for inline IPSEC offload (with the
IFCAP2_IPSEC_OFFLOAD capability set and registering method table struct
if_ipsec_accel_methods), kernel offers the SPD and SAD to driver.
Driver decides which policies and SAs can be offloaded based on
hardware capacity, and acks/nacks each SA for given interface to
kernel. Kernel needs to keep this information to make a decision to
skip software processing on TX, and to assume processing already done
on RX. This shadow SPD/SAD database of offloads is rooted from
policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs
(struct secasvar accel_ipfs, struct ifp_handle_sav).

Some extensions to the PF_KEY socket allow to limit interfaces for
which given SP/SA could be offloaded (proposed for offload). Also,
additional statistics extensions allow to observe allocation/octet/use
counters for specific SA.

Since SPs and SAs are typically instantiated in non-sleepable context,
while offloading them into card is expected to require costly async
manipulations of the card state, calls to the driver for offload and
termination are executed in the threaded taskqueue. It also solves
the issue of allocating resources needed for the offload database.
Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the
owning SP/SA, the offload must be terminated before last reference is
dropped. ipsec_accel only adds transient references to ensure safe
pointer ownership by taskqueue.

Maintaining the SA counters for hardware-accelerated packets is the
duty of the driver. The helper ipsec_accel_drv_sa_lifetime_update()
is provided to hide accel infrastructure from drivers which would use
expected callout to query hardware periodically for updates.

Reviewed by: rscheff (transport, stack integration), np
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44219

show more ...

sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

ipsec.c: typos in the comment

Sponsored by: NVIDIA Networking
MFC after: 3 days

ipsec: add key_havesp_any

Saves on work in a common case of checking both directions.

Note further work in the area is impending to elide these in the common
case to begin with.

Sponsored by: Rubi

ipsec: add key_havesp_any

Saves on work in a common case of checking both directions.

Note further work in the area is impending to elide these in the common
case to begin with.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D36485

show more ...

ipsec: replace SECASVAR mtx by rmlock

This mutex is a significant point of contention in the ipsec code, and
can be relatively trivially replaced by a read-mostly lock.
It does require a separate l

ipsec: replace SECASVAR mtx by rmlock

This mutex is a significant point of contention in the ipsec code, and
can be relatively trivially replaced by a read-mostly lock.
It does require a separate lock for the replay protection, which we do
here by adding a separate mutex.

This improves throughput (without replay protection) by 10-15%.

MFC after: 3 weeks
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D35763

show more ...

ipsec: Check PMTU before sending a frame.

If an encapsulated frame is going to have DF bit set check its desitnitions'
PMTU and if it won't fit drop it and:

Generate ICMP 3/4 message if the packet

ipsec: Check PMTU before sending a frame.

If an encapsulated frame is going to have DF bit set check its desitnitions'
PMTU and if it won't fit drop it and:

Generate ICMP 3/4 message if the packet was to be forwarded.
Return EMSGSIZE error otherwise.

Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D30993

show more ...

ipsec: Handle ICMP NEEDFRAG message.

It will be needed for upcoming PMTU implementation in ipsec.
For now simply create/update an entry in tcp hostcache when needed.
The code is based on

ipsec: Handle ICMP NEEDFRAG message.

It will be needed for upcoming PMTU implementation in ipsec.
For now simply create/update an entry in tcp hostcache when needed.
The code is based on https://people.freebsd.org/~ae/ipsec_transport_mode_ctlinput.diff

Authored by: Kornel Duleba <mindal@semihalf.com>
Differential revision: https://reviews.freebsd.org/D30992
Reviewed by: tuxen
Sponsored by: Stormshield
Obtained from: Semihalf

show more ...

Implement anti-replay algorithm with ESN support

As RFC 4304 describes there is anti-replay algorithm responsibility
to provide appropriate value of Extended Sequence Number.

This patch introduces

Implement anti-replay algorithm with ESN support

As RFC 4304 describes there is anti-replay algorithm responsibility
to provide appropriate value of Extended Sequence Number.

This patch introduces anti-replay algorithm with ESN support based on
RFC 4304, however to avoid performance regressions window implementation
was based on RFC 6479, which was already implemented in FreeBSD.

To keep things clean and improve code readability, implementation of window
is kept in seperate functions.

Submitted by: Grzegorz Jaszczyk <jaz@semihalf.com>
Patryk Duda <pdk@semihalf.com>
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D22367
Obtained from: Semihalf
Sponsored by: Stormshield

show more ...

net: clean up empty lines in .c and .h files

Remove support for IPsec algorithms deprecated in r348205 and r360202.

Examples of depecrated algorithms in manual pages and sample configs
are updated where relevant. I removed the one example of

Remove support for IPsec algorithms deprecated in r348205 and r360202.

Examples of depecrated algorithms in manual pages and sample configs
are updated where relevant. I removed the one example of combining
ESP and AH (vs using a cipher and auth in ESP) as RFC 8221 says this
combination is NOT RECOMMENDED.

Specifically, this removes support for the following ciphers:
- des-cbc
- 3des-cbc
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- camellia-cbc

This also removes support for the following authentication algorithms:
- hmac-md5
- keyed-md5
- keyed-sha1
- hmac-ripemd160

Reviewed by: cem, gnn (older verisons)
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D24342

show more ...

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

Merge ^/head r351732 through r352104.

Fix broken window replay check that will allow old packet to be accepted.
This was introduced in r309144.

Submitted by: Jean-Francois HREN <jean-francois.hren@stormshield.eu>
Approved by: ae@
MFC af

Fix broken window replay check that will allow old packet to be accepted.
This was introduced in r309144.

Submitted by: Jean-Francois HREN <jean-francois.hren@stormshield.eu>
Approved by: ae@
MFC after: 3 days

show more ...

MFHead @349234

Sponsored by: The FreeBSD Foundation

Make the warning intervals for deprecated crypto algorithms tunable.

New sysctl/tunables can now set the interval (in seconds) between
rate-limited crypto warnings. The new sysctls are:
- kern.cryp

Make the warning intervals for deprecated crypto algorithms tunable.

New sysctl/tunables can now set the interval (in seconds) between
rate-limited crypto warnings. The new sysctls are:
- kern.cryptodev_warn_interval for /dev/crypto
- net.inet.ipsec.crypto_warn_interval for IPsec
- kern.kgssapi_warn_interval for KGSSAPI

Reviewed by: cem
MFC after: 1 month
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D20555

show more ...

ipsec offload: make hw-decrypted plain text packet like sw decrypted.

Mark hw-decrypted mbufs with M_DECRYPTED in the CHECK_POLICY() hook,
when the flag is owned by IPSEC.

Convert PACKET_TAG_IPSEC_

ipsec offload: make hw-decrypted plain text packet like sw decrypted.

Mark hw-decrypted mbufs with M_DECRYPTED in the CHECK_POLICY() hook,
when the flag is owned by IPSEC.

Convert PACKET_TAG_IPSEC_ACCEL_IN to PACKET_TAG_IPSEC_IN_DONE to
provide the xform history for ipsec transform history check.

The hw-decrypted packets are then subject to exactly the same checks at
CHECK_POLICY() hooks as the sw-decrypted packet. This includes the
policy checking, and updating the corresponding policy' lastused field,
needed for IKE daemons to track association lifetime.

Reviewed by: Ariel Ehrenberg <aehrenberg@nvidia.com>, slavash
Sponsored by: Nvidia networking

show more ...

netinet: allow per protocol random IP id control, single out IPSEC

A globally enabled random IP id generation maybe useful in most IP
contexts, but it may be unnecessary in the case of IPsec encapsu

netinet: allow per protocol random IP id control, single out IPSEC

A globally enabled random IP id generation maybe useful in most IP
contexts, but it may be unnecessary in the case of IPsec encapsulated
packets because IPsec can be configured to use anti-replay windows.

This commit adds a new net.inet.ipsec.random_id sysctl to control whether
or not IPsec packets should use random IP id generation.

Rest of the protocols/modules are still controlled by the global
net.inet.ip.random_id, but can be easily augmented with a knob.

Reviewed by: glebius
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D49164

show more ...

ipsec: allow it to work with unmapped mbufs

Only map mbuf when a policy is looked up and indicates that IPSEC needs
to transform the packet. If IPSEC is inline offloaded, it is up to the
interface

ipsec: allow it to work with unmapped mbufs

Only map mbuf when a policy is looked up and indicates that IPSEC needs
to transform the packet. If IPSEC is inline offloaded, it is up to the
interface driver to request remap if needed.

Fetch the IP header using m_copydata() instead of using mtod() to select
policy/SA.

Reviewed by: markj
Sponsored by: NVidia networking
Differential revision: https://reviews.freebsd.org/D48265

show more ...

netipsec: move declaration of the sysctl net.inet{,6}.ipsec nodes to header

Reviewed by: kp
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D46045

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content i

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content is not touched by CPU during TX
operations, kernel only does the required policy and security
association lookups to find out that given flow is offloaded, and then
packet is transmitted as plain text to the card. For driver convenience,
a metadata is attached to the packet identifying SA which must process
the packet. Card does encryption of the payload, padding, calculates
authentication, and does the reformat according to the policy.

Similarly, on receive, card does the decapsulation, decryption, and
authentification. Kernel receives the identifier of SA that was
used to process the packet, together with the plain-text packet.

Overall, payload octets are only read or written by card DMA engine,
removing a lot of memory subsystem overhead, and saving CPU time because
IPSEC algos calculations are avoided.

If driver declares support for inline IPSEC offload (with the
IFCAP2_IPSEC_OFFLOAD capability set and registering method table struct
if_ipsec_accel_methods), kernel offers the SPD and SAD to driver.
Driver decides which policies and SAs can be offloaded based on
hardware capacity, and acks/nacks each SA for given interface to
kernel. Kernel needs to keep this information to make a decision to
skip software processing on TX, and to assume processing already done
on RX. This shadow SPD/SAD database of offloads is rooted from
policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs
(struct secasvar accel_ipfs, struct ifp_handle_sav).

Some extensions to the PF_KEY socket allow to limit interfaces for
which given SP/SA could be offloaded (proposed for offload). Also,
additional statistics extensions allow to observe allocation/octet/use
counters for specific SA.

Since SPs and SAs are typically instantiated in non-sleepable context,
while offloading them into card is expected to require costly async
manipulations of the card state, calls to the driver for offload and
termination are executed in the threaded taskqueue. It also solves
the issue of allocating resources needed for the offload database.
Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the
owning SP/SA, the offload must be terminated before last reference is
dropped. ipsec_accel only adds transient references to ensure safe
pointer ownership by taskqueue.

Maintaining the SA counters for hardware-accelerated packets is the
duty of the driver. The helper ipsec_accel_drv_sa_lifetime_update()
is provided to hide accel infrastructure from drivers which would use
expected callout to query hardware periodically for updates.

Reviewed by: rscheff (transport, stack integration), np
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44219

show more ...