filesystems/nfs/localio.rst

8 The LOCALIO auxiliary RPC protocol allows the Linux NFS client and
9 server to reliably handshake to determine if they are on the same
15 bypass the network RPC protocol for read, write and commit operations.
16 Due to this XDR and RPC bypass, these operations will operate faster.
26 But unlike the LOCALIO protocol, the sockaddr-based matching didn't
39 using XDR and RPC for reads, writes and commits can be extreme, e.g.:
42   - With LOCALIO:
48   - Without LOCALIO:
55   - With LOCALIO:
61   - Without LOCALIO:
80    a. Bypass use of the network RPC protocol as much as possible. This
81       includes bypassing XDR and RPC for open, read, write and commit
83    b. Allow client and server to autonomously discover if they are
93    deciding if the NFS client and server are co-located on the same
103    in shared kernel memory if they are truly co-located.
115    onus on the server to somehow discover that the client is co-located
122 6. Why is having the client perform a server-side file OPEN, without
123    using RPC, beneficial?  Is the benefit pNFS specific?
125    Avoiding the use of XDR and RPC for file opens is beneficial to
131    client perform a server-side file open, without using RPC, is ideal.
132    If in the future requirements change then we can adapt accordingly.
139    can then be passed to rpc.mountd to complete the authentication
142    upcall to rpc.mountd. This simplifies the authentication process, and
145 8. How do export options that translate RPC user IDs behave for LOCALIO
151    as they do for non-LOCALIO.
158 RPC  chapter
161 The LOCALIO auxiliary RPC protocol consists of a single "UUID_IS_LOCAL"
162 RPC method that allows the Linux NFS client to verify the local Linux
163 NFS server can see the nonce (single-use UUID) the client generated and
165 standard, nor does it need to be considering it is Linux-to-Linux
166 auxiliary RPC protocol that amounts to an implementation detail.
173 The RPC program number for the NFS_LOCALIO_PROGRAM is 400122 (as assigned
174 by IANA, see https://www.iana.org/assignments/rpc-program-numbers/ ):
200 to generate a nonce (single-use UUID) and associated short-lived
202 verification by the NFS server and if matched the NFS server populates
204 transfer the nfs_uuid_t from its nfs_uuids to the nn->nfsd_serv
211 access nn->nfsd_serv with proper rcu read access). It is this client
213 objects to span from the host kernel's nfsd to per-container knfsd
221 allow proper network namespace (net-ns) and NFSD object refcounting.
225 each net-ns is not destroyed while in use by nfsd_open_local_fh(), and
230     reference for the nfsd_file and associated net-ns using
235     NFSD's net-ns (and nfsd_net by association) may have been destroyed
239 easy to hit crash that would occur if an NFSD instance running in a
244 NFSD's net-ns.
260 both the associated nfsd network namespace and nn->nfsd_serv in terms of
261 RCU. If nfsd_open_local_fh() finds that the client no longer sees valid
262 nfsd objects (be it struct net or nn->nfsd_serv) it returns -ENXIO
265 recovery is needed if/when an nfsd instance running in a container were
275 With normal NFS that makes use of RPC to issue IO to the server, if an
279 the NFS client. But if all applications properly align their IO, LOCALIO
280 can be configured to use end-to-end O_DIRECT semantics from the NFS
287 Once enabled, it will cause LOCALIO to use end-to-end O_DIRECT semantics
288 (but again, this may cause IO to fail if applications do not properly
294 LOCALIO is only supported when UNIX-style authentication (AUTH_UNIX, aka
304 the server's per-namespace nfsd_net struct. With traditional NFS, the
313 controls if LOCALIO is enabled, defaults to Y. If client and server are
317 controls if O_DIRECT extends down to the underlying filesystem, defaults
322 controls if NFSv3 read and write IOs will trigger (re)enabling of
324 (disabled). Must be power-of-2, admin keeps all the pieces if they
325 misconfigure (too low a value or non-power-of-2).
333 - Client and server both on the same host.
335 - All permutations of client and server support enablement for both
338 - Testing against NFS storage products that don't support the LOCALIO
341 - Client on host, server within a container (for both v3 and v4.2).
345 - Formalizing these test scenarios in terms of existing test
346   infrastructure is on-going. Initial regular coverage is provided in
347   terms of ktest running xfstests against a LOCALIO-enabled NFS loopback
349   https://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next
352 - Various kdevops testing (in terms of "Chuck's BuildBot") has been
354   regressions to non-LOCALIO NFS use cases.
356 - All of Hammerspace's various sanity tests pass with LOCALIO enabled