libnv: add tests to verify potential overflow issues

Differential Revision: https://reviews.freebsd.org/D46131

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

libnv: Allow use in non-sleepable contexts

44c125c4cebc2fd87c6260b90eddae11201f5232 switched the nvlist allocations
to be M_WAITOK, but this precludes the use in non-sleepable contexts.
(E.g. with a

libnv: Allow use in non-sleepable contexts

44c125c4cebc2fd87c6260b90eddae11201f5232 switched the nvlist allocations
to be M_WAITOK, but this precludes the use in non-sleepable contexts.
(E.g. with a nonsleepable lock held).

All callers for these allocation functions already cope with memory
alloation failures, so there's no reason to allow sleeping during
allocations.

Reviewed by: melifaro, oshogbo
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29556

show more ...

Merge ^/head r343956 through r344177.

libnv: fix memory leaks

nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().

nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): fre

libnv: fix memory leaks

nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().

nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array(). This affects
all nvlist_*_nvlist_array() use

Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by: clang/gcc ASAN
MFC after: 2 weeks

show more ...

libnv: add tests to verify potential overflow issues

Differential Revision: https://reviews.freebsd.org/D46131

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

libnv: Allow use in non-sleepable contexts

44c125c4cebc2fd87c6260b90eddae11201f5232 switched the nvlist allocations
to be M_WAITOK, but this precludes the use in non-sleepable contexts.
(E.g. with a

libnv: Allow use in non-sleepable contexts

44c125c4cebc2fd87c6260b90eddae11201f5232 switched the nvlist allocations
to be M_WAITOK, but this precludes the use in non-sleepable contexts.
(E.g. with a nonsleepable lock held).

All callers for these allocation functions already cope with memory
alloation failures, so there's no reason to allow sleeping during
allocations.

Reviewed by: melifaro, oshogbo
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29556

show more ...

Merge ^/head r343956 through r344177.

libnv: fix memory leaks

nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().

nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): fre

libnv: fix memory leaks

nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().

nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array(). This affects
all nvlist_*_nvlist_array() use

Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by: clang/gcc ASAN
MFC after: 2 weeks

show more ...

libnv: Add nvlist_append_*_array() family of functions.

The nvlist_append_{bool,number,string,nvlist,descriptor}_array() functions
allows to dynamically extend array stored in the nvlist.

Submitted

libnv: Add nvlist_append_*_array() family of functions.

The nvlist_append_{bool,number,string,nvlist,descriptor}_array() functions
allows to dynamically extend array stored in the nvlist.

Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>

show more ...

Add SPDX tags for nv(9).

MFC after: 2 weeks

libnv: Use mallocarray(9) for the nv_calloc.

Add support for the arrays in nvlist library.

- Add
nvlist_{add,get,take,move,exists,free}_{number,bool,string,nvlist,
descriptor} functions.
- Add support for (un)packing arrays.
- Add the nvl_

Add support for the arrays in nvlist library.

- Add
nvlist_{add,get,take,move,exists,free}_{number,bool,string,nvlist,
descriptor} functions.
- Add support for (un)packing arrays.
- Add the nvl_array_next field to the nvlist structure.
If an array is added by the nvlist_{move,add}_nvlist_array function
this field will contains next element in the array.
- Add the nitems field to the nvpair and nvpair_header structure.
This field contains number of elements in the array.
- Add special flag (NV_FLAG_IN_ARRAY) which is set if nvlist is a part of
an array.
- Add special type (NV_TYPE_NVLIST_ARRAY_NEXT).This type is used only
on packing/unpacking.
- Add new API for traversing arrays (nvlist_get_array_next).
- Add the nvlist_get_pararr function which combines the
nvlist_get_array_next and nvlist_get_parent functions. If nvlist is in
the array it will return next element from array. If nvlist is last
element in array or it isn't in array it will return his
container (parent). This function should simplify traveling over nvlist.
- Add tests for new features.
- Add documentation for new functions.
- Add my copyright.
- Regenerate the sys/cddl/compat/opensolaris/sys/nvpair.h file.

PR: 191083
Reviewed by: allanjude (doc)
Approved by: pjd (mentor)

show more ...

The nvlist_move_nvpair() function can fail in two cases, if:
- the nvlist error is set, or
- the nvlist case ignore flag is not set and there is attend to
add element with duplicated name.
In both

The nvlist_move_nvpair() function can fail in two cases, if:
- the nvlist error is set, or
- the nvlist case ignore flag is not set and there is attend to
add element with duplicated name.
In both cases the nvlist_move_nvpair() function free nvpair structure.
If library will try to unpack a binary blob which contains duplicated
names it will end up with using memory after free.

To prevent that, the nvlist_move_nvpair() function interface is changed
to report about failure and checks are added to the nvpair_xunpack()
function.

Discovered thanks to the american fuzzy lop.

Approved by: pjd (mentor)

show more ...

Move the nvlist source and private includes from sys/kern to seperate
directory sys/contrib/libnv.

The goal of this operation is to NOT install header files which shouldn't
be used outside the nvlis

Move the nvlist source and private includes from sys/kern to seperate
directory sys/contrib/libnv.

The goal of this operation is to NOT install header files which shouldn't
be used outside the nvlist library.

Approved by: pjd (mentor)

show more ...

Rename macros to use prefix ERRNO. Add macro ERRNO_SET. Now
ERRNO_{RESTORE/SAVE} must by used together, additional variable is not
needed. Always use ERRNO_{SAVE/RESTORE/SET} macros.

Approved by: pj

Rename macros to use prefix ERRNO. Add macro ERRNO_SET. Now
ERRNO_{RESTORE/SAVE} must by used together, additional variable is not
needed. Always use ERRNO_{SAVE/RESTORE/SET} macros.

Approved by: pjd (mentor)

show more ...

Remove the nvlist_.*[fv] functions.

Those functions are problematic, because there is no way to report
memory allocation problems without complicating the API, so we can
either abort or potentially

Remove the nvlist_.*[fv] functions.

Those functions are problematic, because there is no way to report
memory allocation problems without complicating the API, so we can
either abort or potentially return invalid results. None of which is
acceptable.

In most cases the caller knows the size of the name, so he can allocate
buffer on the stack and use snprintf(3) to prepare the name.

After some discussion the conclusion is to removed those functions,
which also simplifies the API.

Discussed with: pjd, rstone
Approved by: pjd (mentor)

show more ...

Nvlist functionality is not used within interrupt context, so we should
use M_WAITOK to allocate memory.

Approved by: pjd (mentor)

Style fixes.

Approved by: pjd (mentor)

Move libnv into the kernel and hook it into the kernel build

Differential Revision: https://reviews.freebsd.org/D1883
Reviewed by: jfv
MFC after: 1 month
Sponsored by: Sandvine Inc.

Add macros to make code compile in kernel

Make it possible to compile libnv in the kernel. Mostly this
involves wrapping functions that have a different signature in
the kernel and in userland (e.g

Add macros to make code compile in kernel

Make it possible to compile libnv in the kernel. Mostly this
involves wrapping functions that have a different signature in
the kernel and in userland (e.g. malloc()) in a macro that will
conditionally expand to the right API depending on whether the
code is being compiled for the kernel or not.

I have also #ifdef'ed out all of file descriptor-handling code,
as well as the unsafe varargs functions.

Differential Revision: https://reviews.freebsd.org/D1882
Reviewed by: jfv
MFC after: 1 month
Sponsored by: Sandvine Inc

show more ...

Don't allocate memory for operations that do not insert

Almost every operation performed on an nvlist was allocating a
new string to hold the key name. The nvlist_exists* family of
functions would

Don't allocate memory for operations that do not insert

Almost every operation performed on an nvlist was allocating a
new string to hold the key name. The nvlist_exists* family of
functions would always return false if they failed to allocate
the string. The rest of the functions would outright abort().
Fix the non-varargs variants of the functions to perform the
requested operations directly and the varargs versions to
allocate the string and call into the non-varargs versions.
The varargs versions are still broken and really can't be fixed,
so we might consider axing them entirely. However, now the non-
varargs functions are always safe to call.

Differential Revision: https://reviews.freebsd.org/D1879
Reviewed by: pjd, jfv
MFC after: 1 month
Sponsored by: Sandvine Inc.

show more ...

Use non-recursive algorithm for traversing nvlists. This also removes
the limit on number of nested nvlists.

Submitted by: Mariusz Zaborski

Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 6

Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 64bit unsigned integer;
- NV_TYPE_STRING - C string;
- NV_TYPE_NVLIST - nested nvlist;
- NV_TYPE_DESCRIPTOR - file descriptor;
- NV_TYPE_BINARY - binary data.

For detailed documentation and examples see nv(3) manual page.

Sponsored by: The FreeBSD Foundation

show more ...