1# Zstd-On-ZFS Library Manual 2 3## Introduction 4 5This subtree contains the Zstd library used in ZFS. It is heavily cut-down by 6dropping any unneeded files, and combined into a single file, but otherwise is 7intentionally unmodified. Please do not alter the file containing the zstd 8library, besides upgrading to a newer Zstd release. 9 10Tree structure: 11 12* `zfs_zstd.c` are the actual `zfs` kernel module hooks. 13* `lib/` contains the unmodified version of the `Zstandard` library 14* `zstd-in.c` is our template file for generating the single-file library 15* `include/`: This directory contains supplemental includes for platform 16 compatibility, which are not expected to be used by ZFS elsewhere in the 17 future. Thus we keep them private to Zstd. 18 19## Zstd update policy 20 21Since the exact compressed byte stream may change between Zstd versions, updates 22should follow this policy: 23 241. Zstd may be updated, as needed, after a new .0 release is tagged. 252. Critical patches may be applied up until the next release freeze, 26 _potentially_ even updating to a newer upstream version. 273. The Zstd version will not be upgraded within a major release. 284. Multiple Zstd versions are not supported concurrently within a release. 295. The library import commit must be a clean, unmodified upstream import. Any 30 OpenZFS-specific integration or local adjustments go into follow-up commits. 316. Release notes should highlight Zstd updates and any expected impact (e.g. 32 changes in compression ratio/performance, and differences in compressed byte 33 streams which may affect deduplication or NOP writes). 34 35## Updating Zstd 36 37To update Zstd the following steps need to be taken: 38 391. Grab the latest release of [Zstd](https://github.com/facebook/zstd/releases). 402. Copy the files output by the following script to `module/zstd/lib/`: 41 ``` 42 grep include [path to zstd]/build/single_file_libs/zstd-in.c | awk '{ print $2 }' 43 ``` 443. Remove debug.c, threading.c, and zstdmt_compress.c. 454. Update Makefiles with resulting file lists. 465. Follow symbol renaming notes in `include/zstd_compat_wrapper.h`. 47 48## Altering Zstd and breaking changes 49 50If Zstd made changes that break compatibility or you need to make breaking 51changes to the way we handle Zstd, it is required to maintain backwards 52compatibility. 53 54We already save the Zstd version number within the block header to be used 55to add future compatibility checks and/or fixes. However, currently it is 56not actually used in such a way. 57