crypto/iaa/iaa-crypto.rst

1 .. SPDX-License-Identifier: GPL-2.0
13 The IAA hardware spec can be found here:
18 higher-level compression devices such as zswap.
20 Users can select IAA compress/decompress acceleration by specifying
24 For example, a zswap device can select the IAA 'fixed' mode
25 represented by selecting the 'deflate-iaa' crypto compression
28   # echo deflate-iaa > /sys/module/zswap/parameters/compressor
38 'deflate-iaa'.  (Because the IAA hardware has a 4k history-window
52   Cryptographic API -> Hardware crypto devices -> Support for Intel(R) IAA Compression Accelerator
59 …Cryptographic API -> Hardware crypto devices -> Support for Intel(R) IAA Compression -> Enable Int…
74 IAA is one of the first Intel accelerator IPs that can work in
78   - Scalable
79   - Legacy
80   - No IOMMU
84 -------------
91 with VT-d turned on in BIOS.
98 …Socket Configuration > IIO Configuration > Intel VT for Directed I/O (VT-d) > Intel VT for Directe…
104 -----------
110 or VT-d is not turned on in BIOS.
112 If you have booted into Linux and not sure if VT-d is on, do a "dmesg
113 | grep -i dmar". If you don't see a number of DMAR devices enumerated,
114 most likely VT-d is not on.
120 -------------
132 accel-config
133 ------------
138 the user must first disable the IAA devices and workqueues, reset the
139 configuration, and then re-register the deflate-iaa algorithm with the
143 section below can be used to disable the default configuration.
153 The userspace tool to help doing that is called accel-config.  Using
154 accel-config to configure device or loading a previously saved config
155 is highly recommended.  The device can be controlled via sysfs
158 cover the sysfs interface but assumes you will be using accel-config.
160 The :ref:`iaa_sysfs_config` section in the appendix below can be
163 The accel-config tool along with instructions for building it can be
166   https://github.com/intel/idxd-config/#readme
169 -------------
177 specified as 'iax' rather than 'iaa' - this is because upstream still
182 …accel-config config-wq --group-id=0 --mode=dedicated --type=kernel --name="iaa_crypto" --device_na…
186   accel-config enable-device iax1
190   accel-config enable-wq iax1/wq1.0
214 algorithms can be unregistered by removing the module.
218 -----------------
220 There are a couple user-configurable driver attributes that can be
232   - verify_compress
236     codes if unsuccessful.  This can be toggled with 0/1::
240     The default setting is '1' - verify all compresses.
242   - sync_mode
249     so in a synchronous manner - it fills and submits the IDXD
256     support for callers that can make use of it.  In this mode, it
258     with -EINPROGRESS.  The caller can then either poll for completion
267     This mode can be enabled by writing 'async_irq' to the sync_mode
272     Async mode without interrupts (caller must poll) can be enabled by
277     The mode that does the polling in the iaa_crypto driver can be
287 -------------------------
311 to have the new configuration applied to the deflate-iaa crypto
312 algorithm, it needs to be re-registered by removing and reinserting
314 Cases' section below can be used to disable the default configuration.
320 driver will generate statistics which can be accessed in debugfs at::
322   # ls -al /sys/kernel/debug/iaa-crypto/
324   drwxr-xr-x  2 root root 0 Mar  3 09:35 .
325   drwx------ 47 root root 0 Mar  3 09:35 ..
326   -rw-r--r--  1 root root 0 Mar  3 09:35 max_acomp_delay_ns
327   -rw-r--r--  1 root root 0 Mar  3 09:35 max_adecomp_delay_ns
328   -rw-r--r--  1 root root 0 Mar  3 09:35 max_comp_delay_ns
329   -rw-r--r--  1 root root 0 Mar  3 09:35 max_decomp_delay_ns
330   -rw-r--r--  1 root root 0 Mar  3 09:35 stats_reset
331   -rw-r--r--  1 root root 0 Mar  3 09:35 total_comp_bytes_out
332   -rw-r--r--  1 root root 0 Mar  3 09:35 total_comp_calls
333   -rw-r--r--  1 root root 0 Mar  3 09:35 total_decomp_bytes_in
334   -rw-r--r--  1 root root 0 Mar  3 09:35 total_decomp_calls
335   -rw-r--r--  1 root root 0 Mar  3 09:35 wq_stats
337 Most of the above statisticss are self-explanatory.  The wq_stats file
338 shows per-wq stats, a set for each iaa device and wq in addition to
394 per-device and per-wq stats::
413 -----------------
448 commands can be used to enable debug output::
450   # echo -n 'module iaa_crypto +p' > /sys/kernel/debug/dynamic_debug/control
451   # echo -n 'module idxd +p' > /sys/kernel/debug/dynamic_debug/control
457   # echo deflate-iaa > /sys/module/zswap/parameters/compressor
465 Now you can now run the zswap workload you want to measure. For
478 …comp_acompress: dma_map_sg, src_addr 223925c000, nr_sgs 1, req->src 00000000ee7cb5e6, req->slen 40…
479 …comp_acompress: dma_map_sg, dst_addr 21dadf8000, nr_sgs 1, req->dst 000000008d6acea8, req->dlen 40…
480 …press: desc->src1_addr 223925c000, desc->src1_size 4096, desc->dst_addr 21dadf8000, desc->max_dst_…
481 …y: (verify) desc->src1_addr 21dadf8000, desc->src1_size 228, desc->dst_addr 223925c000, desc->max_…
484 Now that basic functionality has been demonstrated, the defaults can
486 first disable zswap::
489   # swapoff -a
496 below to disable the default configuration.
500   # swapon -a
502 Following all that the IAA device(s) can now be re-configured and
514   lspci -d:0cfe
516   lspci -d:0cfe | wc -l
522   num_iaa=$(lspci -d:${iaa_dev_id} | wc -l)
526   # disable iaa wqs and devices
528   echo "Disable IAA"
531       echo disable wq iax${i}/wq${i}.0
532       accel-config disable-wq iax${i}/wq${i}.0
533       echo disable iaa iax${i}
534       accel-config disable-device iax${i}
537   echo "End Disable IAA"
544 …accel-config config-wq --group-id=0 --mode=dedicated --size=128 --priority=10 --type=kernel --name…
556       accel-config enable-device iaa${i}
558       accel-config enable-wq iaa${i}/wq${i}.0
565 enabled debug output (echo -n 'module iaa_crypto +p' >
594 Now run the following zswap-specific setup commands to have zswap use
599   echo deflate-iaa > /sys/module/zswap/parameters/compressor
608 Finally, you can now run the zswap workload you want to measure. For
620 you've enabled debug output (echo -n 'module iaa_crypto +p' >
623 …comp_acompress: dma_map_sg, src_addr 223925c000, nr_sgs 1, req->src 00000000ee7cb5e6, req->slen 40…
624 …comp_acompress: dma_map_sg, dst_addr 21dadf8000, nr_sgs 1, req->dst 000000008d6acea8, req->dlen 40…
625 …press: desc->src1_addr 223925c000, desc->src1_size 4096, desc->dst_addr 21dadf8000, desc->max_dst_…
626 …y: (verify) desc->src1_addr 21dadf8000, desc->src1_size 228, desc->dst_addr 223925c000, desc->max_…
627 …mp_adecompress: dma_map_sg, src_addr 21ddd8b100, nr_sgs 1, req->src 0000000084adab64, req->slen 22…
628 …mp_adecompress: dma_map_sg, dst_addr 21ee3dc000, nr_sgs 1, req->dst 000000004e2990d0, req->dlen 40…
629 …decompress: desc->src1_addr 21ddd8b100, desc->src1_size 228, desc->dst_addr 21ee3dc000, desc->max_…
630 …mp_adecompress: dma_map_sg, src_addr 21ddd8b100, nr_sgs 1, req->src 0000000084adab64, req->slen 22…
631 …mp_adecompress: dma_map_sg, dst_addr 21f1551000, nr_sgs 1, req->dst 000000004e2990d0, req->dlen 40…
632 …decompress: desc->src1_addr 21ddd8b100, desc->src1_size 228, desc->dst_addr 21f1551000, desc->max_…
643   swapoff -a
650 devices can be disabled.
654 IAA disable script
655 ------------------
662   lspci -d:0cfe
664   lspci -d:0cfe | wc -l
670   num_iaa=$(lspci -d:${iaa_dev_id} | wc -l)
674   # disable iaa wqs and devices
676   echo "Disable IAA"
679       echo disable wq iax${i}/wq${i}.0
680       accel-config disable-wq iax${i}/wq${i}.0
681       echo disable iaa iax${i}
682       accel-config disable-device iax${i}
685   echo "End Disable IAA"
687 Finally, at this point the iaa_crypto module can be removed, which
693 memory_madvise.c (gcc -o memory_memadvise memory_madvise.c)::
721 …    addr = mmap(NULL, nr_pages * PG_SZ, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
755                 loop --;
766 --------------------------
771 since accel-config can do everything the sysfs interface can and in
772 fact accel-config is based on it under the covers.
778 odd-numbered devices; the even-numbered devices are DSA devices and
779 can be ignored for IAA).
788 /sys/bus/dsa/drivers/crypto/unbind are used to disable IAA devices and
795   1) Disable any workqueues enabled on the device.  For example to
796      disable workques 0 and 1 on IAA device 3::
801   2) Disable the device. For example to disable IAA device 3::