system/s390x/vfio-ap.rst

7 ------------
19 -------------------------
28   on the machine model. Adapters assigned to the LPAR in which a linux host is
29   running will be available to the linux host. Each adapter is identified by a
51   An AP queue is the means by which an AP command-request message is sent to an
57   which the AP command-request message is to be sent for processing.
63   * NQAP: to enqueue an AP command-request message to a queue
64   * DQAP: to dequeue an AP command-reply message from a queue
68   command; this must be one of the usage domains. An AP command may modify a
73 ----------------------------------------------
84   an APID from 0-255. If a bit is set, the corresponding adapter is valid for
89   corresponds to  an AP queue index (APQI) from 0-255. If a bit is set, the
94   changed by an AP command-request message sent to a usage domain from the
96   0-255. If a bit is set, the corresponding domain can be modified by an AP
97   command-request message sent to a usage domain.
100 an APQN to identify the AP adapter and AP queue to which an AP command-request
102 command-reply message is to be received (DQAP instruction). The validity of an
109 The APQNs can provide secure key functionality - i.e., a private key is stored
110 on the adapter card for each of its domains - so each APQN must be assigned to
111 at most one guest or the linux host.
116 +----------+--------+--------+
120 +----------+--------+--------+
122 +----------+--------+--------+
132 +----------+--------+--------+
136 +----------+--------+--------+
138 +----------+--------+--------+
148 +----------+--------+--------+
152 +----------+--------+--------+
154 +----------+--------+--------+
159 AP Matrix Configuration on Linux Host
160 -------------------------------------
185 host system would look like this::
201 device that can be assigned to the LPAR on which a linux host is running::
227 The ``apmask`` is a 256-bit mask that identifies a set of AP adapter IDs
230 0-255. If a bit is set, the APID is marked as usable only by the default AP
234 The ``aqmask`` is a 256-bit mask that identifies a set of AP queue indexes
237 0-255. If a bit is set, the APQI is marked as usable only by the default AP
247       1, 2, 3, 4, 5, and 7-255 belong to the default drivers' pool, and 0 and 6
250 The APQN of each AP queue device assigned to the linux host is checked by the
262     * An absolute hex string starting with 0x - like "0x12345678" - sets
278       number string must be prepended with a (``+``) or minus (``-``) to indicate
279       the corresponding bit is to be switched on (``+``) or off (``-``). Some
283            "-13"   switches bit 13 off
285            "-0xff" switches bit 255 off
289               +0,-6,+0x47,-0xf0
298     command line like this::
314             default drivers pool:    adapter 0-15, domain 1
315             alternate drivers pool:  adapter 16-255, domains 0, 2-255
331   ............ [vfio_ap-passthrough]
353   ............ [vfio_ap-passthrough]
366   ............ [vfio_ap-passthrough]
479 must be specified on the QEMU command line::
481    -device vfio_ap,sysfsdev=$path-to-mdev
489   /sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/devices/$uuid
505    identified in the ADM field of the guest's CRYCB. The AP command must be sent
517    are installed on the host system. The feature is s390-specific and is
518    represented as a parameter of the -cpu option on the QEMU command line::
520       qemu-system-s390x -cpu $model,ap=on|off
526         the host system if not specified).
532         vfio-ap device (``-device vfio-ap,sysfsdev=$path``) is configured
539    only if the QCI facility is installed on the host system. The feature is
540    s390-specific and is represented as a parameter of the -cpu option on the
541    QEMU command line::
543       qemu-system-s390x -cpu $model,apqci=on|off
569    only if the APFT facility is installed on the host system. The feature is
570    s390-specific and is represented as a parameter of the -cpu option on the
571    QEMU command line::
573       qemu-system-s390x -cpu $model,apft=on|off
579         the host system if not specified).
599 Hot plug a vfio-ap device into a running guest
602 Only one vfio-ap device can be attached to the virtual machine's ap-bus, so a
603 vfio-ap device can be hot plugged if and only if no vfio-ap device is attached
604 to the bus already, whether via the QEMU command line or a prior hot plug
607 To hot plug a vfio-ap device, use the QEMU ``device_add`` command::
609     (qemu) device_add vfio-ap,sysfsdev="$path-to-mdev",id="$id"
611 Where the ``$path-to-mdev`` value specifies the absolute path to a mediated
620 The command will fail if:
622 * A vfio-ap device has already been attached to the virtual machine's ap-bus.
627 Hot unplug a vfio-ap device from a running guest
630 A vfio-ap device can be unplugged from a running KVM guest if a vfio-ap device
631 has been attached to the virtual machine's ap-bus via the QEMU command line
634 To hot unplug a vfio-ap device, use the QEMU ``device_del`` command::
645 The command will fail if the ``$path-to-mdev`` specified on the ``device_del`` command
646 does not match the value specified when the vfio-ap device was attached to
647 the virtual machine's ap-bus.
650 -----------------------------------------------------
654 three guests such that executing the lszcrypt command on the guests would
660   ------------------------------
661   05          CEX5C CCA-Coproc
662   05.0004     CEX5C CCA-Coproc
663   05.00ab     CEX5C CCA-Coproc
666   06.00ab     CEX5C CCA-Coproc
671   ------------------------------
679   ------------------------------
686 1. Install the vfio_ap module on the linux host. The dependency chain for the
710      -> Device Drivers
711         -> IOMMU Hardware Support
713         -> VFIO Non-Privileged userspace driver framework
714            -> Mediated device driver framework
715               -> VFIO driver for Mediated devices
716      -> I/O subsystem
717         -> VFIO support for AP devices
719 2. Secure the AP queues to be used by the three guests so that the host can not
725       echo -5,-6 > /sys/bus/ap/apmask
727       echo -4,-0x47,-0xab,-0xff > /sys/bus/ap/aqmask
768      ...... [vfio_ap-passthrough] (passthrough mediated matrix device type)
792      ...... [vfio_ap-passthrough]
865 …/usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on -device vfio-ap,sysfsdev=/sys/devi…
869 …/usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on -device vfio-ap,sysfsdev=/sys/devi…
873 …/usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on -device vfio-ap,sysfsdev=/sys/devi…
881    ...... [vfio_ap-passthrough]
896 host. If the mdev matrix device is removed, one may want to also reconfigure
900 -----------
906   assigned lest the host be given access to the private data of the AP queue
910   mediated matrix device - see `Configuring an AP matrix for a linux guest`_
911   section above - while a running guest is using it is currently not supported.
914   is using AP devices, the vfio-ap device configured for the guest must be
915   unplugged before migrating the guest (see `Hot unplug a vfio-ap device from a