1# Virtio Data Path Acceleration 2 3vDPA aims at achieving bare-metal performance for devices passed into a virtual 4machine. It is an alternative to VFIO, as it provides a simpler solution for 5achieving migration. 6 7It is a kernel framework introduced recently to handle devices complying with 8the VIRTIO specification on their data-path, while the control path is vendor 9specific. In practice, virtqueues are accessed directly through DMA mechanism 10between the hardware and the guest. The control path is accessed through the 11vDPA framework, being exposed through the vhost interface as a vhost-vdpa 12device. 13 14Because DMA accesses between device and guest are going through virtqueues, 15migration can be achieved without requiring device's driver to implement any 16specific migration support. In case of VFIO, each vendor is expected to provide 17an implementation of the VFIO migration framework, complicating things as it 18must be done for each and every device's driver. 19 20The official [website](https://vdpa-dev.gitlab.io/) contains some extensive 21documentation on the topic. 22 23## Usage 24 25`VdpaConfig` (known as `--vdpa` from the CLI perspective) contains the list of 26parameters available for the vDPA device. 27 28```rust 29struct VdpaConfig { 30 path: PathBuf, 31 num_queues: usize, 32 id: Option<String>, 33 pci_segment: u16, 34} 35``` 36 37``` 38--vdpa <vdpa> vDPA device "path=<device_path>,num_queues=<number_of_queues>,iommu=on|off,id=<device_id>,pci_segment=<segment_id>" 39``` 40 41### `path` 42 43Path of the vDPA device. Usually `/dev/vhost-vdpa-X`. 44 45This parameter is mandatory. 46 47Value is a string. 48 49_Example_ 50 51``` 52--vdpa path=/dev/vhost-vdpa-0 53``` 54 55### `num_queues` 56 57Number of virtqueues supported by the vDPA device. 58 59This parameter is optional. 60 61Value is an unsigned integer set to `1` by default. 62 63_Example_ 64 65``` 66--vdpa path=/dev/vhost-vdpa-0,num_queues=2 67``` 68 69### `id` 70 71Identifier of the vDPA device. 72 73This parameter is optional. If provided, it must be unique across the entire 74virtual machine. 75 76Value is a string. 77 78_Example_ 79 80``` 81--vdpa path=/dev/vhost-vdpa-0,id=vdpa0 82``` 83 84### `pci_segment` 85 86PCI segment number to which the vDPA device should be attached to. 87 88This parameter is optional. 89 90Value is an unsigned integer of 16 bits set to `0` by default. 91 92_Example_ 93 94``` 95--vdpa path=/dev/vhost-vdpa-0,pci_segment=1 96``` 97 98## Example with vDPA block simulator 99 100The vDPA framework provides a simulator with both `virtio-block` and 101`virtio-net` implementations. This is very useful for testing vDPA when we 102don't have access to the specific hardware. 103 104Given the host kernel has the appropriate modules available, let's load them 105all: 106 107``` 108sudo modprobe vdpa 109sudo modprobe vhost_vdpa 110sudo modprobe vdpa_sim 111sudo modprobe vdpa_sim_blk 112``` 113 114Given you have the `iproute2/vdpa` tool installed, let's now create the 115`virtio-block` vDPA device: 116 117```sh 118sudo vdpa dev add name vdpa-blk1 mgmtdev vdpasim_blk 119sudo chown $USER:$USER /dev/vhost-vdpa-0 120sudo chmod 660 /dev/vhost-vdpa-0 121``` 122 123Increase the maximum locked memory to ensure setting up IOMMU mappings will 124succeed: 125 126```sh 127ulimit -l unlimited 128``` 129 130Start Cloud Hypervisor: 131 132```sh 133cloud-hypervisor \ 134 --cpus boot=1 \ 135 --memory size=1G,hugepages=on \ 136 --disk path=focal-server-cloudimg-amd64.raw \ 137 --kernel vmlinux \ 138 --cmdline "root=/dev/vda1 console=hvc0" \ 139 --vdpa path=/dev/vhost-vdpa-0,num_queues=1 140``` 141 142The `virtio-block` device backed by the vDPA simulator can be found as 143`/dev/vdb` in the guest: 144 145``` 146cloud@cloud:~$ lsblk 147NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT 148nullb0 252:0 0 250G 0 disk 149vda 254:0 0 2.2G 0 disk 150├─vda1 254:1 0 2.1G 0 part / 151├─vda14 254:14 0 4M 0 part 152└─vda15 254:15 0 106M 0 part /boot/efi 153vdb 254:16 0 128M 0 disk 154``` 155