1# How to use virtio-fs 2 3In the context of virtualization, it is always convenient to be able to share a 4directory from the host with the guest. 5 6__virtio-fs__, also known as __vhost-user-fs__ is a virtual device defined by 7the VIRTIO specification which allows any VMM to perform filesystem sharing. 8 9## Pre-requisites 10 11### The daemon 12 13This virtual device relies on the _vhost-user_ protocol, which assumes the 14backend (device emulation) is handled by a dedicated process running on the 15host. This daemon is called __virtiofsd__ and needs to be present on the host. 16 17_Build virtiofsd_ 18```bash 19git clone https://gitlab.com/virtio-fs/virtiofsd 20pushd virtiofsd 21cargo build --release 22sudo setcap cap_sys_admin+epi target/release/virtiofsd 23``` 24 25_Create shared directory_ 26```bash 27mkdir /tmp/shared_dir 28``` 29_Run virtiofsd_ 30```bash 31./virtiofsd \ 32 --log-level debug \ 33 --socket-path=/tmp/virtiofs \ 34 --shared-dir=/tmp/shared_dir \ 35 --cache=never \ 36 --thread-pool-size=$N 37``` 38 39The `cache=never` option is the default when using `virtiofsd` with 40Cloud Hypervisor. This prevents from using the host page cache, reducing the 41overall footprint on host memory. This increases the maximum density of virtual 42machines that can be launched on a single host. 43 44The `cache=always` option will allow the host page cache to be used, which can 45result in better performance for the guest's workload at the cost of increasing 46the footprint on host memory. 47 48The `thread-pool-size` option controls how many IO threads are spawned. For 49very fast storage like NVMe spawning enough worker threads is critical to 50getting an acceptable performance compared to native. 51 52### Kernel support 53 54Modern Linux kernels (at least v5.10) have support for virtio-fs. Use of older 55kernels, with additional patches, are not supported. 56 57## How to share directories with cloud-hypervisor 58 59### Start the VM 60 61Once the daemon is running, the option `--fs` from Cloud Hypervisor needs 62to be used. 63 64Both direct kernel boot and EFI firmware can be used to boot a VM with 65virtio-fs, given that the cloud image contains a recent enough kernel. 66 67Correct functioning of `--fs` requires `--memory shared=on` to facilitate 68interprocess memory sharing. 69 70Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your 71system, here is the Cloud Hypervisor command you need to run: 72```bash 73./cloud-hypervisor \ 74 --cpus boot=1 \ 75 --memory size=1G,shared=on \ 76 --disk path=focal-server-cloudimg-amd64.raw \ 77 --kernel vmlinux \ 78 --cmdline "console=hvc0 root=/dev/vda1 rw" \ 79 --fs tag=myfs,socket=/tmp/virtiofs,num_queues=1,queue_size=512 80``` 81 82### Mount the shared directory 83 84The last step is to mount the shared directory inside the guest, using the 85`virtiofs` filesystem type. 86 87```bash 88mkdir mount_dir 89mount -t virtiofs myfs mount_dir/ 90``` 91 92The `tag` needs to be consistent with what has been provided through the 93Cloud Hypervisor command line, which happens to be `myfs` in this example. 94 95## DAX feature 96 97Given the DAX feature is not stable yet from a daemon standpoint, it is not 98available in Cloud Hypervisor. 99