more/cloud-hypervisor/README.md

1 - [1. What is Cloud Hypervisor?](#1-what-is-cloud-hypervisor)
2   - [Objectives](#objectives)
3     - [High Level](#high-level)
4     - [Architectures](#architectures)
5     - [Guest OS](#guest-os)
6 - [2. Getting Started](#2-getting-started)
7   - [Host OS](#host-os)
8   - [Use Pre-built Binaries](#use-pre-built-binaries)
9   - [Packages](#packages)
10   - [Building from Source](#building-from-source)
11   - [Booting Linux](#booting-linux)
12     - [Firmware Booting](#firmware-booting)
13     - [Custom Kernel and Disk Image](#custom-kernel-and-disk-image)
14       - [Building your Kernel](#building-your-kernel)
15       - [Disk image](#disk-image)
16       - [Booting the guest VM](#booting-the-guest-vm)
17 - [3. Status](#3-status)
18   - [Hot Plug](#hot-plug)
19   - [Device Model](#device-model)
20   - [Roadmap](#roadmap)
21 - [4. Relationship with _Rust VMM_ Project](#4-relationship-with-rust-vmm-project)
22   - [Differences with Firecracker and crosvm](#differences-with-firecracker-and-crosvm)
23 - [5. Community](#5-community)
24   - [Contribute](#contribute)
25   - [Slack](#slack)
26   - [Mailing list](#mailing-list)
27   - [Security issues](#security-issues)
40 64-bit CPUs.
42 Cloud Hypervisor is implemented in [Rust](https://www.rust-lang.org/) and is
43 based on the [Rust VMM](https://github.com/rust-vmm) crates.
49 - Runs on KVM or MSHV
50 - Minimal emulation
51 - Low latency
52 - Low memory footprint
53 - Low complexity
54 - High performance
55 - Small attack surface
56 - 64-bit support only
57 - CPU, memory, PCI hotplug
58 - Machine to machine migration
62 Cloud Hypervisor supports the `x86-64`, `AArch64` and `riscv64`
64 functionality differences between `x86-64` and `AArch64` are documented
65 in [#1125](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1125).
72 Cloud Hypervisor supports `64-bit Linux` and Windows 10/Windows Server 2019.
80 - AArch64 servers (recommended) or development boards equipped with the GICv3
89 ## Use Pre-built Binaries
92 pre-built binary. Binaries are available for the [latest
93 release](https://github.com/cloud-hypervisor/cloud-hypervisor/releases/latest).
94 Use `cloud-hypervisor-static` for `x86-64` or `cloud-hypervisor-static-aarch64`
102 README](https://github.com/cloud-hypervisor/obs-packaging) explains how to
105 [obs-packaging](https://github.com/cloud-hypervisor/obs-packaging) repository.
110 do not wish to use the pre-built binaries.
114 Cloud Hypervisor supports direct kernel boot (the x86-64 kernel requires the kernel
116 Firmware](https://github.com/cloud-hypervisor/rust-hypervisor-firmware) or an
121 Firmware](https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/latest)
123 repository](https://github.com/cloud-hypervisor/edk2/releases/latest)
138 $ wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
139 $ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.r…
140 $ wget https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/download/0.4.2/hypervi…
144 use a `cloud-init` disk image to customise the image on the first boot. A basic
145 `cloud-init` image is generated by this [script](scripts/create-cloud-init.sh).
147 is only necessary to add this disk image on the first boot. Script also assigns
148 default IP address using `test_data/cloud-init/ubuntu/local/network-config` details
149 with `--net "mac=12:34:56:78:90:ab,tap="` option. Then the matching mac address
150 interface will be enabled as per `network-config` details.
153 $ sudo setcap cap_net_admin+ep ./cloud-hypervisor
154 $ ./create-cloud-init.sh
155 $ ./cloud-hypervisor \
156 	--kernel ./hypervisor-fw \
157 	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
158 	--cpus boot=4 \
159 	--memory size=1024M \
160 	--net "tap=,mac=,ip=,mask="
165 `virtio-console`.
168 $ ./cloud-hypervisor \
169 	--kernel ./hypervisor-fw \
170 	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
171 	--cpus boot=4 \
172 	--memory size=1024M \
173 	--net "tap=,mac=,ip=,mask=" \
174 	--serial tty \
175 	--console off
182 Cloud Hypervisor also supports direct kernel boot. For x86-64, a `vmlinux` ELF kernel (compiled wit…
188 $ git clone --depth 1 https://github.com/cloud-hypervisor/linux.git -b ch-6.12.8 linux-cloud-hyperv…
189 $ pushd linux-cloud-hypervisor
191 # Do native build of the x86-64 kernel
192 $ KCFLAGS="-Wa,-mx86-used-note=no" make bzImage -j `nproc`
194 $ make -j `nproc`
198 For x86-64, the `vmlinux` kernel image will then be located at
199 `linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin`.
201 `linux-cloud-hypervisor/arch/arm64/boot/Image`.
209 $ wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img # x86-64
210 $ wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-arm64.img # AArch64
211 $ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.r…
212 $ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-arm64.img focal-server-cloudimg-arm64.r…
220 - x86-64
223 $ sudo setcap cap_net_admin+ep ./cloud-hypervisor
224 $ ./create-cloud-init.sh
225 $ ./cloud-hypervisor \
226 	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
227 	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
228 	--cmdline "console=hvc0 root=/dev/vda1 rw" \
229 	--cpus boot=4 \
230 	--memory size=1024M \
231 	--net "tap=,mac=,ip=,mask="
234 - AArch64
237 $ sudo setcap cap_net_admin+ep ./cloud-hypervisor
238 $ ./create-cloud-init.sh
239 $ ./cloud-hypervisor \
240 	--kernel ./linux-cloud-hypervisor/arch/arm64/boot/Image \
241 	--disk path=focal-server-cloudimg-arm64.raw path=/tmp/ubuntu-cloudinit.img \
242 	--cmdline "console=hvc0 root=/dev/vda1 rw" \
243 	--cpus boot=4 \
244 	--memory size=1024M \
245 	--net "tap=,mac=,ip=,mask="
248 If earlier kernel messages are required the serial console should be used instead of `virtio-consol…
250 - x86-64
253 $ ./cloud-hypervisor \
254 	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
255 	--console off \
256 	--serial tty \
257 	--disk path=focal-server-cloudimg-amd64.raw \
258 	--cmdline "console=ttyS0 root=/dev/vda1 rw" \
259 	--cpus boot=4 \
260 	--memory size=1024M \
261 	--net "tap=,mac=,ip=,mask="
264 - AArch64
267 $ ./cloud-hypervisor \
268 	--kernel ./linux-cloud-hypervisor/arch/arm64/boot/Image \
269 	--console off \
270 	--serial tty \
271 	--disk path=focal-server-cloudimg-arm64.raw \
272 	--cmdline "console=ttyAMA0 root=/dev/vda1 rw" \
273 	--cpus boot=4 \
274 	--memory size=1024M \
275 	--net "tap=,mac=,ip=,mask="
297   substantially between releases: TDX, vfio-user, vDPA.
301 As of 2023-01-03, the following cloud images are supported:
303 - [Ubuntu Focal](https://cloud-images.ubuntu.com/focal/current/) (focal-server-cloudimg-{amd64,arm6…
304 - [Ubuntu Jammy](https://cloud-images.ubuntu.com/jammy/current/) (jammy-server-cloudimg-{amd64,arm6…
305 - [Ubuntu Noble](https://cloud-images.ubuntu.com/noble/current/) (noble-server-cloudimg-{amd64,arm6…
306 - [Fedora 36](https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/36/Cloud/) ([Fed…
315 `virtio-{net,block,pmem,fs,vsock}` and memory resizing. This
316 [document](docs/hotplug.md) details how to add devices to a running VM.
318 ## Device Model
320 Details of the device model can be found in this
326 project](https://github.com/orgs/cloud-hypervisor/projects/6).
330 In order to satisfy the design goal of having a high-performance,
331 security-focused hypervisor the decision was made to use the
332 [Rust](https://www.rust-lang.org/) programming language. The language's strong
337 importing the [Rust VMM](https://github.com/rust-vmm) crates, and sharing code
339 [Firecracker](https://firecracker-microvm.github.io/) and Google's
343 to share and re-use as many virtualization crates as possible.
361 described in the [Community](https://github.com/cloud-hypervisor/community)
378 Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNj…
379 …[join us on Slack](https://cloud-hypervisor.slack.com/), and [participate in our community activit…
384 tracker](https://github.com/cloud-hypervisor/cloud-hypervisor/issues) but for