1# Performance Metrics 2 3Cloud Hypervisor provides a [performance metrics](https://github.com/cloud-hypervisor/cloud-hypervisor/tree/main/performance-metrics) 4binary for users to generate metrics data from their own 5environment. This document describes how to generate metrics data 6quickly by using Cloud Hypervisor's development script, 7e.g. `dev_cli.sh`. The only prerequisite is [Docker installation](https://docs.docker.com/engine/install/). 8Please note that upon its first invocation, this script will pull a 9fairly large container image. 10 11## Run the Performance Tests 12 13To generate metrics data for all available performance tests (including 14boot time, block I/O throughput, and network throughput & latency) and 15output the result into a json file: 16 17``` 18$ ./scripts/dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json 19``` 20 21To get a list of available performance tests: 22 23``` 24$ ./scripts/dev_cli.sh tests --metrics -- -- --list-tests 25``` 26 27To generate metrics data for selected performance tests, e.g. boot time only: 28 29``` 30$ ./scripts/dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json --test-filter boot_time 31``` 32 33To set custom timeout or test iterations for all performance tests: 34 35``` 36$ ./scripts/dev_cli.sh tests --metrics -- -- --timeout 5 --iterations 10 37``` 38 39## Performance Tests Details 40 41The following table lists the supported performance tests with default 42timeout and number of iterations. The `timeout` defines the maximum 43execution time of each test for each iteration. The `iteration` defines 44how many times a test needs to be executed to generate the final metrics 45data. 46 47| **Type** | **Metric** | **Timeout(s)** | **Iterations** | 48|------------|--------------------------------------------|----------------|----------------| 49| Boot Time | boot_time_ms | 2 | 10 | 50| | boot_time_pmem_ms | 2 | 10 | 51| | boot_time_16_vcpus_ms | 2 | 10 | 52| | boot_time_16_vcpus_pmem_ms | 2 | 10 | 53| Virtio Net | virtio_net_latency_us | 10 | 5 | 54| | virtio_net_throughput_single_queue_rx_gbps | 10 | 5 | 55| | virtio_net_throughput_single_queue_tx_gbps | 10 | 5 | 56| | virtio_net_throughput_multi_queue_rx_gbps | 10 | 5 | 57| | virtio_net_throughput_multi_queue_tx_gbps | 10 | 5 | 58| Block | block_read_MiBps | 10 | 5 | 59| | block_write_MiBps | 10 | 5 | 60| | block_random_read_MiBps | 10 | 5 | 61| | block_random_write_MiBps | 10 | 5 | 62| | block_multi_queue_read_MiBps | 10 | 5 | 63| | block_multi_queue_write_MiBps | 10 | 5 | 64| | block_multi_queue_random_read_MiBps | 10 | 5 | 65| | block_multi_queue_random_write_MiBps | 10 | 5 | 66| | block_read_IOPS | 10 | 5 | 67| | block_write_IOPS | 10 | 5 | 68| | block_random_read_IOPS | 10 | 5 | 69| | block_random_write_IOPS | 10 | 5 | 70| | block_multi_queue_read_IOPS | 10 | 5 | 71| | block_multi_queue_write_IOPS | 10 | 5 | 72| | block_multi_queue_random_read_IOPS | 10 | 5 | 73| | block_multi_queue_random_write_IOPS | 10 | 5 | 74| Other | restore_latency_time_ms | 2 | 10 | 75 76## Output Format 77 78Performance-metrics output the result into a json file if `report-file` 79param is set. The fields included in JSON include: 80 81| Field Name | Content | 82|--------------------|------------------------------------------| 83| git_human_readable | Recent tag information of git repository | 84| git_revision | Commit id of HEAD | 85| git_commit_date | Commit date of HEAD | 86| date | Date for executing the program | 87| results | A list of metrics | 88 89## Example 90 91Here is an example of generating metrics data for the boot time using 92`pmem`: 93 94```bash 95$ ./scripts/dev_cli.sh tests --metrics -- -- --test-filter boot_time_pmem_ms 96``` 97 98Here is a sample output: 99 100```json 101{ 102 "git_human_readable": "v40.0", 103 "git_revision": "e9b263975786abbf895469b93dfc00f21ce39a88", 104 "git_commit_date": "Fri Jun 21 08:40:44 2024 +0000", 105 "date": "Tue Jul 16 16:35:29 UTC 2024", 106 "results": [ 107 { 108 "name": "boot_time_pmem_ms", 109 "mean": 105.9461, 110 "std_dev": 7.140993312558129, 111 "max": 120.01499999999999, 112 "min": 92.37600000000002 113 } 114 ] 115} 116``` 117 118Note that the metrics data above is for illustration purpose only and 119does not represent the actual performance of Cloud Hypervisor on your 120system. 121