1# I/O Throttling 2 3Cloud Hypervisor now supports I/O throttling on virtio-block and virtio-net 4devices. This support is based on the [`rate-limiter` module](https://github.com/firecracker-microvm/firecracker/tree/7a1231b141e958d15d5b2c079dd5e0880528b4b0/src/rate_limiter) 5from Firecracker. This document explains the user interface of this 6feature, and highlights some internal implementations that can help users 7better understand the expected behavior of I/O throttling in practice. 8 9Cloud Hypervisor allows to limit both the I/O bandwidth (e.g. bytes/s) 10and I/O operations (ops/s) independently. For virtio-net devices, while 11sharing the same "rate limit" from user inputs (on both bandwidth and 12operations), the RX and TX queues are throttled independently. 13To limit the I/O bandwidth, Cloud Hypervisor 14provides three user options, i.e., `bw_size` (bytes), `bw_one_time_burst` 15(bytes), and `bw_refill_time` (ms). Both `bw_size` and `bw_refill_time` 16are required, while `bw_one_time_burst` is optional. 17Internally, these options define a TokenBucket with a maximum capacity 18(`bw_size` bytes), an initial burst size (`bw_one_time_burst`) and an 19interval for refilling purposes (`bw_refill_time`). The "refill-rate" is 20`bw_size` bytes per `bw_refill_time` ms, and it is the constant rate at 21which the tokens replenish. The refill process only starts happening 22after the initial burst budget is consumed. Consumption from the token 23bucket is unbounded in speed which allows for bursts bound in size by 24the amount of tokens available. Once the token bucket is empty, 25consumption speed is bound by the "refill-rate". Similarly, Cloud 26Hypervisor provides another three options for limiting I/O operations, 27i.e., `ops_size` (I/O operations), `ops_one_time_burst` (I/O operations), 28and `ops_refill_time` (ms). 29 30One caveat in the I/O throttling is that every-time the bucket gets 31empty, it will stop I/O operations for a fixed amount of time 32(`cool_down_time`). The `cool_down_time` now is fixed at `100 ms`, it 33can have big implications to the actual rate limit (which can be a lot 34different the expected "refill-rate" derived from user inputs). For 35example, to have a 1000 IOPS limit on a virtio-blk device, users should 36be able to provide either of the following two options: 37`ops_size=1000,ops_refill_time=1000` or 38`ops_size=10,ops_refill_time=10`. However, the actual IOPS limits are 39likely to be ~1000 IOPS and ~100 IOPS respectively. The reason is the 40actual rate limit users get can be as low as 41`ops_size/(ops_refill_time+cool_down_time)`. As a result, it is 42generally advisable to keep `bw/ops_refill_time` larger than `100 ms` 43(`cool_down_time`) to make sure the actual rate limit is close to users' 44expectation ("refill-rate"). 45 46## Rate Limit Groups 47 48It is possible to throttle the aggregate bandwidth or operations 49of multiple virtio-blk devices using a `rate_limit_group`. virtio-blk devices may be 50dynamically added and removed from a `rate_limit_group`. The following example 51demonstrates how to throttle the aggregate bandwidth of two disks to 10 MiB/s. 52 53``` 54--disk path=disk0.raw,rate_limit_group=group0 \ 55 path=disk1.raw,rate_limit_group=group0 \ 56--rate-limit-group bw_size=1048576,bw_refill_time,bw_refill_time=100 57``` 58