Use 4 request queues for aggregate virtiofs devices - #41169
Merged
Conversation
Now that DrvFs shares are carried by a single aggregate virtiofs device (#41129 / #41151), only a small, fixed number of virtiofs devices exist per VM. That leaves enough memory-aperture headroom to give each aggregate multiple request queues, which significantly improves concurrent DrvFs throughput. Non-aggregate (per-share file-backed) and section-backed devices stay at a single queue: many of them can exist, and raising their queue count would reintroduce the aperture exhaustion that the one-queue default was added to avoid. fio on /mnt/c (16 parallel jobs, psync), 4 queues vs 1: randread 4k +56% IOPS (-36% latency) randwrite 4k +622% IOPS (-85% latency) smallfile randrw +48% IOPS (-33% latency) seqread 1M +15% IOPS seqwrite 1M -10% IOPS (single-stream, bandwidth-bound; within noise) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4ab64eb9-4271-4241-b838-3da58ec39305
Ben Hillis (benhillis)
force-pushed
the
user/benhill/virtiofs-queue-count
branch
from
July 24, 2026 22:02
52a9b6b to
276a8e7
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR increases virtio-fs request parallelism for the new aggregate DrvFs virtiofs device(s) by assigning multiple request queues, while keeping non-aggregate devices at a single queue to avoid reintroducing MMIO aperture pressure.
Changes:
- Set the virtiofs
queueCountto 4 forVirtiofsShareKind_Aggregatedevices. - Keep
queueCountat 1 for all other virtiofs device kinds (per-share file-backed and section-backed).
Blue (OneBlue)
approved these changes
Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Now that DrvFs shares are carried by a single aggregate virtiofs device (#41129 / #41151), only a small, fixed number of virtiofs devices exist per VM. That leaves enough memory-aperture headroom to give each aggregate 4 request queues instead of 1, significantly improving concurrent DrvFs throughput.
Non-aggregate (per-share file-backed) and section-backed devices stay at a single queue: many of them can exist, and raising their queue count would reintroduce the aperture exhaustion that the one-queue default was added to avoid.
Benchmarks
fio on
/mnt/c(16 parallel jobs, psync engine), 4 queues vs 1 queue:Multiqueue is a decisive win on concurrent random/metadata workloads. The only regression is single-stream sequential write (â10%), which is bandwidth-bound rather than concurrency-bound and within run-to-run noise.
Testing
drvfsa) exposes 4 request queues (virtio1-requests.0â3) while per-share devices remain at 1.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com