Skip to content

Use single virtiofs device for all shares - #41129

Merged
Daman Mulye (damanm24) merged 15 commits into
masterfrom
user/damanmulye/aggregate_share
Jul 22, 2026
Merged

Use single virtiofs device for all shares#41129
Daman Mulye (damanm24) merged 15 commits into
masterfrom
user/damanmulye/aggregate_share

Conversation

@damanm24

Copy link
Copy Markdown
Contributor

With virtio-fs enabled, each drvfs share was backed by its own virtio-fs device. Each device consumes an MMIO aperture, of which there are a fixed amount available for a VM. When a VM runs out of them subsequent calls to mount shares will hang.

To work around this limit, all drvfs shares are backed by a single aggregate virtio-fs device per server (one for the non-admin drvfs, and another for the admin). The aggregate device exposes a synthetic root with one child per share keyed by a GUID subname. Adding a share now adds a child to the existing device instead of creating a new device. In the guest the aggregate device is mounted once at a hidden location (/run/wsl/virtiofs-mounts/drvfsa) and each share is a bind-mount of <device>/<child-name> -> target.

Key Changes:
Wsl Service:

  • WslCoreVm::AddVirtioFsShare now creates the aggregate device on first use (VirtiofsShareKind_Aggregate) and calls AddVirtiofsChild(device, childName, options, path) for every share
  • New GuestDeviceManager::AddVirtiofsChild / DeviceHostProxy::AddVirtiofsChild + GetVirtiofsDevice, forwarding to the device-host IWslVirtiofsDevice::AddChild.

Protocol changes:

  • LX_INIT_ADD_VIRTIOFS_SHARE_RESPONSE_MESSAGE gains ChildNameOffset so the guest learns which child to bind.

Guest init changes:

  • MountVirtioFs mounts the aggregate root once, then bind-mounts <VIRTIOFS_MOUNT_DIR>// → target. Falls back to Plan 9 if the share-add or the bind fails (so shares on volumes the server can't fully back still mount).
  • QueryVirtiofsMountSource takes a Root argument to resolve an aggregate child back to its Windows path; config.cpp mount enumeration/remount handles the aggregate (non-/ root) case and skips the internal hidden mounts.
  • UtilFindMount excludes internal mounts under VIRTIOFS_MOUNT_DIR from reverse (Windows→Linux) translation, so wslpath resolves to the user-facing /mnt/c instead of the internal /run/wsl/virtiofs-mounts/... path.

Copilot AI review requested due to automatic review settings July 21, 2026 16:16
@damanm24
Daman Mulye (damanm24) requested a review from a team as a code owner July 21, 2026 16:16
Comment thread src/linux/init/drvfs.cpp Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR changes WSL’s virtio-fs-backed drvfs sharing model from “one virtio-fs device per share” to a single aggregate virtio-fs device per server (separate devices for admin vs non-admin), avoiding MMIO aperture exhaustion that can otherwise cause subsequent mounts to hang.

Changes:

  • Introduces an aggregate virtio-fs device that exposes per-share children keyed by GUID, and updates the host↔guest protocol to return the child name for binding.
  • Updates Linux init to mount the aggregate root once under a hidden directory and bind-mount per-share children to user-visible targets; updates mount enumeration / reverse path translation to ignore internal mounts.
  • Updates Windows and Linux tests to reflect aggregate virtio-fs behavior (mount root semantics, mount-id lookup, and exercising read-only mounts).

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/windows/DrvFsTests.cpp Expands virtiofs loop test and adds alternating read-only mounts with validation.
test/linux/unit_tests/lxtmount.c Adjusts mount root checks and mount-id lookup for aggregate virtio-fs semantics.
test/linux/unit_tests/drvfs.c Updates test applicability/behavior notes for virtiofs parity with Plan9 limitations.
src/windows/service/exe/WslCoreVm.h Changes AddVirtioFsShare return type and adds aggregate device instance tracking.
src/windows/service/exe/WslCoreVm.cpp Implements aggregate virtio-fs device + per-share child creation; includes protocol response updates.
src/windows/common/GuestDeviceManager.h Adds AddVirtiofsChild API.
src/windows/common/GuestDeviceManager.cpp Implements AddVirtiofsChild forwarding to the device host.
src/windows/common/DeviceHostProxy.h Adds AddVirtiofsChild and GetVirtiofsDevice helpers.
src/windows/common/DeviceHostProxy.cpp Implements virtio-fs device child creation and virtiofs device lookup.
src/shared/inc/lxinitshared.h Extends add-virtiofs-share response with ChildNameOffset.
src/linux/init/util.cpp Skips internal aggregate mounts during reverse translation; adjusts virtiofs source resolution.
src/linux/init/drvfs.h Defines VIRTIOFS_MOUNT_DIR and extends QueryVirtiofsMountSource signature.
src/linux/init/drvfs.cpp Adds aggregate-root + child bind mounting logic; updates tag mapping and source query behavior.
src/linux/init/config.cpp Updates mount enumeration/remount logic to handle aggregate virtio-fs and skip internal mounts.
packages.config Updates Microsoft.WSL.DeviceHost package version.

Comment thread src/linux/init/drvfs.cpp
Comment thread src/linux/init/drvfs.cpp
Comment thread src/linux/init/drvfs.cpp Outdated
Comment thread src/linux/init/util.cpp
Comment thread src/linux/init/config.cpp
Comment thread src/windows/common/DeviceHostProxy.cpp
Comment thread src/linux/init/drvfs.cpp
Comment thread src/windows/common/DeviceHostProxy.cpp
Comment thread test/linux/unit_tests/drvfs.c
@Brian-Perkins

Copy link
Copy Markdown
Contributor

I suggest we make this a config option and set it to enabled by default, because (1) there may be issues that require disabling it; (2) longer term we are probably moving back to individual devices so it would probably be good to maintain and test that code. Agreed though that it is a pain to have to test both variations.

Copilot AI review requested due to automatic review settings July 21, 2026 19:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Comment thread test/windows/DrvFsTests.cpp Outdated
Comment thread src/windows/common/GuestDeviceManager.h Outdated
Comment thread src/windows/common/GuestDeviceManager.cpp
Copilot AI review requested due to automatic review settings July 21, 2026 19:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Comment thread test/linux/unit_tests/lxtmount.c Outdated
Comment thread src/linux/init/drvfs.cpp
Comment thread src/linux/init/drvfs.cpp Outdated
Comment thread src/linux/init/drvfs.cpp
@benhillis

Copy link
Copy Markdown
Member

It does not look like WSLC was updated to use the new logic.

@benhillis

Copy link
Copy Markdown
Member

It does not look like WSLC was updated to use the new logic.

That can be done as a follow-up if you like.

Copilot AI review requested due to automatic review settings July 21, 2026 21:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Comment thread test/linux/unit_tests/lxtmount.c Outdated
Copilot AI review requested due to automatic review settings July 21, 2026 23:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comment thread src/linux/init/util.cpp
Comment on lines +889 to +892
if (UtilIsPathPrefix(MountEnum.Current().MountPoint, VIRTIOFS_MOUNT_DIR, false) > 0)
{
continue;
}
Comment thread src/linux/init/drvfs.cpp
Comment on lines 905 to 908
//
// Validate the tag is a GUID.
//

@benhillis Ben Hillis (benhillis) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for doing this!

@damanm24
Daman Mulye (damanm24) merged commit 9e58620 into master Jul 22, 2026
12 checks passed
@damanm24
Daman Mulye (damanm24) deleted the user/damanmulye/aggregate_share branch July 22, 2026 15:06
Ben Hillis (benhillis) pushed a commit that referenced this pull request Jul 24, 2026
Now that DrvFs shares are carried by a single aggregate virtiofs device
(#41129 / #41151), only a small, fixed number of virtiofs devices exist
per VM. That leaves enough memory-aperture headroom to give each
aggregate multiple request queues, which significantly improves
concurrent DrvFs throughput.

Non-aggregate (per-share file-backed) and section-backed devices stay at
a single queue: many of them can exist, and raising their queue count
would reintroduce the aperture exhaustion that the one-queue default was
added to avoid.

fio on /mnt/c (16 parallel jobs, psync), 4 queues vs 1:
  randread 4k        +56% IOPS  (-36% latency)
  randwrite 4k       +622% IOPS (-85% latency)
  smallfile randrw   +48% IOPS  (-33% latency)
  seqread 1M         +15% IOPS
  seqwrite 1M        -10% IOPS  (single-stream, bandwidth-bound; within noise)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4ab64eb9-4271-4241-b838-3da58ec39305
Ben Hillis (benhillis) added a commit that referenced this pull request Jul 24, 2026
Now that DrvFs shares are carried by a single aggregate virtiofs device
(#41129 / #41151), only a small, fixed number of virtiofs devices exist
per VM. That leaves enough memory-aperture headroom to give each
aggregate multiple request queues, which significantly improves
concurrent DrvFs throughput.

Non-aggregate (per-share file-backed) and section-backed devices stay at
a single queue: many of them can exist, and raising their queue count
would reintroduce the aperture exhaustion that the one-queue default was
added to avoid.

fio on /mnt/c (16 parallel jobs, psync), 4 queues vs 1:
  randread 4k        +56% IOPS  (-36% latency)
  randwrite 4k       +622% IOPS (-85% latency)
  smallfile randrw   +48% IOPS  (-33% latency)
  seqread 1M         +15% IOPS
  seqwrite 1M        -10% IOPS  (single-stream, bandwidth-bound; within noise)

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4ab64eb9-4271-4241-b838-3da58ec39305
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants