Skip to content

podman run fails with WSL v3.0.1 #29749

Closed
podman-container-tools/podman-machine-os
#319

Description

@l0rd

Issue Description

The following error is returned when trying podman run hello on the latest version of WSL (v3.0.1):

> podman run hello 
Error: preparing container 175dfadfb2db1773aaeb9f82994a347e7fd0d8bff927d5223b5bd65f5e905017 for attach: crun: 
controller `pids` is not available under /sys/fs/cgroup/wsl-user/distro-440/user.slice/user-1000.slice/user@1000.service/user.slice/libpod-175dfadfb2db1773aaeb9f82994a347e7fd0d8bff927d5223b5bd65f5e905017.scope/container/cgroup.controllers: OCI runtime error
🟢 Workaround 1: use --cgroups disabled
podman run --cgroups=disabled hello
🟢 Workaround 2: set cgroup_manager="cgroupfs" in containers.conf
Add-Content -Path "$env:APPDATA\containers\containers.conf" -Value "[engine]`ncgroup_manager=`"cgroupfs`"" -NoNewLine

Steps to reproduce the issue

Steps to reproduce the issue

  1. wsl --update # <-- this installs version 3.0.1
  2. podman machine init --now
  3. podman run hello

podman info output

Client:
  APIVersion: 6.2.0-dev
  Built: 1788887757
  BuiltTime: Tue Sep  8 19:15:57 2026
  GitCommit: c30add3f1ca485088750a73119dfb5f85312bcc1-dirty
  GoVersion: go1.27.0
  Os: windows
  OsArch: windows/arm64
  Version: 6.2.0-dev
host:
  arch: arm64
  buildahVersion: 1.45.0
  cdiSpecDirs:
  - /etc/cdi
  - /var/run/cdi
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.2.1-2.fc44.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.2.1, commit: '
  cpuUtilization:
    idlePercent: 99.12
    systemPercent: 0.72
    userPercent: 0.16
  cpus: 12
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: podman-machine-os
    version: "44"
  eventLogger: journald
  freeLocks: 2047
  hostname: rascarcapac
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 1000000
  kernel: 6.18.40.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memAvailable: 15856488448
  memFree: 15610957824
  memTotal: 16512077824
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    defaultNetwork: podman
    dns:
      package: aardvark-dns-2.1.0-1.20260811105353540362.main.6.g2d364eb.fc44.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 2.1.0
    package: netavark-2.1.0-1.20260818222846495485.main.59.g9a10e462.fc44.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 2.1.0
  ociRuntime:
    name: crun
    package: crun-1.29.1-1.20260909091116491877.main.173.g322729a7.fc44.aarch64
    path: /usr/bin/crun
    version: |-
      crun version UNKNOWN
      commit: 72748b64bf2828c558a955c66e0fe3f5ecd8ca59
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +JSON_C
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20260728.gf8df3f1-2.fc44.aarch64
    version: |
      pasta 0^20260728.gf8df3f1-2.fc44.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: unix:///run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  rootlessPortForwarder: rootlessport
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 0h 2m 20.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 1112948736
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 6.2.0-dev
  BuildOrigin: 'Copr: rhcontainerbot/podman-next'
  Built: 1788825600
  BuiltTime: Tue Sep  8 02:00:00 2026
  GitCommit: 5affbadb26ee52ebe205051368f16921c9497b43
  GoVersion: go1.26.8-X:nodwarf5
  Os: linux
  OsArch: linux/arm64
  Version: 6.2.0-dev

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Activity

  1. added
    kind/bugCategorizes issue or PR as related to a bug.
    on Sep 10, 2026
  2. l0rd commented on Sep 10, 2026

    @l0rd
    ContributorAuthor
  3. ghimisradu commented on Sep 10, 2026

    @ghimisradu

    This happens even though pids is correctly delegated at every level above the container's own leaf cgroup:

    • Root cgroup.subtree_control: cpuset cpu io memory hugetlb pids rdma (includes pids)
    • /etc/systemd/system/user@.service.d/delegate.conf (shipped by default in the machine image): Delegate=memory pids cpu io (includes pids)
    • Intermediate user.slice / user-1000.slice / user@1000.service levels: pids present

    Despite this, the transient libpod-<id>.scope/container cgroup crun creates for the container ends up with an empty cgroup.controllers file — no pids, even though the immediate parent should be delegating it.

    Behavior is inconsistent across images: a plain alpine sleep 600 container sometimes starts successfully, while openbao/openbao fails consistently.

    Spend couple of hours on this, ended up starting my container with --cgroups=disabled.

    everything was freshly installed

  4. WitherZuo commented on Sep 10, 2026

    @WitherZuo

    also have this bug on Windows amd64, with WSL 2.9.11. the image is ghcr.io/pnpm/pnpm

  5. godefroi commented on Sep 12, 2026

    @godefroi

    I just hit this after clean-upgrading from 5.8.2 to 6.1.1. I got the same error text, same kernel (6.18.40.1-1), but on WSL 2.9.8.0 rather than the 2.9.11 pre-release, and on x64 rather than ARM64.

    Setting cgroup_manager = "cgroupfs" was a WSL-specific workaround baked into /etc/containers/containers.conf at machine-init time (see the containersConf const in pkg/machine/wsl/declares.go, pre-6.0). With cgroupfs, podman never asks systemd to create a transient scope, so the WSL delegation gap above never gets hit. That override was removed in 81fd1e6e ("machine/wsl: drop cgroup_manager cgroupfs"), which shipped between v6.0.0-rc1 and v6.0.0.

    The problem seems to be that nothing seeds %APPDATA%\containers with defaults, so the default cgroup manager falls through to systemd.

    Thus, it seems to me that any fresh machine init on the WSL provider from 6.0.0 onward is just broken.

    Workarounds that fixed it for me, for anyone hitting this before (presumably) a fix happens:

    • Per-run: podman run --cgroups=disabled ...
    • Persistent: manually add cgroup_manager = "cgroupfs" under [engine] in %APPDATA%\containers\containers.conf

    I'm happy to share more diagnostic detail if useful.

  6. changed the title [-]`podman run` fails with WSL pre-release (2.9.11) on Windows ARM64[/-] [+]`podman run` fails with WSL pre-release (v2.9.x)[/+] on Sep 17, 2026
  7. l0rd commented on Sep 25, 2026

    @l0rd
    ContributorAuthor

    The root cause is that in WSL 2.9.x, the podman-machine distribution cgroups sit in a non-systemd subtree, along with cgroups for other processes such as WSLg that belong to the WSL root namespace. This prevents the cgroup controllers from propagating when a container starts. More info here.

    Using cgroup_manager=cgroupfs is a valid workaround, and I have opened a PR to make it the default for WSL, but it disables cgroups in rootless mode (the default) and this is not ideal. We should eventually document this.

    The clean solution is to use the built-in WSL systemd support, but that requires changes to our code. I verified that, in that case, the pids controller is correctly propagated with WSL 2.9.x, and I opened a draft PR to run CI checks.

  8. jstavats commented on Sep 29, 2026

    @jstavats

    It's not pre-release anymore. WSL 3.0.1 is out today. I'm running into this issue.

  9. changed the title [-]`podman run` fails with WSL pre-release (v2.9.x)[/-] [+]`podman run` fails with WSL v3.0.1[/+] on Sep 30, 2026
  10. 4 remaining items

  11. added 2 commits that reference this issue on Oct 2, 2026
    2c56b3c
    b420cc6
  12. LeboyX commented on Oct 2, 2026

    @LeboyX

    TL;DR;

    Steps followed and work as expected. Problem worked-around. Thanks so much, @l0rd!

    Details

    As requested, I ran the diagnostic commands. It appears that, indeed, cgroupfs
    isn't available, despite the config files.

    # Powershell
    PS C:\Users\Eric Liebowitz> (podman info -f json | ConvertFrom-Json).host.cgroupManager
    systemd
    
    PS C:\Users\Eric Liebowitz> podman machine ssh cat /etc/containers/containers.conf
    cat: /etc/containers/containers.conf: No such file or directory
    
    # Directly on podman-default-machine
    [user@US-PW0GDNXJ ~]$ cat /etc/containers/containers.conf
    cat: /etc/containers/containers.conf: No such file or directory
    

    I followed your additional workaround steps, and can confirm the fix avoids
    the problem.

    # On podman-default-machine
    [user@US-PW0GDNXJ ~]$ mkdir -p ~/.config/containers
    [user@US-PW0GDNXJ ~]$ printf "[engine]\ncgroup_manager=\"cgroupfs\"\n" > ~/.config/containers/containers.conf
    
    # On Windows
    $: docker run hello
    !... Hello Podman World ...!
    
             .--"--.
           / -     - \
          / (O)   (O) \
       ~~~| -=(,Y,)=- |
        .---. /`  \   |~~
     ~/  o  o \~~~~.----. ~~
      | =(X)= |~  / (O (O) \
       ~~~~~~~  ~| =(Y_)=-  |
      ~~~~    ~~~|   U      |~~
    
    Project:   https://github.com/containers/podman
    Website:   https://podman.io
    Desktop:   https://podman-desktop.io
    Documents: https://docs.podman.io
    YouTube:   https://youtube.com/@Podman
    X/Twitter: @Podman_io
    Mastodon:  @Podman_io@fosstodon.org
    

    And, I can confirm that my Windows user does have a space in the username.
    This causes me problems elsewhere, too, every time I get a new work laptop and
    IT puts that space in there.

  13. LeboyX commented on Oct 2, 2026

    @LeboyX

    Just to follow up, though: I can't run or create containers via Podman Desktop, even w/ these configs set. And thus, presumably, a suite of Java integration test I have (which use Testcontainers to talk to Podman) also can't run. So, while the workaround appears to work from the command line, it doesn't seem to be influencing the behavior for more complex tasks.

  14. pelaxa commented on Oct 2, 2026

    @pelaxa

    some of the instructions above say to check for containers.conf in /etc/containers, but then other say to create it in ~/.config/containers/

    I copied the file from ~/.config/containers to /etc/containers and finally got things working. I did restart the machine after these changes.

  15. godefroi commented on Oct 2, 2026

    @godefroi

    some of the instructions above say to check for containers.conf in /etc/containers, but then other say to create it in ~/.config/containers/

    I believe that the design is that containers.conf in ~/.config/containers will get mounted into your podman machine wsl image. It seems from discussion here that sometimes it doesn't.

  16. pelaxa commented on Oct 2, 2026

    @pelaxa

    some of the instructions above say to check for containers.conf in /etc/containers, but then other say to create it in ~/.config/containers/

    I believe that the design is that containers.conf in ~/.config/containers will get mounted into your podman machine wsl image. It seems from discussion here that sometimes it doesn't.

    That is different then. the instructions above are creating the file in ~/.config/containers/ on the machine. but you are saying to creating it on windows and it should get mounted.

    I tried your way too but did not work for me.

  17. pelaxa commented on Oct 2, 2026

    @pelaxa

    mmm....now port mappings are not working. this seems to all be related to WSL 3.0.1. I do not recall changing anything else and my compose file worked a week ago.

    I can get to the ports from the podman-machine but not from windows.

    Edit --
    Seems like since I was mapping ports to the windows loopback 127.0.0.1:8080:8080, now the service is only bound to the podman-machine's loopback. removing the IP makes it accessible on the podman-machine's IP as well from windows, but still does not bind it to any windows IPs.

  18. l0rd commented on Oct 2, 2026

    @l0rd
    ContributorAuthor

    @LeboyX Podman Desktop configures the machine in Rootful mode and the instructions I have provided only work when the machine is configured in Rootless mode. My bad. Use the following command to make it work for both Rootful and Rootless:

    podman machine ssh --username root 'mkdir -p /usr/share/containers/containers.conf.d/'
    podman machine ssh --username root 'printf "[engine]\ncgroup_manager=\"cgroupfs\"\n" > /usr/share/containers/containers.conf.d/999-podman-machine-wsl-cgroupfs.conf'

    I believe that the design is that containers.conf in ~/.config/containers will get mounted into your podman machine wsl image. It seems from discussion here that sometimes it doesn't.

    @pelaxa folder %APPDATA%/containers gets mounted into the Podman machine, not ~/.config/containers. And because of a bug (thanks @LeboyX for finding this out), that folder doesn't get mounted if the %APPDATA% has spaces (for example, C:\Users\John Doe\AppData\Roaming). This bug is also addressed by this PR.

  19. brunnert commented on Oct 3, 2026

    @brunnert

    mmm....now port mappings are not working. this seems to all be related to WSL 3.0.1. I do not recall changing anything else and my compose file worked a week ago.

    I can get to the ports from the podman-machine but not from windows.

    Edit -- Seems like since I was mapping ports to the windows loopback 127.0.0.1:8080:8080, now the service is only bound to the podman-machine's loopback. removing the IP makes it accessible on the podman-machine's IP as well from windows, but still does not bind it to any windows IPs.

    @pelaxa I am experiencing the same issue as you after applying the initial cgroup fix. With WSL 3.0.1 and Podman Desktop v1.29.3, the container ports are no longer exposed to the Windows host.

    Edit: Switching to rootless mode seems to fix the port binding issue. After applying this command, the ports are available and the cgroups issue disappears.

    podman machine stop
    podman machine set --rootful=false
    wsl --shutdown
    podman machine start
    podman machine ssh --username root "mkdir -p /usr/share/containers/containers.conf.d && printf '[engine]\ncgroup_manager = \047cgroupfs\047\n' > /usr/share/containers/containers.conf.d/999-podman-machine-wsl-cgroupfs.conf"
    podman machine stop
    podman machine start
    
  20. pelaxa commented on Oct 3, 2026

    @pelaxa

    @brunnert thanks for this. your instructions worked great.

    @l0rd FWIW, my APPDATA does not have a space in it: c:\Users\pelaxa\AppData\Roaming. However, when I look at this folder in File Explorer, then it does since it replaces pelaxa with Pelaxa Inc.

  21. l0rd commented on Oct 5, 2026

    @l0rd
    ContributorAuthor

    @pelaxa @brunnert the port mapping problem seams related to #29377 that got fixed in v6.1.1.

  22. added a commit that references this issue on Oct 6, 2026
    8d67185
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.machineremoteProblem is in podman-remotewindowsissue/bug on Windows

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions