Skip to content

dockerfile: run buildkitd within a cgroup namespace for cgroup v2 #6368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tonistiigi merged 2 commits into from
Feb 24, 2026
Merged

dockerfile: run buildkitd within a cgroup namespace for cgroup v2 #6368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9a1bf2a
dockerfile: run buildkitd within a cgroup namespace for cgroup v2
marxarelli Nov 16, 2025
93f176a
hack: write cgroup namespace progress message to stderr
crazy-max Feb 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ FROM scratch AS release
COPY --link --from=releaser /out/ /

FROM alpine:${ALPINE_VERSION} AS buildkit-export-alpine
RUN apk add --no-cache fuse3 git openssh openssl pigz xz iptables ip6tables \
RUN apk add --no-cache fuse3 git openssh openssl pigz xz iptables ip6tables util-linux-misc \
&& ln -s fusermount3 /usr/bin/fusermount
COPY --link examples/buildctl-daemonless/buildctl-daemonless.sh /usr/bin/
VOLUME /var/lib/buildkit
Expand Down Expand Up @@ -398,7 +398,9 @@ EOT
FROM buildkit-export AS buildkit-linux
COPY --link --from=binaries / /usr/bin/
ENV BUILDKIT_SETUP_CGROUPV2_ROOT=1
ENTRYPOINT ["buildkitd"]
COPY --link hack/buildkitd-entrypoint /usr/bin/buildkitd-entrypoint
COPY --link hack/with-cgroupfs-remount /usr/bin/with-cgroupfs-remount
ENTRYPOINT ["/usr/bin/buildkitd-entrypoint"]

@AkihiroSuda AkihiroSuda Dec 3, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of docker run --privileged this script does not seem needed, as Docker unshares cgroupns even for privileged mode.
So this entrypoint script should be opt-in.
It should be also marked as a workaround until Kubernetes supports unsharing cgroupns.

@AkihiroSuda AkihiroSuda Dec 3, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KEP:

@AkihiroSuda AkihiroSuda Dec 19, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @marxarelli

@marxarelli marxarelli Jan 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening that KEP @AkihiroSuda. I added some words of support in the PR discussion.

I'll work on refactoring this PR to be opt-in as you requested.

@marxarelli marxarelli Jan 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of docker run --privileged this script does not seem needed, as Docker unshares cgroupns even for privileged mode. So this entrypoint script should be opt-in. It should be also marked as a workaround until Kubernetes supports unsharing cgroupns.

The entrypoint should already handle this case heuristically by testing whether the current cgroup path is "/" (which is true when run via docker run --privileged unless --cgroupns host is also given). In that case, it skips unshare and the with-cgroupfs-remount wrapper script and just does exec buildkitd.

I could also introduce an additional environment variable if you think that's best, but I personally quite like the heuristic behavior.

@marxarelli marxarelli Jan 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added a comment about it being a workaround.


FROM buildkit-linux AS buildkit-linux-debug
COPY --link --from=dlv /out/dlv /usr/bin/dlv
Expand Down
22 changes: 22 additions & 0 deletions hack/buildkitd-entrypoint
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/sh
#
# For cgroup v2, ensure buildkitd has a namespaced view of /sys/fs/cgroup by
# running in a new cgroup and mount namespace and remounting /sys/fs/cgroup.
# Assume we are already in our own cgroup ns if the current cgroup path is
# "/".
#
# Note this is a workaround for the lack of cgroupns control in the Kubernetes
# API. If KEP-5714 is adopted, this can eventually be removed.
#
# See https://github.com/kubernetes/enhancements/issues/5714

set -e

if [ -e /sys/fs/cgroup/cgroup.controllers ]; then
if [ "$(cut -d: -f3 /proc/self/cgroup)" != "/" ]; then
echo creating cgroup namespace >&2
exec /usr/bin/unshare --cgroup --mount /usr/bin/with-cgroupfs-remount /usr/bin/buildkitd "$@"
fi
fi

exec /usr/bin/buildkitd "$@"
8 changes: 8 additions & 0 deletions hack/with-cgroupfs-remount
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
set -e

options="$(awk '$2 == "/sys/fs/cgroup" { print $4 }' /proc/self/mounts)"
umount /sys/fs/cgroup
mount -t cgroup2 -o "$options" cgroup2 /sys/fs/cgroup

exec "$@"