Repository navigation
solver: add proxy network mode - #6740
Conversation
f75eea5 to
419f7fd
Compare
github-actions
Bot
added
area/dependencies
419f7fd to
b07be8e
Compare
github-actions
Bot
added
area/buildkitd
and removed
area/dockerfile
area/frontend
labels
b07be8e to
a512836
Compare
|
Some examples and tutorial https://gist.github.com/tonistiigi/ced43318d7eaaea4a966de2ccff999ae |
7b17c01 to
eec6d25
Compare
eec6d25 to
0c33017
Compare
| insecure-entitlements = [ "network.host", "security.insecure", "device" ] | ||
| # proxyNetwork enables proxy network enforcement for all builds, disabled by default. | ||
| # It can also be enabled with buildkitd --proxy-network. | ||
| proxyNetwork = true |
There was a problem hiding this comment.
Can we have a comprehensive docs/proxy.md?
|
@AkihiroSuda @crazy-max Rebased. I've tested this quite a bit and it seems to work quite well. I also opened a new update in #6816 . I can combine them, but didn't want to make this PR even larger. |
| func newCA() ([]byte, *x509.Certificate, *rsa.PrivateKey, error) { | ||
| key, err := rsa.GenerateKey(rand.Reader, 2048) | ||
| if err != nil { | ||
| return nil, nil, nil, errors.WithStack(err) | ||
| } | ||
| now := time.Now() | ||
| tmpl := &x509.Certificate{ | ||
| SerialNumber: big.NewInt(1), | ||
| Subject: pkix.Name{CommonName: "BuildKit exec proxy"}, | ||
| NotBefore: now.Add(-time.Hour), | ||
| NotAfter: now.Add(24 * time.Hour), | ||
| KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature, | ||
| BasicConstraintsValid: true, | ||
| IsCA: true, | ||
| } | ||
| der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key) | ||
| if err != nil { | ||
| return nil, nil, nil, errors.WithStack(err) | ||
| } | ||
| cert, err := x509.ParseCertificate(der) | ||
| if err != nil { | ||
| return nil, nil, nil, errors.WithStack(err) | ||
| } | ||
| pemBytes := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}) | ||
| return pemBytes, cert, key, nil | ||
| } |
There was a problem hiding this comment.
iiuc the CA and leaf certs expire after 24h but are never rotated so HTTPS interception would break on any daemon up >24h?
| if err != nil { | ||
| return nil, errors.WithStack(err) | ||
| } | ||
| p.certs[host] = &cert |
There was a problem hiding this comment.
related to previous comment, caching leaf certs permanently and also stamps them with a 24h NotAfter. Even ignoring the CA, the first request to a host pins a leaf that becomes unusable 24h later and is never regenerated
There was a problem hiding this comment.
looking again at p.certs, it grows for every distinct host for the daemon's lifetime with no eviction. So a build that hits many hosts leaks memory potentially?
| return true | ||
| } | ||
|
|
||
| func New(opt Opt) (network.Provider, error) { |
There was a problem hiding this comment.
cniprovider calls cleanOldNamespaces(cp) in its constructor but not proxyprovider.New. So let's say after daemon crash mid-new NS, the bind-mounted netns files under root/net/proxy can leak across restarts? Maybe worth miroring CNI cleanup?
| if e.op.Network == pb.NetMode_PROXY { | ||
| e.proxyCap = network.NewProxyCapture() | ||
| meta.ProxyCapture = e.proxyCap | ||
| } |
There was a problem hiding this comment.
iiuc it creates ProxyCapture when the exec actually runs and for provenance is only records proxy materials if that runtime capture exists. What would happen for a cached proxy exec? I guess it will not re-run the proxy so current source-policy DENY/CONVERT rules are not re-evaluated and complete-materials=true can pass with missing HTTP materials potenaitally?
There was a problem hiding this comment.
Ah I missed the new deny test uses llb.IgnoreCache in policy_test to avoid this case.
Add a build request option that rewrites default exec networking to an internal proxy network while preserving explicit none networking. Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce source policy checks for proxied requests, and inject a temporary CA into Linux rootfs trust bundles for HTTPS interception. Share namespace pooling between CNI and proxy providers, and cover proxy mode with unit and integration tests. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Record successful GET responses through the exec proxy as provenance materials and report incomplete material coverage as a typed solve error. Thread proxy policy and capture state through typed executor/network options. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Route proxy network policy checks through the existing source policy evaluator so session metadata, deny messages, and URL converts use the same path as LLB sources. Keep proxy-specific request rewriting in the proxy provider. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Record each proxied exec request and print a redacted method and URL list in the exec progress logs after the process completes. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Apply proxy network as an explicit LLB mutation before digest recompute, while keeping runtime load options such as platform normalization applied when creating vertices. This preserves distinct cache keys for proxy-network builds without breaking gateway warning and source-map lookups that use the original LLB digests from the frontend. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add a proxyNetwork TOML setting and --proxy-network daemon flag to enable exec proxy enforcement for every build. Wire the default through controller and solver setup while preserving per-build enablement. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Add integration coverage for exec proxy source policy conversion. The test requests /foo, rewrites it to /bar, and verifies exported content and provenance materials use the converted source. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
0c33017 to
6f08a4a
Compare
Refresh generated proxy leaf certificates before they expire and cap cached hosts to avoid daemon-lifetime growth. Clean stale proxy network namespaces on provider startup to mirror CNI cleanup after daemon crashes. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
|
Really nice, I was just digging into this. |
|
It also seems like this makes the passthrough HTTP_PROXY not work when in proxy mode? |
Do they look up CAs from non-standard places? Interested in any examples that don't currently work.
Indeed that case would need some special handling. Essentially, we would need a double proxy in here. The |
|
Findings (including ones that don't need extra help):
|
Add a build request option that rewrites default exec networking to an
internal proxy network while preserving explicit none networking.
Route HTTP and HTTPS traffic through a BuildKit-owned proxy namespace, enforce
source policy checks for proxied requests, and inject a temporary CA into Linux
rootfs trust bundles for HTTPS interception.
Share namespace pooling between CNI and proxy providers, and cover proxy mode
with unit and integration tests.
Record successful GET responses through the exec proxy as provenance
materials and report incomplete material coverage as a typed solve error.
Thread proxy policy and capture state through typed executor/network options.
Route proxy network policy checks through the existing source policy evaluator so
session metadata, deny messages, and URL converts use the same path as LLB
sources. Keep proxy-specific request rewriting in the proxy provider.