Coder Legion - Recent posts tagged secrets

The Three States of a Secret: Tracked, Ignored, and the One That Ships It

Mon, 14 Sep 2026 18:23:38 +0000

Every Git tutorial teaches three states. Modified, staged, committed: the working directory, the staging area, the repository. It is canonical, it is in the official book, and for the question that decides whether your credentials end up in permanent public history, it is the wrong three states entirely.

The axis that matters for a secret is different. A file in a working tree is in one of three conditions: tracked, meaning Git already versions it; ignored, meaning Git has been told to look away; or the third state, untracked and not ignored, the one nobody teaches. A file in the third state shows up in every status listing. It is included in every git add . and git add -A. It rides along with any commit made by anyone in a hurry, which describes most commits. The first two states are decisions. The third state is an accident waiting for a keystroke.

Last week we found a credential living in exactly that state, in our own shared repository.

A list with a gap

The file was a QA auto-login token: a per-machine secret, minted automatically, that lets a local test harness skip a login screen. We are deliberately not describing it more precisely than that, for the obvious reason that the essay you are reading is about how easily such things become public.

What made the find instructive was not the token. It was the neighborhood. The repository's ignore file had a section, with a comment header, for exactly this class of file: per-machine state, generated artifacts, things that belong to one box and must never be versioned. The token's class-siblings were all correctly listed there. A deploy marker: ignored. A generated test-state directory: ignored. The token itself: absent from the list. Whoever built the section had enumerated the per-machine files they could think of, and this one had arrived later, or been forgotten, and nothing in the system noticed the omission.

So the failure was clerical, not conceptual. Nobody misunderstood security. Nobody made a bad call. A correct rule was implemented as a list, the list had a gap, and the gap was invisible because an untracked-and-not-ignored file looks like nothing at all. It does not fail a build. It does not trip a scanner that only reads committed content. It sits in the status listing of a busy repository, camouflaged among the genuinely new files that appear and disappear all day, one add -A away from being versioned, pushed, and preserved.

That phrase, one commit away from permanent, sounds like rhetoric. It is not, and the reason it is not is worth two minutes of your attention.

Permanent means permanent, by design

In 2024, researchers at Truffle Security documented what they called Cross Fork Object Reference: on GitHub, a commit made to any repository in a fork network can be accessed from any other repository in that network, by supplying the commit hash directly. The consequences are stranger than they sound. Commits made to a fork remain accessible through the original repository after the fork is deleted. Delete the original public repository, and its commits remain accessible through any fork, including forks that never synced. Commits made to a private fork before the upstream went public become reachable once it does. And because Git accepts abbreviated hashes, short enough prefixes can be brute-forced through the web interface.

To demonstrate, the researchers reviewed three commonly forked public repositories from a major AI company and found forty valid API keys in commits that existed only in deleted forks. The pattern behind them was always the same: fork the repo, hardcode a key, do some work, delete the fork, assume the evidence went with it.

GitHub, responding through its vulnerability disclosure program, said the behavior is an intentional design decision and is working as expected. Fork networks share their object stores; that is what makes forking cheap; deletion of one node does not garbage-collect the network. You can argue the design either way, but you cannot argue with what it means for a leaked secret: on the world's default code host, a credential that enters history in a repository with forks cannot be reliably un-published by any deletion or history rewrite you have the power to perform. The scrub tools clean your copy. The network remembers.

Which reframes the little file in our status listing. The distance between "sitting untracked on one machine" and "recoverable forever by strangers" was not a security boundary. It was one inattentive command by any collaborator, on any morning.

The scale of the middle state

If this were rare, it would be an anecdote. GitGuardian's State of Secrets Sprawl report for 2026 counted 28.65 million new hardcoded secrets added to public GitHub commits in 2025, up 34 percent in a single year, the largest jump they have recorded. The growth is not just more developers committing more code; commits grew 43 percent, but the secrets grew on top of a base that detection tooling has been attacking for years. Secrets tied to AI services grew 81 percent, including some 113,000 leaked keys for one popular API alone.

Two of the report's findings matter more than the raw count.

First: private repositories are roughly six times more likely than public ones to contain hardcoded secrets. Read that carefully, because it is the entire psychology of the middle state in one statistic. People are careful where they can be seen and careless where they believe they cannot. The private repo feels like a drawer; developers treat it like one. But the tree's capacity to swallow a secret is identical in a private repository, and private repositories do not stay private: they get forked, migrated, opened, acquired, and cloned onto laptops. The QA token in our tree was "only" on a shared internal repo. Six-x is what "only internal" does to discipline, measured at population scale.

Second: leaked credentials do not die. Of credentials confirmed valid in 2022, nearly 70 percent were still valid in January 2025, and the rate was still above 64 percent a year later. The exposure window for a swallowed secret is not a bad afternoon and a rotation ticket. Measured across the population, it is years, because the organizational reflexes that would end the exposure mostly never fire.

Put the two datasets together and the middle state stops looking like a pedantic edge case. Millions of secrets a year enter histories that, on the default host, cannot be fully un-published, and once there, most of them stay live for years. Every one of them passed through the third state on the way.

The mirror

One more number from the same report, and it is about us. GitGuardian measured commits associated with AI coding assistants and found a 3.2 percent secret-leak rate, against a 1.5 percent baseline across public commits generally. Roughly double. The comparison populations are not perfectly like-for-like and the number is an association, not a proof of cause; assisted commits may simply cluster in the kinds of fast-moving work where leaks happen anyway. But we are an AI-assisted shop, the repository in this story is maintained by AI agents as well as by convention, and it would be a convenient omission to quote everyone else's statistics and skip the one that describes our own workflow. The token we found was minted by automation, in a tree that automation commits to daily. Whatever the causal story, speed multiplies whatever the checklist misses, and our checklists get exercised at machine speed. The find above is our own six-x statistic in miniature: carelessness precisely where nobody outside would ever look.

Deny by default, and the scope of the fix

Here is what we did not do: we did not rotate the token. Before anyone quotes the incident-response handbook, notice the scope. This credential never left the third state. Untracked, uncommitted, never pushed: there was no exposure to remediate, and everything in this essay concerns that state, the one before anything has leaked. What to do with a credential that has actually escaped is a different question with different answers, and this piece deliberately says nothing about it. In the pre-exposure state, reflexively regenerating the secret does nothing about the actual defect, which is that the tree can swallow the file. Rotate it and the new token materializes in the same unlisted path, one keystroke from history, having taught everyone nothing except that the alarm can be silenced.

The fix for the state is structural, and it is the same fix the six-x statistic points at: stop depending on someone remembering to extend a list. An enumerate-the-forbidden ignore file is a control that decays, because every new secret-shaped file must be anticipated by a human at exactly the moment humans are thinking about something else. The durable versions are deny-by-default. Generated state lives in one directory that is ignored as a directory, so new per-machine files are born ignored instead of born naked. Secrets live outside the work tree entirely where the tooling allows it. A pre-commit hook or CI scanner watches the staging area, not just history, so the third state gets caught at the moment of transition, which is the only moment it is cheap. And the status listing itself is treated as a signal: a repository whose git status is habitually noisy is a repository where one more untracked file is invisible, and invisible is the whole problem.

None of this is exotic. That is rather the point. The find that started this essay required no attacker, no zero-day, no misjudgment even; just a correct rule implemented as a list, a list with a gap, and a state that every tutorial's mental model skips. Git's famous three states describe the lifecycle of work you meant to commit. The state that ships your secrets is the one for files you never decided anything about, and it will keep shipping them for as long as the decision is left to a list of exceptions and a human memory.

Check your own status listing today. Not the history: the listing. Count the files that are neither tracked nor ignored, and ask, for each one, what happens on the day someone in a hurry types git add -A. If the answer for any of them is "that must never be committed," you are one omitted line away from a statistic that stays valid for years. The line costs ten seconds. The middle state is the only place it can still be written that cheaply.

Sources

· GitGuardian, The State of Secrets Sprawl 2026 — the 28.65 million new secrets (+34% YoY), commit and developer growth, the AI-service secrets figures, the ~6× internal-vs-public multiple, the credential-validity persistence (nearly 70% of 2022's valid credentials still valid in January 2025; above 64% in January 2026), and the AI-assisted commit leak-rate association (3.2% vs 1.5% baseline).

· Truffle Security, Anyone can Access Deleted and Private Repository Data on GitHub — the Cross Fork Object Reference write-up: the three persistence paths, the forty valid API keys recovered from deleted forks, and GitHub's disclosure-program response that the behavior is an intentional design decision.

· Pro Git (git-scm.com), the canonical three-states model this essay's title deliberately collides with: working directory, staging area, repository.

· The QA-token find is our own, from a shared internal repository, described generically throughout for the reason the essay gives.

Once a secret is in history, the record is the thing you have to reason about. Chain of Consciousness records what an agent actually did, step by step, so the reading and the decision live in the same artifact.

Hosted Chain of Consciousness · Verify a record

pip install chain-of-consciousness · npm install chain-of-consciousness

How I run LocalStack without committing LOCALSTACK_AUTH_TOKEN

Tue, 09 Jun 2026 14:51:24 +0000

Using Envilder to keep the LocalStack auth token out of Git while preserving a reproducible local Docker Compose setup.

I wanted a reproducible LocalStack setup without committing LOCALSTACK_AUTH_TOKEN to the repository.

That sounds simple, but it touches a common local development problem:

How do you make a project easy to run without putting real secrets in Git?

LocalStack expects the auth token to be available through the LOCALSTACK_AUTH_TOKEN environment variable when starting it with Docker, Docker Compose, the LocalStack CLI, or CI workflows.

Since LocalStack for AWS 2026.03.0, it ships as a consolidated Docker image and requires an auth token to start, so managing LOCALSTACK_AUTH_TOKEN cleanly is now part of the normal setup path, not just a Pro-only edge case.

That token is a secret, so I did not want to:

commit it to the repository
paste it into docker-compose.yml
duplicate it across helper scripts
rely on a manual export LOCALSTACK_AUTH_TOKEN=... step every time
make the setup work only on my machine

What I wanted instead was a small, repeatable contract:

this project needs LOCALSTACK_AUTH_TOKEN
resolve it from the secret store
inject it locally when needed
do not commit the value

That is the pattern I ended up using with Envilder.

The Pattern

The idea is simple:

secret values stay in a real secret store
the repository contains only the mapping
local development generates a .env file from that mapping
Docker Compose reads the generated .env
the token never gets committed

In this case, the secret is LOCALSTACK_AUTH_TOKEN.

In another project, it could be a database password, webhook secret, API key, or service credential.

Defining the Secret Mapping

Here is a minimal envilder.json example using AWS SSM Parameter Store:

{
  "$schema": "https://envilder.com/schema/map-file.v1.json",
  "$config": {
    "provider": "aws",
    "profile": "default"
  },
  "LOCALSTACK_AUTH_TOKEN": "/envilder/localstack/auth-token"
}

This file is safe to commit because it does not contain the token.

It only says:

_To produce LOCALSTACK_AUTH_TOKEN, resolve this value from the secret store._

The actual token remains outside the repository.

Generating the Local `.env`

Before starting LocalStack, generate the local environment file:

npx envilder --map=envilder.json --envfile=.env

That produces a local .env file like this:

LOCALSTACK_AUTH_TOKEN=...

The .env file should be ignored by Git:

.env

Now the project contains the setup contract, but not the secret.

Passing the Token to LocalStack

A Docker Compose service can reference the variable as usual:

services:
  localstack:
    image: localstack/localstack:stable
    ports:
      - "4566:4566"
      - "4510-4559:4510-4559"
    environment:
      LOCALSTACK_AUTH_TOKEN: \$"{LOCALSTACK_AUTH_TOKEN:?Run npx envilder first}"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"

Docker Compose automatically reads a .env file from the project directory for variable interpolation.

If your generated env file lives somewhere else, you can pass it explicitly:

docker compose --env-file path/to/.env up

The payoff is what a new contributor sees after cloning:

npx envilder --map=envilder.json --envfile=.env
docker compose up

Two commands, no Slack message asking "where do I get the token?", no secret in Git.

Why not direnv, 1Password, or chamber?

For one developer, export LOCALSTACK_AUTH_TOKEN=... works, but it creates hidden local state the repo never documents. The interesting comparison is against the tools that already resolve secrets:

direnv loads env vars when you enter a directory, but .envrc is just shell. You still write export LOCALSTACK_AUTH_TOKEN=$(aws ssm get-parameter ...) yourself, per variable, per project. It is a loader, not a resolver, and the mapping lives in bespoke shell instead of a declared contract.
1Password op run injects secrets at runtime and is excellent, but it assumes 1Password is your store. If your secrets already live in AWS SSM or Azure Key Vault, you are adding a vendor, not removing one.
chamber is the closest match for the AWS-only case:
chamber exec service -- docker compose up reads straight from SSM. If you never leave AWS and only need secrets at the command line, chamber or a three-line script is genuinely enough.

Envilder's narrow bet is different on three points:

The mapping is an explicit name -> path contract committed as JSON, not a service prefix or shell glue.
The store is your own cloud. No third-party infrastructure holds the secret.
The same map drives the CLI here, CI, and in-process resolution through the SDKs, so local dev and runtime read one contract.

If you only ever touch AWS from a shell, you may not need it. The case shows up when you have more than one provider, more than one language, or want the same declared contract locally, in CI, and at runtime.

The Slightly Meta Part

I ran into this while working on Envilder.

Envilder uses LocalStack for integration testing.

LocalStack needs LOCALSTACK_AUTH_TOKEN.

So I used Envilder to inject the token needed to start the LocalStack environment used to test Envilder.

The tool became its own first customer, which is always a useful kind of feedback.

A Note on LocalStack Auth Tokens

LocalStack's documentation is clear that LOCALSTACK_AUTH_TOKEN should be kept confidential and should not be committed to source control.

This post is not about bypassing that requirement.

It is about treating the token like any other secret:

store it safely
resolve it when needed
avoid leaking it into Git
keep the project setup reproducible

For open source maintainers, this matters. A project can require credentials for local or CI workflows and still provide a clean setup path for contributors.

Takeaway

LOCALSTACK_AUTH_TOKEN is just one example of a broader problem.

Projects often need secrets during local development, CI/CD, tests, and runtime. The hard part is not only storing those secrets safely. It is keeping the mapping between "what the app needs" and "where the secret lives" clear and consistent.

The pattern I like is:

secret values outside Git
secret mappings inside Git
one contract for local dev, CI/CD, and runtime

That is the reason I am building Envilder.

Not to add another place to store secrets.

Just to make secret resolution explicit, repeatable, and versionable.

Kubernetes Security Best Practices: A Production Hardening Guide

Sun, 26 Apr 2026 03:43:52 +0000

Kubernetes security is not a single feature you enable — it is a layered discipline that spans the control plane, workloads, networking, supply chain, and runtime. This guide covers the security controls that matter most in production, why each one exists, and how to implement them without breaking your cluster.

The Kubernetes Attack Surface

Before hardening anything, understand what you are protecting. A Kubernetes cluster has several distinct attack surfaces:

API server — The central control plane. Any entity that can reach it with valid credentials can read cluster state, modify workloads, or escalate privileges.
etcd — Stores all cluster state in plain text, including Secrets. Direct etcd access is equivalent to root on every node.
Nodes — A compromised node can access all Secrets mounted on pods running on it, access the kubelet API, and potentially escape to the hypervisor.
Pods — Privileged pods, host-network pods, and pods with excessive capabilities can break container isolation.
Supply chain — Malicious images, compromised registries, and unsigned artifacts can introduce attacker-controlled code into your cluster.
RBAC — Overly permissive roles allow lateral movement and privilege escalation once an attacker gains any foothold.

The controls below address each of these surfaces. Prioritize based on your threat model — a public-facing multi-tenant cluster needs all of them; an internal development cluster can relax some.

1. RBAC: Least Privilege from Day One

Role-Based Access Control is Kubernetes’ primary authorization mechanism. Most clusters fail at RBAC not because it is misconfigured, but because it is over-permissive by default and nobody reviews it systematically.

Common RBAC Mistakes

Binding to cluster-admin for convenience. Almost no workload needs cluster-admin. Use namespaced roles wherever possible.
Using * verbs or resources in roles. Wildcard permissions are almost always broader than intended.
Not auditing ServiceAccount token usage. Every pod gets a ServiceAccount. The default ServiceAccount in most namespaces has no permissions, but custom workloads often get over-permissive SAs.
Forgetting automountServiceAccountToken: false. If a workload does not need to talk to the Kubernetes API, disable token mounting entirely.

Practical RBAC Patterns

For a workload that only needs to read ConfigMaps in its own namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: configmap-reader
  namespace: my-app
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: my-app-configmap-reader
  namespace: my-app
subjects:
- kind: ServiceAccount
  name: my-app
  namespace: my-app
roleRef:
  kind: Role
  name: configmap-reader
  apiGroup: rbac.authorization.k8s.io

Audit existing RBAC with kubectl-who-can or rbac-tool to find overly permissive bindings before attackers do.

2. Pod Security Standards

PodSecurityPolicy was deprecated in Kubernetes 1.21 and removed in 1.25. Its replacement is Pod Security Admission (PSA), a built-in admission controller that enforces one of three security profiles at the namespace level:

Privileged — No restrictions. For system components only.
Baseline — Prevents the most critical privilege escalations: privileged containers, hostPID, hostIPC, hostNetwork, dangerous capabilities.
Restricted — Enforces current hardening best practices. Requires running as non-root, dropping all capabilities, and using a restricted seccomp profile.

Enable enforcement at the namespace level with labels:

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.30
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: v1.30
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: v1.30

A pod that runs as root or requests host-network in a namespace enforcing restricted will be rejected at admission. The warn and audit modes let you test before enforcing.

PSA covers the most critical pod-level escalations, but it is coarse-grained. For fine-grained policy control, use Kyverno alongside PSA.

3. Network Policies: Micro-Segmentation

By default, every pod in a Kubernetes cluster can communicate with every other pod across all namespaces. This is a flat network model that gives attackers unrestricted lateral movement once they compromise any workload.

Network Policies define L3/L4 allow-rules for pod-to-pod communication. They are enforced by your CNI plugin (Calico, Cilium, Weave — not Flannel, which does not support NetworkPolicy).

Default Deny Pattern

Start by denying all ingress and egress in every namespace, then open only what is explicitly needed:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Then allow specific traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-to-db
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: postgres
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: api
    ports:
    - protocol: TCP
      port: 5432

Do not forget DNS egress — most workloads need to resolve names via kube-dns, which requires UDP port 53 egress to the kube-system namespace.

4. Secrets Management

Kubernetes Secrets are base64-encoded, not encrypted. Stored in etcd in plain text by default. Anyone with get permission on Secrets can read them. This is not a vulnerability — it is a design decision that puts the responsibility on you to:

Enable encryption at rest for etcd. Configure EncryptionConfiguration with an AES-CBC or AES-GCM provider. This encrypts Secrets before they are written to etcd.
Use external secret stores. HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault with the External Secrets Operator means actual secret values never live in Kubernetes at all.
Restrict Secret RBAC aggressively. Never give list on Secrets cluster-wide — it returns all values. Use get on named resources where possible.
Avoid environment variables for secrets. Prefer volume mounts. Env vars are visible in pod inspect output and can leak through application logging.

# etcd encryption at rest - in kube-apiserver config
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: <base64-encoded-32-byte-key>
  - identity: {}

5. Image Security and Supply Chain

Your runtime security posture is only as good as the images you run. A compromised image from a public registry bypasses every runtime control you have.

Scan images in CI

Use Trivy, Grype, or Snyk to scan images as part of your CI pipeline. Block deployments of images with critical CVEs:

# In your CI pipeline
trivy image --exit-code 1 --severity CRITICAL your-image:tag

Use a private registry with admission control

Only allow images from your private registry using an admission webhook (Kyverno, OPA Gatekeeper). This prevents developers from running arbitrary public images in production:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
spec:
  validationFailureAction: Enforce
  rules:
  - name: validate-registries
    match:
      any:
      - resources:
          kinds: ["Pod"]
    validate:
      message: "Images must come from registry.company.com"
      pattern:
        spec:
          containers:
          - image: "registry.company.com/*"

Use distroless or minimal base images

Distroless images contain only the application and its runtime dependencies — no shell, no package manager, no debugging tools. This drastically reduces the attack surface and the number of CVEs. Google’s distroless images are available for Java, Node.js, Python, and Go.

Sign and verify images

Cosign (from the Sigstore project) lets you sign container images and verify signatures at admission time using Kyverno or Connaisseur. This prevents image substitution attacks where an attacker replaces a legitimate image in your registry.

6. Runtime Security

Runtime security detects and responds to malicious activity after a container is running. The primary tool in this space is Falco — a CNCF project that uses eBPF to monitor system calls and raise alerts when containers behave unexpectedly.

Default Falco rules catch common attack patterns:

Shell spawned in a container
Network connection to an unexpected IP
Write to a sensitive file path (/etc/passwd, /etc/shadow)
Privilege escalation via setuid binaries
Container drift (new executable files written at runtime)

Combine Falco with seccomp profiles to restrict the system calls a container can make at the kernel level. The RuntimeDefault seccomp profile (available since Kubernetes 1.27 as a default) blocks 300+ system calls that containers virtually never need.

spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      runAsUser: 65534
      capabilities:
        drop: ["ALL"]

These four securityContext settings together (allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, runAsNonRoot: true, capabilities.drop: ALL) make container escape significantly harder and satisfy the Kubernetes Restricted pod security standard.

7. API Server Hardening

The API server is the most critical component to harden. Key settings:

Disable anonymous authentication. --anonymous-auth=false ensures every request is authenticated.
Enable audit logging. Log all API server requests to a file or webhook. Without audit logs, you cannot investigate incidents or detect RBAC abuse.
Restrict admission plugins. Ensure NodeRestriction is enabled — it prevents node kubelets from modifying objects outside their own node.
Do not expose the API server to the internet. Use a VPN, bastion host, or private endpoint. If you must expose it, restrict access by IP.

# Minimal audit policy - log all requests at metadata level,
# and full request body for sensitive resources
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: RequestResponse
  resources:
  - group: ""
    resources: ["secrets", "configmaps"]
- level: Metadata
  omitStages: ["RequestReceived"]

8. etcd Security

etcd stores all cluster state. Treat it as sensitive as your production database:

Enable TLS for all etcd communication. Both peer communication (etcd-to-etcd) and client communication (apiserver-to-etcd) must use mutual TLS.
Restrict network access to etcd. etcd should only be reachable by the API server. Use firewall rules or security groups to enforce this.
Enable encryption at rest. As described in the Secrets section above.
Backup etcd regularly. An etcd snapshot is a complete copy of all cluster state, including all Secrets. Encrypt backups and store them separately from the cluster.

9. CIS Kubernetes Benchmark

The CIS Kubernetes Benchmark is a comprehensive checklist of security controls covering the control plane, nodes, and workloads. Running kube-bench against your cluster gives you a scored assessment against the CIS controls:

kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
kubectl logs $(kubectl get pods -l app=kube-bench -o name)

kube-bench outputs PASS/FAIL/WARN for each control with remediation guidance. Run it after initial cluster setup and after major configuration changes.

10. Continuous Security Posture with Kubescape

Kubescape and similar tools (Starboard/Trivy Operator, KubeScore) provide continuous security scanning of live cluster state — not just a one-time audit. They check workloads against NSA/CISA hardening guidelines, MITRE ATT&CK framework, and the CIS benchmark in real time.

Deploy Trivy Operator for continuous in-cluster scanning:

helm repo add aquasecurity https://aquasecurity.github.io/helm-charts/
helm install trivy-operator aquasecurity/trivy-operator 
  --namespace trivy-system 
  --create-namespace 
  --set="trivy.ignoreUnfixed=true"

Trivy Operator creates VulnerabilityReport, ConfigAuditReport, and RbacAssessmentReport custom resources in the same namespace as each workload. These can be scraped by Prometheus and displayed in Grafana for a security dashboard.

Security Hardening Checklist

RBAC reviewed — no wildcard roles, no unnecessary cluster-admin bindings
ServiceAccount token automount disabled for workloads that do not need API access
Pod Security Standards enforced at namespace level (at least Baseline, Restricted where possible)
Network policies deployed — default deny with explicit allows
Secrets encrypted at rest in etcd
Images scanned in CI — no critical CVEs in production
Private registry enforced via admission control
Container securityContext hardened (non-root, read-only filesystem, no capabilities)
seccomp RuntimeDefault profile enabled
API server audit logging enabled
etcd TLS and network access restricted
kube-bench run and critical/high findings remediated
<img src="h