Coder Legion - Recent posts tagged lazysre

Lazy SRE's guide to secure systems, part 2: the actions you didn't pin

Sun, 07 Jun 2026 21:55:44 +0000

Originally published at harshit.cloud on 2026-04-12.

Last March, someone with write access to the trivy-action repo rewrote 76 of its 77 version tags in place. The tags still resolved to aquasecurity/trivy-action — they just resolved to different commits than they did the week before. Every pipeline that ran aquasecurity/trivy-action@0.20.0 (and every other tagged version) ran the attacker's commit instead. Secrets exfiltrated. The stolen credentials chained into PyPI and took down LiteLLM. Nobody noticed for hours, because the workflow file diff was still clean.

This is part 2. Part 1 covered npm: the dependencies you didn't read. Part 2 is the same problem one level up: the workflows you didn't pin. Part 3 is the unsexy list — Tailscale, PrivateLink, IAP, the PAT you forgot.

The thesis from Part 1 stands. The best security work for a small team is the work Future You at 3am will actually execute. The configuration that makes the wrong thing impossible beats the runbook that only discourages it. With GitHub Actions, "the wrong thing" has gotten very specific over the last twelve months, and the configs to block each variety have gotten correspondingly precise.

pinning is necessary but not sufficient

The first thing the trivy-action incident proves: hash-pinning to @0.20.0 is not pinning. It's a name lookup. The owner of the repo is allowed to rewrite that name. The pin you actually wanted was:

- uses: aquasecurity/trivy-action@9b9a3f5c8a5c7e1b6e4d2f1c9b8a7e6d5c4b3a2f # v0.20.0

Full forty-character SHA. Immutable. The version comment is so the next reader knows what they're looking at; the SHA is so the workflow runs the code you reviewed.

Two GitHub features shipped in 2025 that change the math:

SHA pinning enforcement (Aug 2025). An org-level policy that fails workflow runs using unpinned actions, instead of warning about them. Settings → Actions → General → Action pinning. Turn it on. There is no "we'll get to it" version of this toggle.
Immutable Releases (Oct 2025, GA). Action authors opt in to making release tags non-rewritable after publication. If you publish actions, turn this on for downstream consumers. If you consume actions, prefer ones that have.

The lazy stance: enforcement at the org level. The workflow that doesn't have a forty-character SHA fails the run. The PR can't merge. The work of remembering to pin moves from every engineer's head to one setting.

What this doesn't catch: an attacker who compromises the maintainer account and ships a new tag at a new SHA. The SHA is real. Pinning by SHA doesn't help, because the workflow author will rev to the new version when they read the maintainer's release notes. Which is the next config.

cooldown is the same trick that worked for npm

Part 1's load-bearing config was SAFE_CHAIN_MINIMUM_PACKAGE_AGE_HOURS=48. The principle: most published malware is detected and pulled within hours. If you can wait, the wait does the work for you.

The action ecosystem has the same property, with a longer window. yossarian's analysis puts the cooldown that catches most supply-chain attacks at 7-14 days. So:

pinact --min-age 7 .github/workflows/*.yml

Refuses to write a pin younger than seven days. Add to pre-commit, your CI lint, or whatever your dependabot equivalent runs before opening the bump PR.

For Renovate users, the equivalent lives in the action manager:

{
  "packageRules": [
    { "matchManagers": ["github-actions"], "minimumReleaseAge": "7 days" }
  ]
}

That's it. Same trick, different ecosystem.

Fig. 2 — the wait is doing the work. Seven days closes most of the door; fourteen closes most of the rest.

The empirical question is whether seven days is enough. The trivy-action force-push was detected at about nine — seven would have caught most consumers, not all of them. The cost of fourteen is "your action versions lag upstream by two weeks." If your action surface is small (most teams are running actions/checkout, actions/setup-node, one cloud-login action, maybe a deploy action), set fourteen and forget.

pull_request_target is the new postinstall

Part 1 named postinstall as the single trigger that does the most damage and the single switch (ignore-scripts=true) that closes the most doors. Actions has the same shape and the same fix.

pull_request_target runs in the context of the base repository, with access to repository secrets, but is triggered by a PR from a fork. The legitimate use case is small: comment on PRs, label them, run lightweight metadata jobs. The illegitimate use case is enormous: check out the fork's code and execute it. The attack writes itself. Open a fork, modify a script the trusted workflow runs, watch the runner exfiltrate every secret in the env.

Astral, who maintain uv and ruff, wrote it cleanly: "these triggers are almost impossible to use securely." GitHub partially mitigated this in November 2025 by forcing pull_request_target to always use the default branch's version of the workflow, so an attacker can't push a vulnerable workflow on a feature branch and trigger it. But the foot-cannon still ships loaded if your default-branch workflow checks out PR-head code.

Fig. 3 — same workflow, different trigger, opposite blast radius.

The lazy stance:

Don't use pull_request_target unless you've named the specific reason and one other person has signed off.
If you do, never actions/checkout the PR head from inside it. Check out the base SHA, do the metadata thing, exit.
For everything else, use pull_request. It runs without secrets. Attacker-controlled code stays attacker-jailed.

Same shape as ignore-scripts=true. The setting that closes the class.

the safe defaults that go in every workflow

The four-line workflow header that does the most work per character:

permissions:
  contents: read

defaults:
  run:
    shell: bash -euo pipefail {0}

contents: read overrides the org-level default. If a step needs to push a tag or open a PR, that job opts back up to contents: write explicitly. The default is the safe one.

At the checkout step:

- uses: actions/checkout@<sha> # v4.2.0
  with:
    persist-credentials: false

The default behavior of actions/checkout is to leave a credential sitting in .git/config for the rest of the workflow. Later steps have shipped this credential into uploaded artifacts more than once. Opt out unless a later step in the same job needs to push.

Three secret-access rules with the same flavor:

Step-scoped env:, never workflow-scoped, for any secret.
Never ${{ toJson(secrets) }}. Exposes every secret in the project to the runner. There is no use case.
Never secrets: inherit on reusable workflows. Pass each secret by name. The reusable workflow gets exactly what it asked for.

The trivy-action exfiltration worked partly because secrets were workflow-scoped. The malicious step inherited every credential in the env, not just the one the legitimate scan needed. Step-scoping wouldn't have prevented the credential theft — but it would have bounded the blast radius to one secret instead of all of them.

OIDC, the promise from part 1

Part 1 ended on "the next-tier defenses are real, Part 3 names them." OIDC is the part of that conversation that lives here.

The trade: instead of storing an AWS_ACCESS_KEY_ID in repo secrets and praying nobody exfiltrates it, you configure AWS to trust GitHub's OIDC issuer for a specific repo, branch, and workflow. GitHub mints a short-lived (five-minute) OIDC identity token for the workflow run. The workflow trades that for STS credentials whose lifetime you set (default one hour). Nothing long-lived ever sits in the env.

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: aws-actions/configure-aws-credentials@<sha> # v4.0.2
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-deploy
          aws-region: us-east-1
      - run: aws s3 sync ./dist s3://my-bucket

The role's trust policy restricts the OIDC subject to your exact repo and (ideally) branch. An attacker who compromises a fork PR can't assume the role, because they don't match the trust condition. The OIDC JWT itself lasts five minutes and the STS credential is scoped to whatever you configure (default one hour). Even an exfiltrated credential gets the attacker a bounded window of scoped access, not a permanent IAM user.

For Google Cloud, the equivalent is Workload Identity Federation. For HashiCorp Vault, the JWT auth backend. Same shape across providers.

The labor here is genuinely one-time. Configure the trust relationship once per repo, delete the long-lived key, forget about rotation forever. The rotation runbook you're not maintaining is one of the better quiet wins in this post.

zizmor is the local proxy for workflows

Part 1's safe-chain sat in front of every package install and refused malware before bytes hit disk. The action ecosystem's equivalent is zizmor — a workflow linter that reads your YAML and catches the patterns this post is about, before they merge.

brew install zizmor
zizmor .github/workflows/

It catches unpinned actions, pull_request_target with PR-head checkouts, template-injection patterns where attacker-controlled input lands in a run: string, jobs with excessive permissions. Add it to pre-commit:

# .pre-commit-config.yaml
- repo: https://github.com/woodruffw/zizmor-pre-commit
  rev: v1.x  # pin the rev, obviously
  hooks:
    - id: zizmor

The principle is identical to safe-chain. Move the security check from "after the incident, in the postmortem" to "before the PR can merge, on the dev machine." The CI run is the second line of defense. The pre-commit is the first.

the receipts

The above stack is approximately one afternoon: org-level SHA pinning enforcement, pinact --min-age 7 or Renovate minimumReleaseAge: 7 days, the four-line workflow header, persist-credentials: false, no pull_request_target with PR-head checkouts, OIDC for every cloud credential, zizmor in pre-commit.

It will not catch a maintainer-account compromise that ships clean-looking code which activates weeks later. It will not catch a determined attacker who studies your build and writes a payload that survives every linter and looks innocent at PR review. Nothing in this post will. Part 3 will name the controls that buy partial mitigation against that class: sigstore, npm provenance, reproducible builds, attested deployments. And the ones that exist to make the postmortem readable, not to prevent the incident.

For a small team, the delta from this post is moving from "we're one tag-rewrite away from a credential theft cascade" to "an attacker would need a credentialed insider, or a fifteen-minute window of luck against a scoped IAM role." That's the only delta that matters at this scale.

If you do one thing this week, turn on SHA pinning enforcement at the org level. Everything else gates off that.

Lazy SRE's guide to secure systems, part 1: the dependencies you didn't read

Sun, 07 Jun 2026 21:55:28 +0000

Originally published at harshit.cloud on 2026-04-05.

A few months ago a friend's CI pipeline tried to install a package none of us had heard of. The build failed. The error wasn't a missing dep. The error was a local proxy saying this is malware, I'm not letting it touch disk. The package was a transitive dependency, six levels deep, that had been published to npm 38 minutes earlier. Nobody on the team had asked for it.

I do platform work at a startup. The job is: keep production up, keep the bill down, keep the kind of person who reads HN comments from getting a free shell on the cluster. The thesis for the series is short. The best security work for a small team is the work Future You at 3am will actually execute. The lazy answer is also, almost always, the right one: the configuration that makes the wrong thing impossible, rather than only discouraged. This is part 1. Part 2 is GitHub (tj-actions, OIDC, fork PRs). Part 3 is the unsexy list (IAP, Tailscale, PrivateLink, Okta, default helm creds, the PAT you forgot).

the picture

You opened your editor, ran npm install, and onboarded somewhere between 800 and 2,000 packages maintained by people you have never audited. Your team reviewed five direct dependencies. The other 1,995 came in for the ride.

The job is not to read all 1,995. The job is to make sure that when one of them is the problem, the blast radius is small and the alarm goes off.

chainjacking

Chainjacking is the umbrella term for "someone got control of a package you depend on and pushed a bad version." The attacker doesn't break npm. They get the credentials of the human who publishes the package, ship a patch version, and semver puts it on your machine the next time anyone runs npm install. event-stream (2018), ua-parser-js / coa / rc (2021), lottie-player (2024), and the shai-hulud worm (Sept 2025, with a 2.0 wave in Nov 2025) that self-replicated by stealing tokens from compromised maintainer machines via TruffleHog-style secret scans and then republishing every other package that maintainer owned. The economics still work for the attacker. It is going to keep happening.

Fig. 2 — twenty-four hours from maintainer phish to credential resale. Nobody noticed the version bump.

What matters in that timeline is that the human steps are slow and the automated steps are fast. The window between "malicious version published" and "your CI runs npm install" is whatever your dependabot cron is. If you auto-merge minor and patch bumps, that window is ninety seconds.

dependency confusion

You have a private package called internal-utils. Your CI is configured with both your private registry and the public npm registry. Somebody publishes internal-utils to public npm at a higher version. CI installs the public one. Birsan did this against Apple, Microsoft, Tesla, and PayPal in a weekend in 2021 for bug bounties.

Fix, in .npmrc:

@yourorg:registry=https://npm.yourorg.internal
registry=https://registry.npmjs.org/

Scope everything internal. Register your scope on public npm as a parked placeholder. It costs nothing.

postinstall

Most install-time npm supply-chain incidents I have read the postmortem on shipped their malicious code in a postinstall script — not in runtime code. (Some recent ones, like the chalk/debug compromise of Sept 2025, activate at runtime in the browser or on first import; the switch below doesn't help against those. It does help against the install-time class, which is still the majority.) The install hook runs before your tests, before your linter, as part of the install. Default is enabled. The one-line change with the highest blast-radius reduction:

# .npmrc
ignore-scripts=true

You'll need to allowlist two to five packages that genuinely need it (typically bcrypt-shaped things). That number is small. The alternative is letting every package run code on install. Pick.

the install that buys the most

Aikido's safe-chain is an open-source local proxy that sits in front of npm, npx, yarn, pnpm, pnpx, bun, pip, uv, poetry, pipx. Every package download is intercepted and checked against Aikido Intel, an open malware feed. Malware is blocked before bytes hit disk. Which is before postinstall runs. Free. No account.

Fig. 3 — safe-chain in one picture. A local proxy in front of every package manager, checked against an open threat-intel feed.

On a dev machine:

curl -fsSL https://github.com/AikidoSec/safe-chain/releases/latest/download/install-safe-chain.sh | sh
# restart your shell
npm safe-chain-verify
# expected: OK: Safe-chain works!

In CI:

- name: Install safe-chain
  run: |
    curl -fsSL https://github.com/AikidoSec/safe-chain/releases/latest/download/install-safe-chain.sh \
      | sh -s -- --install-dir /usr/local/.safe-chain
    echo "$HOME/.safe-chain/bin" >> "$GITHUB_PATH"
- run: npm ci

And, the part that quietly does the most work — refuse to install anything younger than 48 hours, because that's the window in which most npm malware is caught and removed:

export SAFE_CHAIN_MINIMUM_PACKAGE_AGE_HOURS=48

the receipts

The above stack is approximately one afternoon of work: npm ci from a committed lockfile, ignore-scripts=true with a tiny allowlist, scoped private packages with locked registry resolution, safe-chain in front of every install, minimum package age of 48 hours. It will catch most known-bad packages, kill dependency-confusion at the registry level, and reduce postinstall blast radius to zero for the long tail. It will not catch a maintainer-account compromise that ships clean-looking malware that only activates in production weeks later. Nothing in this post will. The next-tier defenses (sigstore signing, npm provenance, reproducible builds) are real, and Part 3 will name them.

For a startup the delta from this post is moving from "one of the next ten incidents has a non-trivial chance of being yours" to "you would have to be very unlucky." That's the only delta that matters.

If you do one thing this week, go register your npm scope.

diagrams: what i tried

Three diagrams, three different tools, one brief: "explain a supply-chain attack to a tired SRE in one image." Prompts kept short. Results below.

1 — the napkin contrast (coleam00 excalidraw-diagram skill)

Brief: "two-panel hand-drawn napkin. Left panel 'what i installed': five direct deps off a 'your app' box. Right panel 'what npm install pulled in': same five direct deps, transitive sprawl under each, one of them is a red evil-helper@1.2.3, callout reads 'shipping crypto miners.'"

Result: Fig. 1. Built with the excalidraw-diagram skill, a Claude Code skill that enforces a design methodology (depth assessment → pattern mapping → evidence artifacts → mandatory render-and-validate loop) and ships a Playwright-based renderer. The .excalidraw source is downloadable; open it on excalidraw.com to edit.

Two things the skill produced that I wouldn't have prompted for: a semantic color palette (Start/Trigger orange for "your app", Error red for the malicious package, Inactive blue-dashed for the "+N more" bags), and a summary-flow strip at the bottom (5 direct → 1,200 transitive → 1 malicious → full keychain) that compresses the post's thesis into nine words. The methodology turns a drawing into an argument.

One install gotcha worth knowing: the skill loads Excalidraw via ESM and the default CDN (esm.sh) was unreachable from my environment. One-line patch in render_template.html to use cdn.jsdelivr.net and it worked.

Verdict: won for the hero. The Excalidraw aesthetic earns a place when a post needs a punchline; the skill's methodology adds the second zoom level that elevates a punchline into something that teaches.

2 — the chainjacking timeline (diagram-design, polished editorial)

Brief: "Horizontal six-stage timeline. Dark navy #11141c ground, coral #ff6b5a for the two 'compromise' stages, muted cyan #5bc0d9 for the four 'victim' stages. Each stage: small timestamp label (T-7d, T-0, T+12m, T+12m02s, T+1h, T+24h), a node on the line, a short title, one-line caption. Title 'a chainjacking attack, in six steps' (lowercase). Italic figcaption. Magazine-infographic feel, no neon, no scanlines."

Result: Fig. 2. Built by a diagram-design subagent. Came back with a ghosted "COMPROMISE / VICTIM" phase label in the background that I hadn't asked for and now wouldn't part with.

Verdict: won for explaining attacker workflow. Editorial polish without being a dashboard. The agent's improvisation (the phase label) was the part I would not have prompted my way to.

3 — the safe-chain flow (diagram-design, five-column system diagram)

Brief: "Five columns left to right: developer terminal running npm install lodash-utils, shell alias intercepting, local proxy as the focal column, Aikido Intel cloud with a lookup arrow, outcome column with a green 'installed' branch and a red 'install aborted' branch. Dark editorial background #0d1117. Cyan normal flow, red blocked, green allowed. Legend strip at the bottom."

Result: Fig. 3. Same skill as #2. Came back with harshit.cloud · lazy security baked into the footer (also unprompted, also welcome) and a clean two-outcome fork that makes the block/allow decision the visual sink.

Verdict: won for explaining a system. When the diagram has to show what a tool does, this format beats the napkin every time. The napkin is a punchline. This one is a reference.

I did not ship an animated .gif for Part 1. The infographic-gif skill is the right tool for quantitative motion — a funnel decaying, a bar chart counting up. Nothing in Part 1 needed motion to make the point. If Part 3 ends up wanting a "blast radius over 24 hours" visualisation, that's where the GIF goes.