[doot]

karakeep-workers.service on nmd has been dead since 05:23 PT today, and it stays dead — every start aborts immediately.

What happens

karakeep-workers.service: Main process exited, code=killed, status=6/ABRT
karakeep-workers.service: Failed with result 'signal'.
  #  Assertion failed: (env) != nullptr
  #  node[...]: void node::RemoveEnvironmentCleanupHook(...) at ../../src/api/hooks.cc:142
  3: Statement::~Statement() [.../better-sqlite3/build/Release/better_sqlite3.node]

node 24.19.0 aborts when better-sqlite3 finalizes a Statement. Reproduced again at 19:19 UTC on a manual systemctl restart, so it is deterministic, not a shutdown race.

How it reached the host

• PR Update flake.lock #835 bumped nixpkgs-unstable to 26.11pre1052672.867dcbc30baf (merged 07:20 UTC).
• That rev is the first to carry nixpkgs commit d14174cf7 — nodejs_24: 24.18.1 -> 24.19.0 (Aug 3).
• nixos-upgrade on nmd pulled it at ~04:40 PT; karakeep restarted; ABRT at 05:23 PT.

The preceding lock (265b7e2dbbdb) still had 24.18.1, which is why this appeared exactly today.

Fix

Build karakeep against a nixpkgs rev pinned at nodejs 24.18.1, scoped to that one package on that one host. Nothing else changes node version.

Upstream (karakeep#2989, fixed by #2996) pins node for Docker builds only, so nixpkgs consumers are still exposed. There is no nixpkgs issue filed for this yet.

Revert the input and the .override once nodejs_24 in unstable carries the node-side fix.

Notes for review

• flake.lock is not updated in this branch — CI will resolve the new nixpkgs-node input. Happy to commit a lock update instead if you'd rather it be explicit.
• Alternative considered and rejected: overrideAttrs to swap node's src in place. nodejs.nix derives build flags from the argument version in let bindings that overrideAttrs cannot reach, so it would build 24.18.1 sources with 24.19.0 feature flags. Pinning a rev is the honest version.
• This is unverified by a real build — I have no nix daemon in my container. Worth a nixos-rebuild build on nmd before merging.

Monitoring gap this exposed

The daily health cron did not catch it, for two independent reasons — both fixed outside this PR:

1. It ran at 15:00 PT, ~10h after the alert, and had been erroring out for 3 days on an unrelated model-pin guard. Moved to 07:00 PT (after the ~04:00 upgrade window) and pinned.
2. The probe printed firing alerts by rule name only, so "which host, which unit" never reached the agent. It now emits instance + unit + age.

[Copilot]

Pull request overview

Pins Node.js 24.18.1 for Karakeep on nix-media-docker to avoid the better-sqlite3 crash.

Changes:

• Adds a pinned nixpkgs input.
• Overrides Karakeep’s Node.js dependency.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
flake.nix	Adds the Node.js-compatible nixpkgs revision.
systems/nix-media-docker/default.nix	Applies the pinned Node.js package to Karakeep.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.