Repository navigation
Fix crash loop caused by better-sqlite3 and node. - #2996
Merged
Merged
Conversation
As outlined in karakeep-app#2989 Karakeep keeps crashing due to a bad interaction between node 24.19 and better-sqlite3. This commit pins node to 24.18.1 for Docker builds, which still works fine. Once the bug is fixed in node and the fixed version has been uploaded to Docker hub, this commit can be reverted.
1 task done
Contributor
Greptile SummaryThe PR temporarily pins all production Docker image stages to Node 24.18.1 to avoid the documented Node 24.19 and better-sqlite3 crash loop.
Confidence Score: 5/5The PR appears safe to merge and consistently applies the intended Node workaround to every production Docker target. All Node-based stages are pinned directly or inherit from the pinned all-in-one builder, and the configured Docker build platforms support the selected image tag. Important Files Changed
Reviews (1): Last reviewed commit: "Fix crash loop caused by better-sqlite3 ..." | Re-trigger Greptile |
MohamedBassem
approved these changes
Aug 11, 2026
MohamedBassem
left a comment
MohamedBassem
left a comment
Member
There was a problem hiding this comment.
Thanks for the PR! The change looks good to me, but the tests are blocked on #3000. Trying to come up with a solution to this now.
ajgon
pushed a commit
to deedee-ops/home-ops
that referenced
this pull request
Aug 11, 2026
…0.33.2) (#600) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/karakeep-app/karakeep](https://github.com/karakeep-app/karakeep) | patch | `0.33.1` → `0.33.2` | --- >⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/7) for more information. --- ### Release Notes <details> <summary>karakeep-app/karakeep (ghcr.io/karakeep-app/karakeep)</summary> ### [`v0.33.2`](https://github.com/karakeep-app/karakeep/releases/tag/v0.33.2): 0.33.2 [Compare Source](karakeep-app/karakeep@v0.33.1...v0.33.2) ##### 0.33.2 This is a patch release for after [0.33.1](https://github.com/karakeep-app/karakeep/releases/tag/v0.33.1). It contains the following fixes: **Bug fixes:** - Change default "use max completion tokens" token env variable to `true` when using the default OpenAI configuration now that we're using 5.6 luna and this flag is required there ([#​3007](karakeep-app/karakeep#3007)). - We're migrating to a custom karakeep chrome container (based on <https://github.com/chromedp/docker-headless-shell>) now that the old alpine-chrome is no longer fetch-able (and has been long abandoned by the maintainer). Check the migration guide [here](https://docs.karakeep.app/next/administration/chrome-image-migration). The old image will continue to work, but it's recommended that you migrate to the new one as it also upgrades chrome to 151 (compared to 124 in the old image). If you've been using browserless (e.g. on unraid), you're not impacted by this, and you don't need to migrate. - Large singlefile archives (typically because of videos) used to fail metadata extraction as the parser used to OOM. We strip inline videos now to make the parsing succeed. - Hardened the link to asset conversion mechanism in the crawler that used to result into silently broken assets. ([#​3002](karakeep-app/karakeep#3002)) - Pinned nodejs to version 24.18.1 thanks to [@​eriktews](https://github.com/eriktews) ([#​2996](karakeep-app/karakeep#2996)) to overcome a bug in the latest node 24.19 version (check [#​2989](karakeep-app/karakeep#2989) for details). The proxmox scripts also were pinned by [@​MickLesk](https://github.com/MickLesk). - Karakeep's favicon is now light/dark mode aware, thanks to [@​ehfreema](https://github.com/ehfreema) ([#​2990](karakeep-app/karakeep#2990)). **New features:** - You can now upload custom SingleFile archives to karakeep directly using the CLI. ##### Changelog - fix: Set max completion tokens to true when using default openai config ([#​3007](karakeep-app/karakeep#3007)) - [@​MohamedBassem](https://github.com/MohamedBassem) in [`98c0c78`](karakeep-app/karakeep@98c0c789) - feat: migrate to a custom karakeep chrome image ([#​3005](karakeep-app/karakeep#3005)) - [@​MohamedBassem](https://github.com/MohamedBassem) in [`951e502`](karakeep-app/karakeep@951e5025) - feat: publish a new karakeep chrome container ([#​3004](karakeep-app/karakeep#3004)) - [@​MohamedBassem](https://github.com/MohamedBassem) in [`b6415b3`](karakeep-app/karakeep@b6415b3c) - feat: favicon adapts to browser theme ([#​2990](karakeep-app/karakeep#2990)) - [@​ehfreema](https://github.com/ehfreema) in [`ad27c05`](karakeep-app/karakeep@ad27c051) - build: Fix crash loop caused by better-sqlite3 and node. ([#​2996](karakeep-app/karakeep#2996)) - [@​eriktews](https://github.com/eriktews) in [`132e566`](karakeep-app/karakeep@132e566b) - fix(crawler): fail on failure to download urls converted to assets ([#​3002](karakeep-app/karakeep#3002)) - [@​MohamedBassem](https://github.com/MohamedBassem) in [`f57aad1`](karakeep-app/karakeep@f57aad1d) - feat(mobile): switch to using context menu in ios - [@​MohamedBassem](https://github.com/MohamedBassem) in [`28e7909`](karakeep-app/karakeep@28e79094) - fix(restate): use logger middleware instead of durable steps - [@​MohamedBassem](https://github.com/MohamedBassem) in [`764ba85`](karakeep-app/karakeep@764ba850) - fix(workers): drop inline media blocks before metadata parsing - [@​MohamedBassem](https://github.com/MohamedBassem) in [`1c0b0a9`](karakeep-app/karakeep@1c0b0a9a) - feat(cli): allow importing singlefile archives via the CLI - [@​MohamedBassem](https://github.com/MohamedBassem) in [`5f6ba26`](karakeep-app/karakeep@5f6ba262) - docs: update intro doc with new features - [@​MohamedBassem](https://github.com/MohamedBassem) in [`3af6f45`](karakeep-app/karakeep@3af6f458) - readme: update the readme and docs with the new features - [@​MohamedBassem](https://github.com/MohamedBassem) in [`a81f935`](karakeep-app/karakeep@a81f935a) - release: release cli, sdk & mcp - [@​MohamedBassem](https://github.com/MohamedBassem) in [`b3fbf6b`](karakeep-app/karakeep@b3fbf6bb) </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/Warsaw) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC44LjEiLCJ1cGRhdGVkSW5WZXIiOiI0NC44LjEiLCJ0YXJnZXRCcmFuY2giOiJtYXN0ZXIiLCJsYWJlbHMiOlsicmVub3ZhdGUvY29udGFpbmVyIiwidHlwZS9wYXRjaCJdfQ==--> Reviewed-on: https://git.ajgon.casa/deedee/mark13/pulls/600
doot
added a commit
to doot/nixos-config
that referenced
this pull request
Aug 14, 2026
`karakeep-workers.service` on nmd has been dead since 05:23 PT today, and it stays dead — every start aborts immediately. ## What happens ``` karakeep-workers.service: Main process exited, code=killed, status=6/ABRT karakeep-workers.service: Failed with result 'signal'. # Assertion failed: (env) != nullptr # node[...]: void node::RemoveEnvironmentCleanupHook(...) at ../../src/api/hooks.cc:142 3: Statement::~Statement() [.../better-sqlite3/build/Release/better_sqlite3.node] ``` node 24.19.0 aborts when better-sqlite3 finalizes a `Statement`. Reproduced again at 19:19 UTC on a manual `systemctl restart`, so it is deterministic, not a shutdown race. ## How it reached the host - PR #835 bumped `nixpkgs-unstable` to `26.11pre1052672.867dcbc30baf` (merged 07:20 UTC). - That rev is the first to carry nixpkgs commit `d14174cf7` — `nodejs_24: 24.18.1 -> 24.19.0` (Aug 3). - `nixos-upgrade` on nmd pulled it at ~04:40 PT; karakeep restarted; ABRT at 05:23 PT. The preceding lock (`265b7e2dbbdb`) still had 24.18.1, which is why this appeared exactly today. ## Fix Build karakeep against a nixpkgs rev pinned at nodejs 24.18.1, scoped to that one package on that one host. Nothing else changes node version. Upstream ([karakeep#2989](karakeep-app/karakeep#2989), fixed by [#2996](karakeep-app/karakeep#2996)) pins node **for Docker builds only**, so nixpkgs consumers are still exposed. There is no nixpkgs issue filed for this yet. Revert the input and the `.override` once `nodejs_24` in unstable carries the node-side fix. ## Notes for review - `flake.lock` is not updated in this branch — CI will resolve the new `nixpkgs-node` input. Happy to commit a lock update instead if you'd rather it be explicit. - Alternative considered and rejected: `overrideAttrs` to swap node's `src` in place. `nodejs.nix` derives build flags from the *argument* `version` in `let` bindings that `overrideAttrs` cannot reach, so it would build 24.18.1 sources with 24.19.0 feature flags. Pinning a rev is the honest version. - This is unverified by a real build — I have no nix daemon in my container. Worth a `nixos-rebuild build` on nmd before merging. ## Monitoring gap this exposed The daily health cron did not catch it, for two independent reasons — both fixed outside this PR: 1. It ran at 15:00 PT, ~10h after the alert, and had been erroring out for 3 days on an unrelated model-pin guard. Moved to 07:00 PT (after the ~04:00 upgrade window) and pinned. 2. The probe printed firing alerts by rule name only, so "which host, which unit" never reached the agent. It now emits instance + unit + age.
4 tasks done
anpryl
added a commit
to anpryl/karakeep
that referenced
this pull request
Aug 26, 2026
Consumers can now use this repository directly:
inputs.karakeep.url = "github:anpryl/karakeep";
imports = [ inputs.karakeep.nixosModules.default ];
disabledModules = [ "services/web-apps/karakeep.nix" ];
instead of taking the source with flake = false and rebuilding nixpkgs' karakeep
derivation around it through an overlay.
WHY. Supplying the source while nixpkgs owned the build meant the two had to agree,
and they no longer do: nixpkgs' karakeep is 0.32.0 and passes pnpm_9, while upstream
v0.33.x declares pnpm@11.2.1. The consumer already carried workarounds for the gap
(dropping dont-lock-pnpm-version.patch, sed-ing packageManager, pinning
pnpmDeps.outputHash, permitting an insecure pnpm). Owning the derivation removes the
whole class, and is the prerequisite for merging this fork up to v0.33.2 at all.
nix/package.nix is vendored from nixpkgs and kept deliberately close to the original
so it can be diffed when that package changes. Adaptations:
- src and version come from the flake, not a fetched tag;
- dont-lock-pnpm-version.patch is replaced by a pattern substitution — the patch is
a line-numbered diff against upstream's line 40 and this tree has it on line 38,
and the substitution also survives pnpm 9 -> 11;
- versionCheckHook replaced by an explicit installCheckPhase (see below);
- pnpmDeps hash recomputed. It came out byte-identical to the hash the consumer's
overlay was already pinning, which is a useful cross-check that this builds the
same dependency set as the deployment in service today.
The install check is deliberate. versionCheckHook cannot work here (it greps
`karakeep --version` for `version`; ours is 0-unstable-<rev> while the binary reports
0.31.0), but simply disabling it would leave the derivation with no evidence that
what it produces can run — the exact failure this package has already had, where it
compiled cleanly and aborted two seconds after start. So the phase runs the CLI and
exercises the better-sqlite3 native module instead.
Its limits are documented because they were measured. Building against nodejs 24.19.0
SUCCEEDS and this check PASSES, so it does not guard the node pin. Four standalone
attempts against that 24.19 build failed to reproduce the abort — one statement then
close, unclosed statements at exit, 2000 statements with a forced GC, the database
left open. WHY they failed to reproduce is not established: four negative results do
not identify a cause, and the abort may need karakeep's actual startup path or timing
those scripts never created. Only the negative conclusion is supported — this check
does not cover that failure.
The node pin lives here now rather than in the consumer. node 24.19.0 makes the
bundled better-sqlite3 abort on startup ("Assertion failed: (env) != nullptr" in
Statement::~Statement()); upstream hit the same bug and fixed it the same way
(karakeep-app#2989, karakeep-app#2996), but their pin is in a Dockerfile and never reaches a Nix build.
There is no node 24.20, so this is not short-lived: when node ships the V8/GC fix,
drop the input rather than bumping it.
nixpkgs is pinned rather than following the consumer, so this builds standalone. The
rev is load-bearing twice over: package.nix is vendored from it, and it is the last
one that still has pnpm_9 — newer nixpkgs removed it as an unconditional throw
("reached EOL on 2026-04-30") that no config can override. Bumping nixpkgs and
moving to pnpm 11 are therefore one change, not two.
Verified: nix flake check --all-systems passes; the package builds on x86_64-linux;
its runtime closure contains nodejs-24.18.1 and nothing else; and it lays out
lib/karakeep/{migrate,start-web,start-workers} plus bin/karakeep exactly as the
module's ExecStart paths expect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
As outlined in
#2989
Karakeep keeps crashing due to a bad interaction
between node 24.19 and better-sqlite3.
This commit pins node to 24.18.1 for Docker
builds, which still works fine. Once the bug is
fixed in node and the fixed version has been
uploaded to Docker hub, this commit can be
reverted.
Fixes #2989
How Has This Been Tested?
Docker build and docker run on a system that was previously in a crash loop and with this patch, it works fine again.
Checklist:
Please describe to which degree, if any, an LLM was used in creating this pull request.
No, no LLM was used.