Skip to content

Merge upstream karakeep-app changes (resolves #5) - #7

Merged
johnford2002 merged 98 commits into
mainfrom
merge/upstream-sync
May 19, 2026
Merged

johnford2002 merged 98 commits into
mainfrom
merge/upstream-sync

Conversation

@johnford2002

@johnford2002 johnford2002 commented May 19, 2026 •

Copy link
Copy Markdown
Owner

Summary

Resolves the conflicts in #5 by merging karakeep-app/main into our PostgreSQL-supporting fork (~95 upstream commits, 446 files, 29.8k+/3.3k LOC). Adds a parallel PG migration so both dialects pick up the upstream schema changes.

Upstream's two schema changes (apiKey.scopes column, removal of ruleEngineRulesTable.listId/FK) needed propagation from schema.ts (now an entry-point shim in our fork) into schema.sqlite.ts and schema.pg.ts, plus an equivalent PostgreSQL migration.

What's coming in from upstream

Features

Core / Web App

Mobile

Extension / CLI / MCP

Fixes

Security / SSRF / Auth

Crawler / Workers

Web / UI

Mobile

Docker / Build

Refactors

Tooling / Infrastructure

Documentation

Conflict resolutions

File Resolution
packages/db/schema.ts Kept entry-point shim (HEAD)
packages/db/schema.sqlite.ts Added scopes column, removed rule-engine listId + FK
packages/db/schema.pg.ts Same as SQLite; used jsonb for scopes
packages/db/package.json Combined upstream's --ignore-path flags with our migrate-to-pg/test scripts
packages/db/migrations/sqlite/{0083,0084}* Accepted upstream files at our renamed path
packages/shared/config.ts Kept both PG and OTLP validations
packages/trpc/models/lists.ts Kept isUniqueConstraintError abstraction; added KarakeepDBTransaction import
docker/docker-compose.dev.yml Accepted upstream (superset of our node_modules volume)
pnpm-lock.yaml Regenerated via pnpm install

PostgreSQL parity

Added packages/db/migrations/pg/0001_upstream_schema_sync.sql to mirror upstream's SQLite migrations 0083/0084. Drizzle-kit's auto-generated diff was missing two things needed for parity:

  1. SQL DEFAULT '["fullaccess"]'::jsonb on apiKey.scopes (so existing rows backfill — matches SQLite's DEFAULT '["fullaccess"]').
  2. Data migration for the event JSON on ruleEngineRules: delete rules with empty listId in addedToList/removedFromList events, then convert single listId to a listIds array.

Both translate the SQLite json_extract/json_set logic to PG's jsonb operators.

Docker overlay updates

Also bumped docker/docker-compose.postgres.yml from postgres:16-alpine to postgres:18-alpine to match the production deployment. While verifying the overlay end-to-end, found that PG18 changed its data-directory convention: data lives under /var/lib/postgresql/<major>/docker/, so the image refuses to start if the volume is still mounted at the legacy /var/lib/postgresql/data path. Fixed the mount by moving it one level up to /var/lib/postgresql, which lets PG18 own its version-specific subdirectory.

Heads up for anyone running a previous PG16 version of this fork: the volume layout is incompatible across the bump; switching requires pg_upgrade or a dump/restore — you can't just pull and restart.

The overlay header now also documents the standalone migration-smoke-test pattern (combine with any base compose file, then up postgres -d, then run pnpm --filter=@karakeep/db migrate from the host).

Once this is merged, PR #5 can be closed.

Test plan

  • pnpm typecheck — clean across 22 packages
  • pnpm lint — clean across 19 packages
  • pnpm --filter=@karakeep/db test — schema-sync test confirms SQLite and PG column lists match
  • pnpm exec drizzle-kit check --config=drizzle.config.pg.ts — migration/schema consistent
  • pnpm exec drizzle-kit check --config=drizzle.config.sqlite.ts — migration/schema consistent
  • pnpm --filter=@karakeep/trpc test — 407 passed
  • pnpm --filter=@karakeep/shared test — 90 passed
  • pnpm --filter=@karakeep/shared-server test — 8 passed
  • pnpm --filter=@karakeep/api test — 2 passed
  • TZ=UTC pnpm --filter=@karakeep/web test — 25 passed (lib/date-format.test.ts is upstream's, assumes UTC; CI runs UTC)
  • PostgreSQL data migration smoke test on postgres:16-alpine and postgres:18-alpine (PG 18.4, matches my production instance): applied 0001_upstream_schema_sync.sql against each, seeded with rules containing old-format event JSON. Identical results on both versions — rules with valid listId got transformed to listIds: [...], rules with empty/missing listId were deleted, rules with other event types were untouched. Confirmed apiKey.scopes defaults to ["fullaccess"] for new rows and that listId/FK are gone.
  • CI green
  • Smoke test SQLite startup (apply 0083, 0084) in a running app

🤖 Generated with Claude Code

MohamedBassem and others added 30 commits April 13, 2026 17:45
…app#2697)

The upstream fix for the status bar inset issue was merged and released
in v1.21.5, so our local patch is no longer needed.

Closes karakeep-app#2581
…t sentry dsn (karakeep-app#2700)

* docs(mobile): add SENTRY_DISABLE_AUTO_UPLOAD=true to build apk without sentry dsn

* include pr suggestion

* put SENTRY_DISABLE_AUTO_UPLOAD=true in package.json scripts instead
* deps: upgrade to zod 4

* fix: preserve zod 4 schema compatibility

Use safeExtend where refined signup schemas are extended so the affected modules can initialize safely under Zod 4. Keep the generated OpenAPI output aligned with runtime validation and centralize the zod-to-openapi initialization.
…p-app#2691)

* feat(mobile): add keep screen on toggle for bookmark viewing

Adds a Reading-section toggle that activates expo-keep-awake while a
bookmark is open, covering all in-app views (reader, browser, screenshot,
archive, pdf, text, asset). Defaults off; releases the wake lock as soon
as the user navigates away from the bookmark detail screen.

Closes karakeep-app#2650

* fix(mobile): disable keep-screen-on toggle while settings are hydrating

Prevents writing stale default settings to SecureStore if the user
toggles before hydration completes.
Co-authored-by: Luis Zuniga <wichogzf@gmail.com>
* feat(import): add OneTab import support

Adds a OneTab import card to the ImportExport settings page.
Implements a parser for OneTab's "URL | Title" line format with
deduplication, and registers "onetab" as an import source.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ne (karakeep-app#2722)

- Wrap reset token replacement in a transaction
- Delete any prior tokens for the user before inserting the new token
* Sanitize user names before storing them

- Strip HTML from signup and OAuth display names
- Enforce non-empty plain-text names in validation
- Add tests for sanitized and HTML-only name inputs

* more changes

* Harden user name validation

* more changes
…karakeep-app#2657)

Monolith-generated full-page archives inline stylesheets and fonts as
data: URIs. The previous CSP blocked these, causing broken rendering.
Adding data: to style-src and a new font-src directive unblocks them,
matching the precedent set by img-src which already includes data:.

Fixes: karakeep-app#2621

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Wraps PATCH /bookmarks/{bookmarkId} so MCP clients can modify an
existing bookmark instead of only creating/reading one. Exposes the
commonly useful patchable fields (title, note, summary, archived,
favourited, url, description, author, publisher, createdAt); less
common/destructive ones (text, assetContent, datePublished,
dateModified) are intentionally omitted to reduce accidental clobbering
of crawler-populated data.

The PATCH response schema does not include tags/content/assets, so the
tool follows up with a GET to return a full compactBookmark view of the
updated record.
…rakeep-app#2655)

signOut({ redirect: false }) returns an absolute URL constructed from
NEXTAUTH_URL (e.g. http://localhost:3000/). When Docker remaps an
external port to the internal one, this sends users to the wrong port.
Using router.push('/') lets Next.js resolve against the user's origin.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(web): auto-add bookmark to list on selection

Remove the 'Add' button from Manage Lists modal. When a user selects
a list from the dropdown, the bookmark is now automatically added to
that list, simplifying the user experience.

Users can still remove a bookmark from a list by clicking the X button
next to the list name.

Fixes karakeep-app#2604

* fix: add loading state and prevent duplicate submission in ManageListsModal

- Added isPending state from useAddBookmarkToList hook
- Added duplicate submission prevention in onChange handler
- Added disabled prop to BookmarkListSelector component

* style: fix indentation in useAddBookmarkToList callback

- Fix incorrect indentation of onSuccess and onError callbacks
- Add trailing newline at end of file

* fix: address review comments for PR karakeep-app#2629

- Add trailing newline at end of file
- Remove redundant in-flight guard check (disabled prop handles it)
- Add guard for getPathById to prevent crash on stale cache
- Disable interactions while list membership data is loading

* fix: use translation for DialogTitle in ManageListsModal

* fix: add isLoading guard to onChange callback in BookmarkListSelector

* fix(web): improve ManageListsModal accessibility and rendering

- Add aria-label to remove button for accessibility (CodeRabbit comment 3006737430)
- Remove allLists gating to allow list.name fallback to render (CodeRabbit comment 3006737429)
- Use optional chaining for path lookup
…ep-app#2693)

* feat: add manual save toggle for browser extension

* fix: add check for appending ellipses
MohamedBassem and others added 28 commits May 7, 2026 12:23
Co-authored-by: CircleCrop <guhengting@gmail.com>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Co-authored-by: OpenAI <noreply-mt-openai@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/ar/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/cs/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/da/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/de/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/el/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/en_US/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/es/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/fa/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/fi/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/fr/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/ga/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/gl/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/hr/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/hu/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/it/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/ja/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/ko/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/nb_NO/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/nl/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/pl/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/pt/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/pt_BR/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/ru/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/sk/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/sl/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/sv/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/tr/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/uk/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/vi/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/zh_Hans/
Translate-URL: https://hosted.weblate.org/projects/karakeep/karakeep/zh_Hant/
Translation: Karakeep/Karakeep
…t side crawling (karakeep-app#2784)

Move `<all_urls>` from `host_permissions` to `optional_host_permissions` so it
isn't requested at install time. The permission is now requested via
`chrome.permissions.request()` when the user enables client-side crawling in the
extension settings, and removed when they disable it. The settings toggle and
the capture path both gate on whether the permission is actually granted.

https://claude.ai/code/session_01NnQnr1B9KSRriTaQkZM6AP

Co-authored-by: Claude <noreply@anthropic.com>
…-app#2793)

The settings screen is presented as an iOS formSheet with an opaque,
non-large header (since 649efb8). iOS sizes the formSheet's inner
content view ~headerHeight pixels taller than the visible sheet area,
so the bottom of the settings list was clipped behind the sheet's
bottom edge.

contentInsetAdjustmentBehavior="automatic" correctly handles the top
(adding the right top inset to clear the opaque header), but does not
add a corresponding bottom inset for the formSheet's invisible bottom
region. Adding useHeaderHeight() to paddingBottom pushes the trailing
content up to a visible position.
…-app#2794)

The MEDIA → Upload Image Quality row in mobile settings crammed the
label, percentage, and slider onto one horizontal line, leaving the
slider only a few dozen pixels wide on phone widths. Switch the row
to a vertical layout: label + percentage on top, full-width slider
beneath.
…#2788)

* Set max width for reader toolbar on tablets

* fix(mobile): remove FAB bottom inset on iPad

iPad's tab bar geometry doesn't need the extra 57–60pt lift the iPhone FAB uses.

* fix(mobile): give reader toolbar icons proper spacing on iPad

Replace the inner row's width/maxWidth + space-between approach with an
intrinsic gap. The earlier cap caused the row to collapse to its natural
icon width inside the hug-fit GlassView, leaving icons cramped against
each other. Using `gap: 28` gives each icon real breathing room and lets
the pill size to its contents naturally.
This is a temporary fix to be able to patch the recent NextJS security
update until a proper fix is available.
Resolves conflicts from upstream's API key scopes feature
and rule-engine multi-list refactor, propagating both into the
dialect-specific schema files (schema.sqlite.ts and schema.pg.ts)
since our fork uses schema.ts as a dialect-routing entry-point shim.

Conflict resolutions:
- packages/db/schema.ts: kept entry-point shim (HEAD)
- packages/db/schema.sqlite.ts: added apiKeys.scopes column,
  removed ruleEngineRulesTable.listId and its FK
- packages/db/schema.pg.ts: same as sqlite, using jsonb for scopes
- packages/db/package.json: combined upstream's --ignore-path flags
  with our migrate-to-pg/test scripts
- packages/db/migrations/sqlite/: accepted upstream 0083 and 0084
  migrations at the new path (renamed from drizzle/ in our fork)
- packages/shared/config.ts: kept both PostgreSQL validation and
  upstream's OTLP endpoint validation
- packages/trpc/models/lists.ts: kept isUniqueConstraintError
  abstraction, added KarakeepDBTransaction import
- docker/docker-compose.dev.yml: accepted upstream (superset of our
  node_modules volume change)
- pnpm-lock.yaml: regenerated via pnpm install
Adds PG equivalent of the SQLite migrations 0083 (apiKey scopes) and
0084 (rule engine multi-list refactor) brought in by the upstream merge.

The auto-generated migration was missing two things needed for parity
with the SQLite version:

1. A SQL DEFAULT for apiKey.scopes to backfill existing rows (matches
   SQLite's `DEFAULT '["fullaccess"]'`).
2. Data migration for the event JSON in ruleEngineRules: delete rules
   with empty listId in addedToList/removedFromList events, then
   transform single listId to a listIds array.

Both data-migration steps mirror what upstream's SQLite migration does
using SQLite's json_extract/json_set, translated to PG's jsonb operators.

Verified with `drizzle-kit check` (schema/migration consistent) and the
schema-sync test (column lists match across dialects).
Bumps the PostgreSQL overlay from postgres:16-alpine to postgres:18-alpine
to match the production deployment, and fixes the volume mount that PG18
rejects.

PG18+ stores data in a version-specific subdirectory under
/var/lib/postgresql (e.g. /var/lib/postgresql/18/docker). The image
refuses to start if the volume is mounted at the legacy
/var/lib/postgresql/data path, even on a fresh install. Mounting one
level up lets PG18 manage its own subdirectory layout.

Also documents the migration-smoke-test command pattern in the overlay
header: pair the overlay with a base compose file and `up postgres -d`
to bring up just postgres, then run `pnpm --filter=@karakeep/db migrate`
from the host. Verified end-to-end against postgres:18-alpine (PG 18.4).

Upgrading an existing PG16 fork deployment to this overlay is a breaking
change — the volume layout is incompatible and requires pg_upgrade or
a dump/restore.
@johnford2002
johnford2002 merged commit 56dd847 into main May 19, 2026
5 checks passed
@johnford2002
johnford2002 deleted the merge/upstream-sync branch May 19, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.