feature: Local Syncing Logseq DB with local devices - #12919
Conversation
Add a local-token mode so the self-hosted Node sync server and the app can sync graphs across devices with no login and no internet dependency. Server (deps/db-sync): - DB_SYNC_LOCAL_TOKEN env: when set, this shared secret is the only accepted credential; Cognito/JWT verification is skipped and requests map to a single local user (DB_SYNC_LOCAL_USER_ID, default local-user) - constant-time token comparison; covered by worker-auth tests Client: - new Access token field in Settings -> Sync Server URL (desktop dialog, reused by mobile); stored in localStorage sync-server-token and only active together with a custom sync server URL - token flows to the db worker as :auth/static-sync-token and replaces the Cognito id-token for ws connects and HTTP auth headers, skipping token refresh entirely - login gates for sync (graph upload menu, header indicator, mobile graph list, rtc start/restart flows) also accept local-sync mode - on startup with a local token and no login, remote graphs are fetched and sync starts for the current graph docs: add docs/self-hosted-sync.md with setup instructions
- Node adapter: when no Cognito issuer is configured, generate a local access token on first run, persist it at <data-dir>/local-token (0600) and print it at startup. DB_SYNC_LOCAL_TOKEN still overrides. - Mobile settings: sync server row shows 'Self-hosted · <host>' when a custom server is configured. - docs: setup no longer requires inventing a token.
Server (node adapter, local mode): - startup banner now prints a pairing link http://<lan-ip>:<port>/pair#<token> and renders it as a terminal QR code (TTY only) - new unauthenticated /pair page: reads the token from the URL fragment (never sent to the server) and builds a logseq://sync-setup deep link Client: - new :sync-server/pair-request event: confirmation dialog, then applies server URL + token, pushes worker config and loads remote graphs - logseq://sync-setup?url&token handled on mobile (deeplink) and desktop (electron protocol handler -> syncServerPair renderer message) Pairing a phone is now: scan QR with the camera -> tap Open in Logseq -> confirm.
Closure advanced compilation renames the .isTTY property access, so the TTY check always returned undefined and the QR code was never printed. Use string-based property access (goog.object/getValueByKeys), which survives renaming.
…cal mode The snapshot download handler returns a stream URL the client fetches like a pre-signed link (no Authorization header). Cloud deployments pre-sign the URL, but the self-hosted node adapter enforces auth, so snapshot downloads 401'd and graph download failed at :fetch-snapshot-stream. Embed the caller's token as a query parameter; token-from-request already accepts ?token=.
Asset PUT/GET built the Authorization header from the Cognito id-token only,
which is nil for a self-hosted (static-token) server, so asset transfers went
out unauthenticated and 401'd ("cannot find asset" on every client but the
uploader). Use sync-util/auth-token, which falls back to the static token.
Asset backfill ran only after a fresh graph download and only on electron; web and mobile relied on a render-time one-shot request that races sync-client startup and never retries, so an asset uploaded by another client after this client started syncing was never pulled. Backfill missing remote assets on sync start (idempotent; skips assets already present locally).
Creating a new synced graph invoked list-remote-graphs on a freshly-spawned worker that had no auth token yet, so registration failed with missing-field :auth-token and the graph silently never appeared on the server. Push the sync auth state to the worker before the first RTC call, matching the upload and start paths.
The self-hosted sync URL and token are read from localStorage, which a CLI-spawned node worker does not have, so headless/CLI sync had neither credentials nor base URL (missing-field :auth-token). Seed the sync config and static token from LOGSEQ_SYNC_URL / LOGSEQ_SYNC_TOKEN when set, and re-pin them in set-db-sync-config so a caller pushing the default cloud URL cannot override them. No-op when the variables are unset; GUI behavior unchanged. Enables headless graph upload and scripted sync.
A graph without a :logseq.kv/graph-rtc-e2ee? datom (e.g. imported or legacy) normalized to e2ee=true, so uploading it required user RSA keys that do not exist in a local (static-token) deployment, failing with missing-field :user-rsa-key-pair. Default nil to false when a static sync token is configured; unchanged for the hosted service.
|
Thanks for this — I've been running it self-hosted (a local db-sync server on my own network, no Logseq account) across desktop (Linux + Windows), the web app, Android, and the bundled CLI. It works, but I hit six issues in local mode and fixed each. Branch with your 4 commits rebased onto current The fixes, each a focused commit:
All six are self-hosted-only paths; none change behavior against the hosted Logseq sync service. (Also rebased past the recent |
|
Hi, buddy @pushakargaikwad! Thanks a lot for helping me testing this new feature! I just fixed the mess with my branch caused by my dump idea of opening the PR from my master branch haha 😅 Thanks again for your time testing this, resolving the bugs and commenting on this PR. |
|
Signed, so the CLA check is green now and CI should be unblocked. 🎉 Thanks for rebasing my six fixes in so fast! I'll keep running the branch self-hosted across desktop, web, mobile and the CLI, and I'll flag anything else that turns up. If the maintainers want changes during review, I'm happy to help address them. Really hoping this lands in a release. Great work on the core feature. 🔥 |
Hello folks! I was reading this discussion here about syncing logseq DB with other devices locally, and I think a got a pretty nice solution I want to present you!
Like many people in that thread, I used to keep my Markdown graph in sync between my desktop, laptop and phone with Syncthing. After moving to the DB version that stopped being an option — syncing a live SQLite database at the file level corrupts graphs (WAL files, per-device
client-opsstate) and can't merge concurrent edits. The real answer is the sync server, and the codebase already ships almost everything needed: the Node.js adapter for self-hosting (ADR 0001) and the custom sync server URL setting. The only thing missing was a way to use them without a Logseq account, fully offline.This PR fills that gap. The whole experience ends up being:
DB_SYNC_DATA_DIR=~/logseq-sync-data node worker/dist/node-adapter.jsThen on my phone I just scan the QR code with the camera → tap "Open in Logseq" → confirm. No account, no typing a token on a phone keyboard, no internet dependency. Graph data never leaves my network.
What's in the PR
1. Local-token mode on the self-hosted Node adapter
When
DB_SYNC_LOCAL_TOKENis set, that shared secret becomes the only accepted credential: JWT/Cognito verification is skipped entirely and every request maps to a single local user (DB_SYNC_LOCAL_USER_ID, defaultlocal-user). The check lives at the single auth choke point (worker/auth.cljs#auth-claims) with a constant-time comparison, so it covers HTTP, WebSocket sync and assets uniformly. The hosted Cloudflare deployment never sets this env var, so its behavior is completely unchanged.On the client, the Sync Server settings (desktop dialog, also reachable from mobile settings) get an Access token field next to the existing custom URL. The token is stored alongside the URL, only ever sent to a custom server (never to the official service), and flows to the DB worker as a static credential that replaces the Cognito id-token — no token refresh, no login. The login gates around sync (upload menu, header indicator, mobile graph list, rtc start/restart flows) now also accept this mode, following the precedent that
rtc-group?already returns true when a custom sync server is configured.2. Zero-config token
You shouldn't need to invent a secret to sync your own notes. If the adapter starts with no Cognito configuration, it generates a token on first run, persists it at
<data-dir>/local-token(mode0600), reuses it on every restart, and prints it at startup.DB_SYNC_LOCAL_TOKENstill overrides it.3. One-scan device pairing
Typing a 64-char token on a phone was the last painful step, so the server also prints a pairing link and a terminal QR code. The QR encodes
http://<lan-ip>:<port>/pair#<token>— note the token travels in the URL fragment, so it's never sent over the network; the unauthenticated/pairpage contains no secret and just turnslocation.origin+ fragment into alogseq://sync-setup?url=…&token=…deep link client-side. The app handles that link on both desktop (electron protocol handler) and mobile (existing deeplink dispatcher) and always shows a confirmation dialog before applying anything, so a malicious link can't silently redirect someone's sync to another server.Security considerations
I deliberately kept a shared secret instead of adding a no-auth mode: the moment a phone syncs, the server listens on the LAN, and the token is the only thing standing between "sync works" and "anyone on the Wi-Fi can read/wipe your notes". The auto-generated token keeps that protection without any setup cost.
In local mode the token is exclusive — a JWT that would otherwise verify is rejected, so there's no accidental mixed-auth surface.
Docs tell users to treat the QR/pairing link as a secret and to only expose the server on a trusted network or behind TLS.
Known limitations (intentional scope)
Local-token mode is single-user: every device acts as the same user, so member invitations/roles don't apply. That matches the "my own devices" use case this targets.
The QR/
/pairpage is Node-adapter only (not the Cloudflare worker) — it only makes sense for self-hosting.Possible follow-ups I left out to keep this reviewable: mDNS discovery (find the server with no URL at all) and showing a pairing QR inside the desktop app's settings.
How I tested it
pnpm test:node-adapter: 203 tests / ~4200 assertions passing, including new coverage for local-token auth (match/mismatch/missing/JWT-rejected-when-local), token generation/persistence/env-precedence.All shadow-cljs builds (
app,db-worker,db-worker-node,electron,mobile) compile with 0 warnings; clj-kondo clean on every touched file.Live end-to-end on my LAN: server on my desktop, real devices pointed at
http://192.168.x.x:8787— wrong/missing token → 401, token → 200, graph upload/download/edit syncing across devices,/pairpage and QR verified on a real terminal (which caught a fun bug: Closure advanced compilation renames.isTTY, hence the string-access fix in the last commit).Docs are included:
docs/self-hosted-sync.md(user-facing walkthrough) plus updates todeps/db-sync/README.md.I know self-hosting has been a much-requested topic and that ADR 0001 already pointed at "pluggable auth providers" as follow-up work — I hope this is a useful step in that direction. Happy to adjust naming, split the PR, or rework any part of the approach based on your feedback. Thanks for the amazing work on the DB version! 💜