Skip to content

feature: Local Syncing Logseq DB with local devices - #12919

Open
FelipeFTN wants to merge 10 commits into
logseq:masterfrom
FelipeFTN:master
Open

feature: Local Syncing Logseq DB with local devices#12919
FelipeFTN wants to merge 10 commits into
logseq:masterfrom
FelipeFTN:master

Conversation

@FelipeFTN

Copy link
Copy Markdown

Hello folks! I was reading this discussion here about syncing logseq DB with other devices locally, and I think a got a pretty nice solution I want to present you!

Like many people in that thread, I used to keep my Markdown graph in sync between my desktop, laptop and phone with Syncthing. After moving to the DB version that stopped being an option — syncing a live SQLite database at the file level corrupts graphs (WAL files, per-device client-ops state) and can't merge concurrent edits. The real answer is the sync server, and the codebase already ships almost everything needed: the Node.js adapter for self-hosting (ADR 0001) and the custom sync server URL setting. The only thing missing was a way to use them without a Logseq account, fully offline.

This PR fills that gap. The whole experience ends up being:

DB_SYNC_DATA_DIR=~/logseq-sync-data node worker/dist/node-adapter.js
Local sync mode: no Cognito auth configured.
Access token: 4f3c…9b21
(persisted at ~/logseq-sync-data/local-token; set DB_SYNC_LOCAL_TOKEN to override)

Pair a device: open http://192.168.1.10:8787/pair#4f3c…9b21
or scan:
█▀▀▀▀▀█ ▀▄█▄▀ █▀▀▀▀▀█
█ ███ █ ▀█▄▀▄ █ ███ █  …

Then on my phone I just scan the QR code with the camera → tap "Open in Logseq" → confirm. No account, no typing a token on a phone keyboard, no internet dependency. Graph data never leaves my network.

What's in the PR

1. Local-token mode on the self-hosted Node adapter

When DB_SYNC_LOCAL_TOKEN is set, that shared secret becomes the only accepted credential: JWT/Cognito verification is skipped entirely and every request maps to a single local user (DB_SYNC_LOCAL_USER_ID, default local-user). The check lives at the single auth choke point (worker/auth.cljs#auth-claims) with a constant-time comparison, so it covers HTTP, WebSocket sync and assets uniformly. The hosted Cloudflare deployment never sets this env var, so its behavior is completely unchanged.

On the client, the Sync Server settings (desktop dialog, also reachable from mobile settings) get an Access token field next to the existing custom URL. The token is stored alongside the URL, only ever sent to a custom server (never to the official service), and flows to the DB worker as a static credential that replaces the Cognito id-token — no token refresh, no login. The login gates around sync (upload menu, header indicator, mobile graph list, rtc start/restart flows) now also accept this mode, following the precedent that rtc-group? already returns true when a custom sync server is configured.

2. Zero-config token

You shouldn't need to invent a secret to sync your own notes. If the adapter starts with no Cognito configuration, it generates a token on first run, persists it at <data-dir>/local-token (mode 0600), reuses it on every restart, and prints it at startup. DB_SYNC_LOCAL_TOKEN still overrides it.

3. One-scan device pairing

Typing a 64-char token on a phone was the last painful step, so the server also prints a pairing link and a terminal QR code. The QR encodes http://<lan-ip>:<port>/pair#<token> — note the token travels in the URL fragment, so it's never sent over the network; the unauthenticated /pair page contains no secret and just turns location.origin + fragment into a logseq://sync-setup?url=…&token=… deep link client-side. The app handles that link on both desktop (electron protocol handler) and mobile (existing deeplink dispatcher) and always shows a confirmation dialog before applying anything, so a malicious link can't silently redirect someone's sync to another server.

Security considerations

I deliberately kept a shared secret instead of adding a no-auth mode: the moment a phone syncs, the server listens on the LAN, and the token is the only thing standing between "sync works" and "anyone on the Wi-Fi can read/wipe your notes". The auto-generated token keeps that protection without any setup cost.
In local mode the token is exclusive — a JWT that would otherwise verify is rejected, so there's no accidental mixed-auth surface.
Docs tell users to treat the QR/pairing link as a secret and to only expose the server on a trusted network or behind TLS.

Known limitations (intentional scope)

Local-token mode is single-user: every device acts as the same user, so member invitations/roles don't apply. That matches the "my own devices" use case this targets.
The QR//pair page is Node-adapter only (not the Cloudflare worker) — it only makes sense for self-hosting.
Possible follow-ups I left out to keep this reviewable: mDNS discovery (find the server with no URL at all) and showing a pairing QR inside the desktop app's settings.

How I tested it

pnpm test:node-adapter: 203 tests / ~4200 assertions passing, including new coverage for local-token auth (match/mismatch/missing/JWT-rejected-when-local), token generation/persistence/env-precedence.
All shadow-cljs builds (app, db-worker, db-worker-node, electron, mobile) compile with 0 warnings; clj-kondo clean on every touched file.
Live end-to-end on my LAN: server on my desktop, real devices pointed at http://192.168.x.x:8787 — wrong/missing token → 401, token → 200, graph upload/download/edit syncing across devices, /pair page and QR verified on a real terminal (which caught a fun bug: Closure advanced compilation renames .isTTY, hence the string-access fix in the last commit).

Docs are included: docs/self-hosted-sync.md (user-facing walkthrough) plus updates to deps/db-sync/README.md.

I know self-hosting has been a much-requested topic and that ADR 0001 already pointed at "pluggable auth providers" as follow-up work — I hope this is a useful step in that direction. Happy to adjust naming, split the PR, or rework any part of the approach based on your feedback. Thanks for the amazing work on the DB version! 💜

@CLAassistant

CLAassistant commented Jul 16, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@FelipeFTN FelipeFTN changed the title Local Syncing Logseq DB with local devices feature: Local Syncing Logseq DB with local devices Jul 16, 2026
FelipeFTN and others added 10 commits July 22, 2026 11:34
Add a local-token mode so the self-hosted Node sync server and the app
can sync graphs across devices with no login and no internet dependency.

Server (deps/db-sync):
- DB_SYNC_LOCAL_TOKEN env: when set, this shared secret is the only
  accepted credential; Cognito/JWT verification is skipped and requests
  map to a single local user (DB_SYNC_LOCAL_USER_ID, default local-user)
- constant-time token comparison; covered by worker-auth tests

Client:
- new Access token field in Settings -> Sync Server URL (desktop dialog,
  reused by mobile); stored in localStorage sync-server-token and only
  active together with a custom sync server URL
- token flows to the db worker as :auth/static-sync-token and replaces
  the Cognito id-token for ws connects and HTTP auth headers, skipping
  token refresh entirely
- login gates for sync (graph upload menu, header indicator, mobile
  graph list, rtc start/restart flows) also accept local-sync mode
- on startup with a local token and no login, remote graphs are fetched
  and sync starts for the current graph

docs: add docs/self-hosted-sync.md with setup instructions
- Node adapter: when no Cognito issuer is configured, generate a local
  access token on first run, persist it at <data-dir>/local-token (0600)
  and print it at startup. DB_SYNC_LOCAL_TOKEN still overrides.
- Mobile settings: sync server row shows 'Self-hosted · <host>' when a
  custom server is configured.
- docs: setup no longer requires inventing a token.
Server (node adapter, local mode):
- startup banner now prints a pairing link http://<lan-ip>:<port>/pair#<token>
  and renders it as a terminal QR code (TTY only)
- new unauthenticated /pair page: reads the token from the URL fragment
  (never sent to the server) and builds a logseq://sync-setup deep link

Client:
- new :sync-server/pair-request event: confirmation dialog, then applies
  server URL + token, pushes worker config and loads remote graphs
- logseq://sync-setup?url&token handled on mobile (deeplink) and desktop
  (electron protocol handler -> syncServerPair renderer message)

Pairing a phone is now: scan QR with the camera -> tap Open in Logseq ->
confirm.
Closure advanced compilation renames the .isTTY property access, so the
TTY check always returned undefined and the QR code was never printed.
Use string-based property access (goog.object/getValueByKeys), which
survives renaming.
…cal mode

The snapshot download handler returns a stream URL the client fetches like a
pre-signed link (no Authorization header). Cloud deployments pre-sign the URL,
but the self-hosted node adapter enforces auth, so snapshot downloads 401'd and
graph download failed at :fetch-snapshot-stream. Embed the caller's token as a
query parameter; token-from-request already accepts ?token=.
Asset PUT/GET built the Authorization header from the Cognito id-token only,
which is nil for a self-hosted (static-token) server, so asset transfers went
out unauthenticated and 401'd ("cannot find asset" on every client but the
uploader). Use sync-util/auth-token, which falls back to the static token.
Asset backfill ran only after a fresh graph download and only on electron; web
and mobile relied on a render-time one-shot request that races sync-client
startup and never retries, so an asset uploaded by another client after this
client started syncing was never pulled. Backfill missing remote assets on sync
start (idempotent; skips assets already present locally).
Creating a new synced graph invoked list-remote-graphs on a freshly-spawned
worker that had no auth token yet, so registration failed with
missing-field :auth-token and the graph silently never appeared on the server.
Push the sync auth state to the worker before the first RTC call, matching the
upload and start paths.
The self-hosted sync URL and token are read from localStorage, which a
CLI-spawned node worker does not have, so headless/CLI sync had neither
credentials nor base URL (missing-field :auth-token). Seed the sync config and
static token from LOGSEQ_SYNC_URL / LOGSEQ_SYNC_TOKEN when set, and re-pin them
in set-db-sync-config so a caller pushing the default cloud URL cannot override
them. No-op when the variables are unset; GUI behavior unchanged. Enables
headless graph upload and scripted sync.
A graph without a :logseq.kv/graph-rtc-e2ee? datom (e.g. imported or legacy)
normalized to e2ee=true, so uploading it required user RSA keys that do not
exist in a local (static-token) deployment, failing with
missing-field :user-rsa-key-pair. Default nil to false when a static sync token
is configured; unchanged for the hosted service.
@pushakargaikwad

Copy link
Copy Markdown

Thanks for this — I've been running it self-hosted (a local db-sync server on my own network, no Logseq account) across desktop (Linux + Windows), the web app, Android, and the bundled CLI. It works, but I hit six issues in local mode and fixed each.

Branch with your 4 commits rebased onto current master + my 6 fixes on top:
https://github.com/pushakargaikwad/logseq-fork/tree/local-sync-selfhost (the top 6 commits are mine). Happy to open a PR against your branch, or you can cherry-pick — whatever's easiest.

The fixes, each a focused commit:

  1. Snapshot download 401s in local mode — the snapshot handler returns a stream URL the client fetches without an Authorization header (pre-signed style), but the node adapter enforces auth, so graph download fails at :fetch-snapshot-stream. Fix: carry the token as ?token= (which token-from-request already accepts).
  2. Assets never transfer — asset PUT/GET builds auth from the Cognito id-token only, which is nil for a static-token server, so uploads/downloads 401. Fix: use sync-util/auth-token (falls back to the static token).
  3. Assets don't backfill on web/mobile — backfill runs only after a fresh graph download and only on electron; other platforms rely on a render-time one-shot that races startup and never retries, so assets from another client are never pulled. Fix: backfill missing remote assets on every sync start (idempotent).
  4. New synced graph silently never registers — creating a graph invokes list-remote-graphs on a fresh worker before auth state is pushed, giving missing-field :auth-token, and the graph never appears on the server. Fix: push auth state before the first RTC call, like the upload/start paths do.
  5. Headless / CLI sync impossible — the sync URL + token live in localStorage, which a CLI-spawned node worker doesn't have. Fix: read LOGSEQ_SYNC_URL / LOGSEQ_SYNC_TOKEN from the environment when set (no-op otherwise; GUI unchanged). Enables logseq sync upload, graph create --enable-sync, and scripted/agent sync with no GUI.
  6. Imported/legacy graphs can't upload — a graph with no :logseq.kv/graph-rtc-e2ee? datom normalizes to e2ee=true, which needs user RSA keys that don't exist in a local deployment, giving missing-field :user-rsa-key-pair. Fix: default nil to false when a static token is configured; unchanged for the hosted service.

All six are self-hosted-only paths; none change behavior against the hosted Logseq sync service. (Also rebased past the recent mldoc dependency fix on master, so that conflict with your branch is resolved.)

@FelipeFTN

Copy link
Copy Markdown
Author

Hi, buddy @pushakargaikwad! Thanks a lot for helping me testing this new feature!

I just fixed the mess with my branch caused by my dump idea of opening the PR from my master branch haha 😅
I just rebased your changes into my master branch, so they are now in this Pull Request too. I just need you to sign the CLA, so the PR pass the CI checks! 🙏

Thanks again for your time testing this, resolving the bugs and commenting on this PR.
I really want this feature working in some release of Logseq! 🔥

@pushakargaikwad

Copy link
Copy Markdown

Signed, so the CLA check is green now and CI should be unblocked. 🎉

Thanks for rebasing my six fixes in so fast! I'll keep running the branch self-hosted across desktop, web, mobile and the CLI, and I'll flag anything else that turns up. If the maintainers want changes during review, I'm happy to help address them. Really hoping this lands in a release. Great work on the core feature. 🔥

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants