Skip to content

feat: GET /idp/account/export — self-service pod data download (#353) - #449

Merged
melvincarvalho merged 10 commits into
gh-pagesfrom
issue-353-pod-data-export
May 14, 2026
Merged

feat: GET /idp/account/export — self-service pod data download (#353)#449
melvincarvalho merged 10 commits into
gh-pagesfrom
issue-353-pod-data-export

Conversation

@melvincarvalho

@melvincarvalho melvincarvalho commented May 14, 2026

Copy link
Copy Markdown
Contributor

Summary

GET /idp/account/export — authenticated owner downloads their pod tree as a streamed tar.gz. The L0-3 backup MVP from the Credible Exit ladder (#448), and the third leg of the user-rights trio (#351 password change, #352 account delete, this).

Archive shape

jss-export/
├── manifest.json   — webId, username, email, podName, mode,
│                     createdAt, exportedAt, jssVersion
├── account.json    — allowlisted fields only (id, webId,
│                     username, email, podName, createdAt, etc.)
│                     Never carries passwordHash, passkey
│                     credentials, OIDC client secrets, or any
│                     non-allowlisted field — adding a new
│                     allowlisted field requires security review.
└── pod/...         — entire pod tree, including /private/

In single-user root pod mode (podDir = dataRoot), ROOT_POD_EXCLUDE keeps server-internal directories out of the archive: .idp/ (IDP accounts incl. passwordHash for every user, IDP signing keys, OIDC adapter state) and .private/ (pay handler's Bitcoin keypair + UTXO state — drainable). Adding a new server-managed dotfile dir at the data root WITHOUT updating this set is a security bug; the regression test pins the property "no IdP secrets in the export" against on-disk seeded files.

Authorization

  • Multi-user: caller's WebID must resolve to a local account record (findByWebId); else 403.
  • Single-user: same — caller's WebID must match the seeded single-user account; an authenticated third-party WebID (external Solid-OIDC, LWS-CID JWT, etc.) gets 403, not the operator's /private/privkey.jsonld.

Design call: include /private/privkey.jsonld?

Yes. Per the Credible Exit framing in #448, the user's secret IS theirs and must leave with them. Refusing to export it would make L4+ identity migration impossible. The endpoint is owner-authenticated; the secret never leaves the WAC perimeter to anyone but the owner.

Streaming

tar.pack → zlib.createGzip → reply. Memory stays constant regardless of pod size — a multi-GB pod doesn't OOM. Stream-error handler destroys the pipeline on mid-pack failures (so clients see an aborted transfer rather than a silently-truncated archive). Client-disconnect handler on reply.raw.on('close') short-circuits the walk so we don't keep reading every file in a multi-GB pod into a dead socket.

Rate limiting

3/min, keyed by source IP, consistent with the other /idp/ endpoints. Per-WebID keying isn't possible in the current code path: the global auth hook in src/server.js skips /idp/* (so request.webId is unset at the rate-limit phase) and @fastify/rate-limit's keyGenerator is sync, so we can't await token verification inline. Per-user keying is a follow-up that needs a preParsing hook resolving auth before the limiter runs. Operators behind a NAT: the limit is shared across all users on the source IP.

Tests

test/idp-export.test.js — 6/6 passing locally, full suite green.

  • 401 unauthenticated
  • 200 for the authenticated owner — valid tar.gz with manifest + account + pod tree
  • account.json allowlist (no passwordHash)
  • Cross-account: alice plants a uniquely-named canary file; bob's archive contains neither the canary name nor the canary body bytes
  • Single-user-root-pod denylist: .idp/accounts/*.json and .private/keypair.json synthesized on disk pre-export, asserted absent from archive
  • Single-user + --provision-keys: archive contains the on-disk /private/privkey.jsonld (Credible Exit), manifest carries username/email/createdAt parity with multi-user
  • Single-user 403: third-party HMAC token with attacker.example.com WebID gets 403, not the operator's pod

process.env.DATA_ROOT is snapshotted/restored in startServer/stopServer to prevent cross-test leakage to subsequent test files in the same run.

New dep

tar-stream@^3.2.0. Pure JS, no native bindings. Functional install footprint on Node is just tar-stream (~60K) — but the npm install also drops streamx, b4a, and the bare-* family (bare-fs, bare-os, bare-path, bare-stream, bare-url, bare-events) totaling ~4.4 MB on disk. Those bare-* packages exist for the Bare runtime and are lazy-loaded only when running outside Node — on Node they sit on disk but never load, contributing zero runtime weight. Verified install + tests run cleanly on the Termux/mobile target.

The disk footprint matters more for our mobile/embedded targets than I'd want — tracked as #450 (replace tar-stream with a leaner producer, e.g. ~200 lines of hand-rolled POSIX ustar). Out of scope here; ship the working endpoint, optimize the dep tree as a focused follow-up.

Out of scope (per #353)

  • Re-import / cross-server pod migration (separate spec problem)
  • Periodic / scheduled backups (operator concern, not per-user right)
  • Partial / per-resource selection (defer until someone asks)
  • UI button on /idp/account (endpoint first, UI follow-up)
  • Per-WebID rate-limit keying (follow-up: needs preParsing-hook auth resolution)

Closes #353. First slice of #448.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add self-service data export — download all my pod data

2 participants