Skip to content

Commit 2b91a26

Browse files
notifications (seam-forcer), terminal, pay wall-report, serve demo, NOTES
- notifications/: solid-0.1 parity via fs.watch on config-supplied podsRoot; subscription auth by loopback HEAD with the subscriber's own credentials — proven against real WAC acls. Forces the top candidate seams: api.events, api.serverInfo, gated response-header hooks. - terminal/: authenticated ws shell, stricter than core (mandatory access control, minimal child env, confined cwd, no orphans). New finding: ws.route cannot refuse a handshake pre-upgrade. - pay/: the wall-report — pipeline-modifying features (LDP 402 from inside the WAC hook) are core, not plugins; demo shows core's 402 shape on plugin-owned routes. Draws the route-owning vs pipeline-modifying line for #564. - NOTES.md: consolidated findings; serve.js: everything-at-once demo.
1 parent 156671d commit 2b91a26

16 files changed

Lines changed: 2389 additions & 2 deletions

File tree

NOTES.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Findings log
2+
3+
What the 0.0.215 plugin api gave us, what it didn't, and what that implies
4+
upstream. One section per theme; port-specific detail lives in each port's
5+
README. **Nothing here has been filed as an issue yet** — proposals below
6+
are candidates, each with a consumer in this repo attached.
7+
8+
## What just worked (no changes wanted)
9+
10+
- **`api.ws.route` carried every realtime port** — relay, webrtc, terminal,
11+
tunnel control, notifications — with zero upgrade-handling code and no
12+
`@fastify/websocket` dependency in any plugin. The decision to build it
13+
on the host's websocket stack (PR #589) rather than raw `'upgrade'`
14+
listeners meant five websocket features coexist in one process without a
15+
single conflict.
16+
- **`api.storage.pluginDir()`** — relay grew persistence core never had, in
17+
~15 lines.
18+
- **Fail-loudly activation** — several test suites rely on
19+
`assert.rejects(listen)` for misconfiguration; the contract reads well
20+
from the consumer side.
21+
- **The loopback pattern** (notifications): a plugin that needs "would the
22+
server allow X?" can ask the server itself — HTTP to the host with the
23+
client's own credentials. Slower than an internal check but definitionally
24+
correct. This removes a whole class of would-be seams (`api.wac.check`)
25+
from the *necessary* list, leaving them merely *nice*.
26+
27+
## Candidate seams (in value order, consumers attached)
28+
29+
1. **`api.events.onResourceChange(cb)`** — consumer: notifications/.
30+
Core has the emitter internally (`src/notifications/events.js`); today a
31+
plugin must fs.watch a config-supplied path, which drifts and misses
32+
non-fs backends. This is also the seam any future "react to pod writes"
33+
app (webhooks, indexing, sync) wants — likely the most demanded seam of
34+
the next wave of real apps.
35+
2. **`api.serverInfo` (`{ baseUrl, port }` resolved at listen)** — consumers:
36+
notifications/ (pub URLs, origin checks, loopback), any plugin minting
37+
absolute URLs. Today the operator repeats the origin in every plugin's
38+
config.
39+
3. **Internal utility modules plugins re-vendor** — consumers: relay/
40+
(`src/nostr/event.js` NIP-01 verify) and potentially pay/ (`src/mrc20.js`).
41+
Both are pure, dependency-light crypto. Candidate: export like auth.js
42+
(`javascript-solid-server/nostr.js`), or bless vendoring as the answer.
43+
4. **Response-header injection on core routes** — consumer: notifications/
44+
(`Updates-Via` discovery). Explicitly NOT proposing a default-on hook:
45+
a plugin rewriting every response is a bigger grant than route ownership.
46+
If it ships, gate it (`capabilities: ['hooks']`).
47+
48+
## The wall (by design): pipeline-modifying features
49+
50+
pay/ documents it fully. Core's pay feature turns *LDP routes* into paid
51+
resources from inside the WAC hook — a plugin cannot touch routes it
52+
doesn't own, so pay/conneg/quotas/WAC are **core, not plugins**. That's the
53+
crisp line #564 needed:
54+
55+
- route-owning features → plugins (proven here: relay, webrtc, terminal,
56+
tunnel, notifications endpoint)
57+
- pipeline-modifying features → core (or a future, separately-gated hooks
58+
capability)
59+
60+
## Smaller notes
61+
62+
- A plugin owns exactly one prefix. Features with scattered paths (core
63+
notifications' `/.well-known/solid/notifications` status endpoint,
64+
tunnel's split control/traffic paths, ActivityPub's webfinger) must
65+
consolidate under one prefix or deviate from core's URLs. Candidate:
66+
`prefixes: []` (plural) if a real consumer is blocked; consolidation was
67+
fine for everything here.
68+
- Test harness dance: a plugin whose config references the server's own
69+
origin forces the port-probe-then-boot pattern (helpers.js `port` option)
70+
— same finding as api.serverInfo, visible in test setup.

helpers.js

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import os from 'node:os';
1010
import path from 'node:path';
1111
import { createServer } from 'javascript-solid-server/src/server.js';
1212

13-
function probePort() {
13+
export function probePort() {
1414
return new Promise((resolve, reject) => {
1515
const probe = net.createServer();
1616
probe.once('error', reject);
@@ -29,7 +29,10 @@ function probePort() {
2929
export async function startJss({ plugins, ...opts } = {}) {
3030
const root = opts.root ?? fs.mkdtempSync(path.join(os.tmpdir(), 'jss-plugins-test-'));
3131
delete opts.root;
32-
const port = await probePort();
32+
// A fixed port lets configs reference the server's own origin (some
33+
// plugins need it — see notifications; finding: api.serverInfo).
34+
const port = opts.port ?? await probePort();
35+
delete opts.port;
3336
const fastify = createServer({
3437
logger: false,
3538
forceCloseConnections: true,

notifications/README.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# notifications — solid-0.1 change notifications plugin
2+
3+
Out-of-tree port of JSS `src/notifications/` (the legacy SolidOS
4+
`solid-0.1` WebSocket protocol). **The deliberate seam-forcer of this
5+
repo** — chosen because core's version leans hardest on internals.
6+
7+
```js
8+
plugins: [{ module: 'notifications/plugin.js', prefix: '/.notifications',
9+
config: {
10+
podsRoot: './data', // REQUIRED (finding 1)
11+
baseUrl: 'https://pod.example', // REQUIRED (finding 2)
12+
loopbackUrl: 'http://127.0.0.1:3000', // optional (defaults to baseUrl)
13+
} }]
14+
```
15+
16+
Protocol parity: `protocol solid-0.1` greeting, `sub`/`ack`/`err … forbidden`,
17+
`pub` with ancestor-container fan-out, `unsub`, subscription limits. Status
18+
endpoint at `<prefix>/status`.
19+
20+
## Findings
21+
22+
1. **No way to learn the data root** — core watches `rootDir` handed to it
23+
by server.js; a plugin must have the operator repeat the path in config
24+
(drift risk: point it at the wrong dir and notifications silently cover
25+
nothing). Candidate seam: `api.events.onResourceChange(cb)` (best — no
26+
filesystem coupling at all, and core already has the
27+
`resourceEvents` emitter internally) or, weaker, `api.storage.serverRoot`
28+
read-only.
29+
2. **No way to learn the server's public origin** — needed for constructing
30+
`pub` URLs, refusing cross-origin subscriptions, and the loopback check.
31+
The operator repeats it in config; a plugin cannot even discover it at
32+
`activate()` time (listen hasn't happened). Candidate seam:
33+
`api.serverInfo = { baseUrl, port }` resolved at listen.
34+
3. **No response-header injection on core routes** — core advertises the
35+
websocket via an `Updates-Via` header on every LDP response. A plugin
36+
cannot add headers to routes it doesn't own; SolidOS clients relying on
37+
discovery won't find a plugin-hosted endpoint. Candidate seam: a scoped
38+
`api.hooks.onSend((request, reply) => …)`, policy question included
39+
(letting plugins touch every response is a bigger grant than the rest of
40+
the api).
41+
4. **WAC checking from outside — solved, pleasantly.** Core calls internal
42+
`checkAccess()`. The port authorizes a subscription by **asking the
43+
server itself**: a loopback `HEAD` to the resource carrying the
44+
subscriber's own `Authorization` header. Slower (one HTTP round-trip per
45+
`sub`, cached nothing), but it *cannot disagree* with the server's real
46+
policy — the test suite proves it against a real WAC `.acl`. This
47+
pattern generalizes: any plugin needing "would the server allow X?" can
48+
ask over loopback rather than needing a `api.wac.check()` seam. A seam
49+
would still be nicer (no TCP round-trip, no loopbackUrl config), but
50+
it's a *convenience* seam, not a *possibility* seam.
51+
5. **`fs.watch` parity note** — core's watcher and this one share the same
52+
blind spot (writes that bypass the filesystem, e.g. future non-fs
53+
storage backends) and the same strength (catches out-of-band edits).
54+
Only `api.events` fixes the blind spot for both.

notifications/plugin.js

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
// Solid notifications (legacy "solid-0.1" protocol) as a #206 loader plugin.
2+
//
3+
// plugins: [{ module: 'notifications/plugin.js', prefix: '/.notifications',
4+
// config: { podsRoot: './data', baseUrl: 'http://localhost:3000' } }]
5+
//
6+
// Out-of-tree port of JSS src/notifications/ (AGPL-3.0-only). This is the
7+
// deliberate seam-forcer of the experiment: core's version has two change
8+
// sources (in-process emitChange() calls from the LDP handlers, plus a
9+
// recursive fs watcher) and injects Updates-Via discovery headers into
10+
// every response. A plugin gets none of that — see README "Findings" for
11+
// the three seams this port demonstrates the need for (api.events,
12+
// api.serverInfo, response-header hooks). What it CAN do honestly:
13+
//
14+
// - watch the pod root via fs.watch (the operator passes the path in —
15+
// the plugin api has no way to learn it);
16+
// - speak the exact solid-0.1 protocol (protocol/sub/ack/err/pub, with
17+
// ancestor-container fan-out) so SolidOS clients work unchanged;
18+
// - authorize subscriptions WITHOUT internal WAC access by loopback:
19+
// a HEAD request to the resource carrying the subscriber's own
20+
// Authorization header — if the server would serve it, they may hear
21+
// about it. Slower than core's in-process checkAccess, but it can
22+
// never disagree with the server's real policy.
23+
24+
import { watch } from 'node:fs';
25+
26+
const MAX_SUBSCRIPTIONS_PER_CONNECTION = 100;
27+
const MAX_URL_LENGTH = 2048;
28+
const DEBOUNCE_MS = 100;
29+
30+
export async function activate(api) {
31+
const wsPath = api.prefix || '/.notifications';
32+
const podsRoot = api.config.podsRoot;
33+
const baseUrl = (api.config.baseUrl || '').replace(/\/$/, '');
34+
if (!podsRoot || !baseUrl) {
35+
throw new Error(
36+
'notifications plugin requires config.podsRoot and config.baseUrl — ' +
37+
'the plugin api exposes neither the data root nor the public origin (see README findings)',
38+
);
39+
}
40+
const loopback = api.config.loopbackUrl || baseUrl; // where WE can reach the server
41+
42+
const subscriptions = new Map(); // socket -> Set<url>
43+
const subscribers = new Map(); // url -> Set<socket>
44+
45+
function subscribe(socket, url) {
46+
subscriptions.get(socket)?.add(url);
47+
if (!subscribers.has(url)) subscribers.set(url, new Set());
48+
subscribers.get(url).add(socket);
49+
}
50+
51+
function unsubscribe(socket, url) {
52+
subscriptions.get(socket)?.delete(url);
53+
subscribers.get(url)?.delete(socket);
54+
if (subscribers.get(url)?.size === 0) subscribers.delete(url);
55+
}
56+
57+
function cleanup(socket) {
58+
for (const url of subscriptions.get(socket) ?? []) subscribers.get(url)?.delete(socket);
59+
subscriptions.delete(socket);
60+
}
61+
62+
function notifySubscribers(url) {
63+
for (const socket of subscribers.get(url) ?? []) {
64+
try {
65+
socket.send(`pub ${url}`);
66+
} catch { /* closing; cleaned up on close */ }
67+
}
68+
}
69+
70+
function getParentContainer(url) {
71+
try {
72+
const u = new URL(url);
73+
if (u.pathname === '/' || u.pathname === '') return null;
74+
const trimmed = u.pathname.endsWith('/') ? u.pathname.slice(0, -1) : u.pathname;
75+
const parent = trimmed.slice(0, trimmed.lastIndexOf('/') + 1);
76+
return u.origin + parent;
77+
} catch {
78+
return null;
79+
}
80+
}
81+
82+
/** pub the resource and every ancestor container (solid-0.1 semantics). */
83+
function broadcast(url) {
84+
notifySubscribers(url);
85+
const originRoot = new URL(url).origin + '/';
86+
let current = url;
87+
let container = getParentContainer(current);
88+
while (container && container !== current && container.length >= originRoot.length) {
89+
notifySubscribers(container);
90+
current = container;
91+
container = getParentContainer(current);
92+
}
93+
}
94+
95+
/**
96+
* May this subscriber hear about this URL? No internal WAC access from a
97+
* plugin, so ask the server itself: HEAD with the subscriber's own
98+
* credentials. 2xx/3xx — including 304 — means readable.
99+
*/
100+
async function canSubscribe(url, authorization) {
101+
let parsed;
102+
try {
103+
parsed = new URL(url);
104+
} catch {
105+
return false;
106+
}
107+
if (parsed.origin !== new URL(baseUrl).origin) return false; // no proxy-probing other hosts
108+
try {
109+
const res = await fetch(new URL(parsed.pathname + parsed.search, loopback), {
110+
method: 'HEAD',
111+
redirect: 'manual',
112+
headers: authorization ? { authorization } : {},
113+
});
114+
// 404 subscribes fine (watching a resource that doesn't exist yet is
115+
// legitimate — core behaves the same for non-existent paths).
116+
return res.status < 400 || res.status === 404;
117+
} catch {
118+
return false;
119+
}
120+
}
121+
122+
// ------------------------------------------------------------ watcher
123+
const debounceMap = new Map();
124+
const watcher = watch(podsRoot, { recursive: true }, (eventType, filename) => {
125+
if (!filename) return;
126+
const base = filename.split('/').pop();
127+
if (base.startsWith('.') || filename.endsWith('~') || filename.endsWith('.swp')) return;
128+
if (filename.startsWith('.')) return; // .idp, .plugins, dot-guarded trees
129+
const now = Date.now();
130+
if (now - (debounceMap.get(filename) ?? 0) < DEBOUNCE_MS) return;
131+
debounceMap.set(filename, now);
132+
if (debounceMap.size > 1000) {
133+
for (const [key, time] of debounceMap) {
134+
if (now - time > 5000) debounceMap.delete(key);
135+
}
136+
}
137+
broadcast(baseUrl + '/' + filename.replace(/\\/g, '/'));
138+
});
139+
watcher.on('error', (err) => api.log.error(`notifications: watcher error: ${err.message}`));
140+
141+
// ------------------------------------------------------------ websocket
142+
await api.ws.route(wsPath, (socket, request) => {
143+
const authorization = request.headers?.authorization ?? null;
144+
socket.send('protocol solid-0.1');
145+
subscriptions.set(socket, new Set());
146+
147+
socket.on('message', async (message) => {
148+
const msg = String(message).trim();
149+
if (msg.startsWith('sub ')) {
150+
const url = msg.slice(4).trim();
151+
if (!url) return;
152+
if (url.length > MAX_URL_LENGTH) return socket.send('error: URL too long');
153+
if ((subscriptions.get(socket)?.size ?? 0) >= MAX_SUBSCRIPTIONS_PER_CONNECTION) {
154+
return socket.send('error: Subscription limit exceeded');
155+
}
156+
if (!(await canSubscribe(url, authorization))) return socket.send(`err ${url} forbidden`);
157+
subscribe(socket, url);
158+
socket.send(`ack ${url}`);
159+
} else if (msg.startsWith('unsub ')) {
160+
const url = msg.slice(6).trim();
161+
if (url) unsubscribe(socket, url);
162+
}
163+
});
164+
socket.on('close', () => cleanup(socket));
165+
socket.on('error', () => cleanup(socket));
166+
});
167+
168+
// Status endpoint (core keeps this at /.well-known/solid/notifications —
169+
// outside any single prefix a plugin can own; ours lives under the mount).
170+
api.fastify.get(wsPath + '/status', async () => ({
171+
connections: subscriptions.size,
172+
subscriptions: [...subscriptions.values()].reduce((n, s) => n + s.size, 0),
173+
protocol: 'solid-0.1',
174+
}));
175+
176+
api.log.info(`notifications: solid-0.1 websocket at ${wsPath}, watching ${podsRoot}`);
177+
return {
178+
deactivate() {
179+
watcher.close();
180+
for (const socket of subscriptions.keys()) {
181+
try { socket.close(); } catch { /* already gone */ }
182+
}
183+
},
184+
};
185+
}

0 commit comments

Comments
 (0)