feat(dev): make the dev harness able to actually deliver a webhook - #183
Conversation
`scripts/dev-api.ts` wired the webhook store and queue but ran neither of the passes that move a delivery out of the engine. Events accumulated in `event_outbox` and stopped there: registering an endpoint through the API appeared to work, and nothing was ever sent. The harness looked healthy while being structurally incapable of firing a webhook — the kind of gap only an end-to-end run finds. - `src/dev/webhook-worker.ts` runs both passes on a timer: the outbox drain fans each committed event to its matching endpoints, then the queue drain signs and POSTs them. One pass at a time, interval measured from the end of a pass, and a pass that throws is reported without stopping the loop. - The API and the worker now share one endpoint store and one queue, so a webhook registered through the API is one the worker can see. - `src/dev/inject-inbound.ts` plus a dev-only `POST /__dev/inbound` route push a synthetic message through the REAL ingest pipeline, so a local run can produce a conversation and the `conversation.message_received` event that follows it. `dev-inbound-email.ts` fakes the provider interface for tests; nothing was pumping it. - `src/dev/dev-blob-store.ts` is an in-memory `BlobStore`, so ingest runs without Supabase Storage credentials. - Shutdown stops the worker before closing the database. Verified against the running harness, not just unit-tested: injected a message, and the endpoint received a POST carrying `X-Helpthread-Event: conversation.message_received`, a `X-Helpthread-Signature` over the exact body, and the real conversation and thread ids. Worker reported `dispatched 1, delivered 1, failed 0`. Full suite: 1762 tests passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe development API now supports synthetic inbound messages, in-memory blob storage, shared webhook queueing, and background webhook delivery. The worker supports scheduled passes, error reporting, idempotent shutdown, and delivery tests. ChangesDevelopment inbound and webhook runtime
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant DevClient
participant DevAPI
participant IngestPipeline
participant WebhookQueue
participant WebhookWorker
DevClient->>DevAPI: POST /__dev/inbound
DevAPI->>IngestPipeline: injectInboundMessage
IngestPipeline->>WebhookQueue: enqueue webhook delivery
WebhookWorker->>WebhookQueue: drain queued deliveries
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| console.error('[dev-api] inbound injection failed', err) | ||
| res.statusCode = 500 | ||
| res.setHeader('Content-Type', 'application/json') | ||
| res.end(JSON.stringify({ error: err instanceof Error ? err.message : String(err) })) |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (4)
src/dev/webhook-worker.test.ts (3)
107-133: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAssert the retry with a deterministic clock or a bounded poll.
The test sleeps 120 ms and then asserts on
callsanderrors. Passes 2 and onward each perform a real outbound delivery attempt, so pass timing depends on network behavior. Replace the fixed sleep with a poll that waits untilcalls > 1, so a loaded CI runner cannot fail the test.♻️ Proposed change
- await new Promise((resolve) => setTimeout(resolve, 120)) + await vi.waitFor(() => { + expect(calls).toBeGreaterThan(1) + })Add
vito thevitestimport if you apply this.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/dev/webhook-worker.test.ts` around lines 107 - 133, Replace the fixed 120 ms sleep in the “keeps looping after a pass throws” test with a bounded poll that waits until calls exceeds 1, using the existing Vitest utilities and adding vi to the vitest import if needed. Keep the errors length assertion and enforce a timeout so the test remains deterministic without waiting on outbound delivery timing.
97-105: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThis test does not exercise the in-flight wait.
stop()runs before the first timer fires, so no pass ever starts andinFlightis still the initial resolved promise. The test verifies idempotentstop()only, not the title's "leaving no pass in flight". Add a wait longer thanintervalMsbeforestop(), or start a pass withrunOnce()and stop concurrently, to cover the wait path.💚 Proposed addition
worker = startWebhookWorker(h, { intervalMs: 10 }) + // Let at least one pass start, so `stop()` actually awaits in-flight work. + await new Promise((resolve) => setTimeout(resolve, 30)) await worker.stop()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/dev/webhook-worker.test.ts` around lines 97 - 105, Update the “stops cleanly, leaving no pass in flight” test to ensure a webhook pass is active before calling stop, either by waiting longer than the configured intervalMs or by starting runOnce() concurrently with worker.stop(). Preserve the existing second-stop assertion so the test continues covering idempotent shutdown as well as the in-flight wait path.
70-85: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAvoid real outbound requests in webhook worker tests.
This test uses
module.example.com, so delivery still performs a real DNS lookup. The route rejects127.0.0.1for the SSRF guard, so use the existing HTTP transport seam: exposerequestImplinstartWebhookWorker(), wirecreateWebhookDeliveryHandler({ ..., send: { requestImpl?: ... } }), and pass a stub from this factory.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/dev/webhook-worker.test.ts` around lines 70 - 85, Update startWebhookWorker and its delivery-handler setup to accept an optional requestImpl and pass it through createWebhookDeliveryHandler’s send configuration. In the test’s harness/factory, provide a stub request implementation so the queued webhook delivery avoids DNS and outbound network access while preserving the existing dispatched and failed assertions.src/dev/webhook-worker.ts (1)
58-79: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueType
handlersto the queue drain contract and drop the cast.
PostgresQueue.drainOnceacceptshandlers: Record<string, QueueMessageHandler<unknown>>, so the handler map can be typed by import asDrainDeps['handlers']and passed directly. The broadRecord<string, ...>cast only broadens the type instead of strengthening it.♻️ Proposed refactor
- const handlers = { + import type { DrainDeps } from '../providers/adapters/postgres-queue/index.js' + const handlers: DrainDeps['handlers'] = { [WEBHOOK_DELIVERY_TOPIC]: createWebhookDeliveryHandler({ webhookEndpoints: deps.webhookEndpoints, }), } @@ - const delivery = await deps.queue.drainOnce({ - handlers: handlers as Record< - string, - Parameters<PostgresQueue['drainOnce']>[0]['handlers'][string] - >, - }) + const delivery = await deps.queue.drainOnce({ handlers })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/dev/webhook-worker.ts` around lines 58 - 79, Type the handlers map in the worker using the queue drain contract, such as DrainDeps['handlers'], and remove the broad Record<string, ...> cast from the deps.queue.drainOnce call. Import the contract type from the existing queue module and keep the current webhook delivery handler mapping unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/dev-api.ts`:
- Around line 217-218: Secure the /__dev/inbound handler by validating the
API_TOKEN bearer header before reading the request body and returning HTTP 401
when absent or invalid. In the same handler, replace the injectInboundMessage
argument cast with runtime validation that mailboxId, from, to, subject, and
text are present with the expected shape, returning HTTP 400 for invalid JSON
data.
- Around line 273-275: Add a re-entrancy flag around the shutdown function in
the SIGINT/SIGTERM handling flow, such as the function containing
webhookWorker.stop(), and return immediately when shutdown has already started.
Set the flag before awaiting cleanup so a second signal cannot call
server.close() again, while preserving the existing cleanup order and allowing
db.close() to complete.
In `@src/dev/inject-inbound.ts`:
- Around line 43-57: Update buildRawMessage to reject any CR or LF characters in
the header values options.from, options.to, options.subject, and
options.inReplyTo before interpolating them into headers. Preserve valid message
construction and the existing optional In-Reply-To/References behavior, but fail
clearly when any value contains newline characters.
---
Nitpick comments:
In `@src/dev/webhook-worker.test.ts`:
- Around line 107-133: Replace the fixed 120 ms sleep in the “keeps looping
after a pass throws” test with a bounded poll that waits until calls exceeds 1,
using the existing Vitest utilities and adding vi to the vitest import if
needed. Keep the errors length assertion and enforce a timeout so the test
remains deterministic without waiting on outbound delivery timing.
- Around line 97-105: Update the “stops cleanly, leaving no pass in flight” test
to ensure a webhook pass is active before calling stop, either by waiting longer
than the configured intervalMs or by starting runOnce() concurrently with
worker.stop(). Preserve the existing second-stop assertion so the test continues
covering idempotent shutdown as well as the in-flight wait path.
- Around line 70-85: Update startWebhookWorker and its delivery-handler setup to
accept an optional requestImpl and pass it through
createWebhookDeliveryHandler’s send configuration. In the test’s
harness/factory, provide a stub request implementation so the queued webhook
delivery avoids DNS and outbound network access while preserving the existing
dispatched and failed assertions.
In `@src/dev/webhook-worker.ts`:
- Around line 58-79: Type the handlers map in the worker using the queue drain
contract, such as DrainDeps['handlers'], and remove the broad Record<string,
...> cast from the deps.queue.drainOnce call. Import the contract type from the
existing queue module and keep the current webhook delivery handler mapping
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: dedf8335-3919-4839-8b9b-a6abf781d9c3
📒 Files selected for processing (5)
scripts/dev-api.tssrc/dev/dev-blob-store.tssrc/dev/inject-inbound.tssrc/dev/webhook-worker.test.tssrc/dev/webhook-worker.ts
| if (req.url === '/__dev/inbound' && req.method === 'POST') { | ||
| void (async () => { |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
/__dev/inbound accepts a request with no caller check and no body check. The route is handled before apiBridge, so it inherits none of the API's input handling. Both findings share that root cause: the handler trusts the caller and trusts the parsed JSON.
scripts/dev-api.ts#L217-L218: require theAPI_TOKENbearer header before reading the body, and return HTTP 401 when it is absent or wrong. Loopback binding does not stop a cross-origintext/plainform POST from a page in the operator's browser.scripts/dev-api.ts#L220-L225: replace theas Parameters<typeof injectInboundMessage>[0]cast with a runtime shape check onmailboxId,from,to,subject, andtext, and return HTTP 400 for a bad body.
📍 Affects 1 file
scripts/dev-api.ts#L217-L218(this comment)scripts/dev-api.ts#L220-L225
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/dev-api.ts` around lines 217 - 218, Secure the /__dev/inbound handler
by validating the API_TOKEN bearer header before reading the request body and
returning HTTP 401 when absent or invalid. In the same handler, replace the
injectInboundMessage argument cast with runtime validation that mailboxId, from,
to, subject, and text are present with the expected shape, returning HTTP 400
for invalid JSON data.
| // Stop the delivery loop and let any pass in flight finish, so nothing | ||
| // is mid-query against a database that is about to close. | ||
| await webhookWorker.stop() |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Guard shutdown against a second signal.
webhookWorker.stop() is idempotent, but shutdown is not. Both SIGINT and SIGTERM call it. On a second signal, server.close() runs against an already-closed server and rejects with ERR_SERVER_NOT_RUNNING. The handlers use void shutdown() with no catch, so that becomes an unhandled rejection during shutdown, and db.close() never runs. Add a re-entrancy flag.
🛡️ Proposed guard
+ let shuttingDown = false
const shutdown = async (): Promise<void> => {
+ if (shuttingDown) return
+ shuttingDown = true
console.log('\n[dev-api] shutting down...')🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/dev-api.ts` around lines 273 - 275, Add a re-entrancy flag around the
shutdown function in the SIGINT/SIGTERM handling flow, such as the function
containing webhookWorker.stop(), and return immediately when shutdown has
already started. Set the flag before awaiting cleanup so a second signal cannot
call server.close() again, while preserving the existing cleanup order and
allowing db.close() to complete.
| export function buildRawMessage(options: InjectInboundOptions, messageId: string): string { | ||
| const headers = [ | ||
| `From: ${options.from}`, | ||
| `To: ${options.to}`, | ||
| `Subject: ${options.subject}`, | ||
| `Message-ID: <${messageId}>`, | ||
| `Date: ${new Date().toUTCString()}`, | ||
| 'MIME-Version: 1.0', | ||
| 'Content-Type: text/plain; charset=utf-8', | ||
| ] | ||
| if (options.inReplyTo !== undefined) { | ||
| headers.push(`In-Reply-To: <${options.inReplyTo}>`) | ||
| headers.push(`References: <${options.inReplyTo}>`) | ||
| } | ||
| return `${headers.join('\r\n')}\r\n\r\n${options.text}\r\n` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Reject CR/LF in header values.
buildRawMessage interpolates from, to, subject, and inReplyTo into header lines without checking for CR or LF. The dev route at scripts/dev-api.ts passes unvalidated JSON into these fields. A value that contains \r\n injects extra headers, or terminates the header block early and corrupts the body. The harness listens on loopback only, so this is a correctness problem for local runs rather than a production exposure.
🛡️ Proposed guard
+function headerValue(name: string, value: string): string {
+ if (/[\r\n]/.test(value)) {
+ throw new Error(`inject-inbound: ${name} must not contain CR or LF`)
+ }
+ return value
+}
+
export function buildRawMessage(options: InjectInboundOptions, messageId: string): string {
const headers = [
- `From: ${options.from}`,
- `To: ${options.to}`,
- `Subject: ${options.subject}`,
+ `From: ${headerValue('from', options.from)}`,
+ `To: ${headerValue('to', options.to)}`,
+ `Subject: ${headerValue('subject', options.subject)}`,
`Message-ID: <${messageId}>`,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/dev/inject-inbound.ts` around lines 43 - 57, Update buildRawMessage to
reject any CR or LF characters in the header values options.from, options.to,
options.subject, and options.inReplyTo before interpolating them into headers.
Preserve valid message construction and the existing optional
In-Reply-To/References behavior, but fail clearly when any value contains
newline characters.
Codex (in place of CodeRabbit) reviewed the harness change and found seven real defects. All are fixed. Every sacred-invariant category came back clean with evidence — the production import graph cannot reach src/dev/**, vercel.json's rewrites never route /__dev/*, the dev route cannot shadow an /api/v1 path, no changed path sends mail or touches draft state, and the fixed dev secrets stay inside scripts/dev-api.ts. Worker: - runOnce() is now serialized with the scheduled loop and tracked by stop(). It was untracked, so an explicit pass could overlap a scheduled one, and stop() could resolve while it was still querying — with the harness closing the database immediately after. It also now refuses to start after stop(). - Observer failures can no longer kill the loop. An onError that threw left a rejected promise with no handler, which on Node's defaults takes the process down; an async onActivity rejection was dropped entirely rather than reported. Both now route through one guarded reporter. - The handler map is typed against the queue's own DrainDeps instead of cast into it, so the payload narrowing is visible rather than silenced. Injector: - Header values carrying CR or LF are refused. A bare newline in a subject or address ends the header — two end the header block — so an unescaped one silently rewrote the message around it. - providerMessageId is caller-suppliable. Regenerating it unconditionally meant a retried POST created a second conversation and a second webhook, and made dedup unreachable from the harness. Route: - POST /__dev/inbound now requires the same Bearer token every other route in the harness does. Loopback-only, so not a production exposure, but the one route that fabricates customer mail should not be the exception. Tests: the four the reviewer called unfalsifiable are replaced. Delivery now asserts an actual signed POST through an injected transport rather than a counter that also passes when delivery dies at DNS. Added: stop waits for an in-flight pass, no pass runs after stop, runOnce rejects after stop, passes never overlap under a race, the loop survives a throwing onError and a rejecting onActivity, and CRLF refusal per header. 10 worker tests, 6 injector tests. Full suite: 1776 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review adjudicationCodex (adversarial, in place of CodeRabbit — not installed on this org): 7 findings — all 7 real, all 7 fixed. None wrong.
The tests that couldn't failThe delivery test asserted Added, each capable of failing: 10 worker tests, 6 injector tests. Sacred invariants — clean, with evidenceThe reviewer was asked to break four things and could not:
Gatestypecheck clean, biome clean, 1776 tests passing. |
🟡 NEEDS YOUR DECISION
Dev-tooling only — no engine behaviour changes. Gates green locally (typecheck clean, biome clean, 1762 tests pass). No bot has reviewed yet; this PR opens before review by protocol.
The bug this fixes
scripts/dev-api.tswired the webhook store and queue but ran neither of the passes that move a delivery out of the engine. Events accumulated inevent_outboxand stopped there.The effect: registering a webhook endpoint through the dev API returned 201 and looked completely healthy, while the harness was structurally incapable of ever sending one. Nothing failed — nothing happened. This was found by trying to run a module against it end to end, not by reading the code.
What's added
src/dev/webhook-worker.ts— runs both passes on a timer. The outbox drain fans each committed event out to its matching endpoints; the queue drain signs and POSTs them. One pass at a time, interval measured from the end of a pass so a slow delivery cannot stack ticks, and a pass that throws is reported without killing the loop.src/dev/inject-inbound.ts+POST /__dev/inbound— pushes a synthetic message through the real ingest pipeline, so a local run can produce a conversation and theconversation.message_receivedevent that follows.dev-inbound-email.tsfakes the provider interface for tests, but nothing in the harness was pumping it.src/dev/dev-blob-store.ts— in-memoryBlobStore, so ingest runs without Supabase Storage credentials.Verified against the running harness
Not just unit-tested. Booted the dev API, registered an endpoint, injected a message, and the endpoint received:
Worker logged
dispatched 1, delivered 1, failed 0.Decision provenance
POST /__dev/inboundroute that fakes an incoming email/__dev/and handled before the API bridge, so it is visibly not part of the engine's API surface. It exists only inscripts/dev-api.ts, which never shipsNote on the SSRF guard
src/webhooks/ssrf.tsrefuses loopback and RFC 1918 with no escape hatch, so a locally-running module cannot receive deliveries from a locally-running engine without a public hostname in between. That is correct behaviour and this PR does not touch it — but it is worth knowing before anyone else tries a fully-local module run and concludes the harness is broken.Summary by CodeRabbit
New Features
Bug Fixes
Tests