-
Notifications
You must be signed in to change notification settings - Fork 0
HT-9: platform provider interfaces #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # `src/providers` — platform provider interfaces | ||
|
|
||
| This directory is the seam CHARTER.md §4 promises: **"the engine's core never | ||
| calls a platform directly: queueing, scheduled and durable work, blob | ||
| storage, and inbound email all sit behind thin provider interfaces the | ||
| project owns, with today's implementations (Vercel Queues, Vercel Cron and | ||
| Workflows, Supabase Storage, Gmail push) as adapters"** rather than | ||
| assumptions baked into engine code. | ||
|
|
||
| ## The rule | ||
|
|
||
| **Engine core imports only from `src/providers`** — the interfaces and types | ||
| defined in this directory — never a platform SDK (`@vercel/*`, | ||
| `@supabase/*`, `googleapis`, etc.) directly. If an engine module needs to | ||
| enqueue work, schedule an action, store a blob, or read an inbound email, it | ||
| takes a dependency on the relevant interface (`QueueProvider`, | ||
| `SchedulerProvider`, `BlobStore`, `InboundEmailProvider`) — never on the | ||
| package that implements it. | ||
|
|
||
| Concrete implementations — **adapters** — live in `src/providers/adapters/<name>/` | ||
| (e.g. `src/providers/adapters/vercel-queues/`). This task defines the | ||
| contracts only; no adapters are built here. | ||
|
|
||
| An adapter is **selected at the composition root** — the small amount of | ||
| top-level wiring code (API route handlers, cron entry points, app | ||
| bootstrap) that constructs concrete provider instances from env/config and | ||
| hands them to engine modules. Engine modules never `import` an adapter | ||
| themselves; they only ever see the interface type. This keeps the | ||
| dependency arrow pointing one way: adapters depend on the interfaces the | ||
| engine defines, not the other way around. | ||
|
|
||
| ## Vercel-first, not Vercel-only | ||
|
|
||
| Per CHARTER.md §4, the first-class deployment target is Vercel + Supabase, | ||
| and the first adapters built against these interfaces will target Vercel | ||
| Queues, Vercel Cron/Workflows, Supabase Storage, and Gmail push. But because | ||
| engine code only ever depends on the interfaces in this directory, a future | ||
| plain-Node-plus-Postgres deployment (or any other platform) stays reachable | ||
| by writing new adapters — not by rewriting the engine. Inbound email forces | ||
| this discipline regardless, since Gmail can't be the only supported mailbox | ||
| forever; this directory applies the same discipline to the other three | ||
| seams the charter names. | ||
|
|
||
| ## Testing payoff | ||
|
|
||
| Because engine code depends on these interfaces rather than concrete SDKs, | ||
| the engine's test suite runs against **in-memory fakes** of `QueueProvider`, | ||
| `SchedulerProvider`, `BlobStore`, and `InboundEmailProvider` — no cloud | ||
| account, network call, or platform emulator required to exercise queueing, | ||
| scheduling, storage, or inbound-mail logic. Adapters get their own | ||
| integration tests against the real platform; engine logic does not need | ||
| those to run. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| /** | ||
| * `BlobStore` — the seam for attachment (and other binary object) storage. | ||
| * | ||
| * See `src/providers/README.md` for the pattern this fits into. First | ||
| * adapter target (CHARTER.md §4): Supabase Storage. | ||
| * | ||
| * ## Key namespacing | ||
| * | ||
| * Callers are responsible for choosing `key` such that it is namespaced | ||
| * per-tenant and, where applicable, per-conversation (e.g. | ||
| * `<tenantId>/<conversationId>/<attachmentId>/<filename>`). `BlobStore` | ||
| * implementations do not enforce or interpret key structure — they treat | ||
| * `key` as an opaque string — but callers MUST namespace keys themselves | ||
| * so that one tenant's or conversation's objects cannot collide with, or | ||
| * be enumerated from, another's. | ||
| * | ||
| * ## Objects are never public | ||
| * | ||
| * A `BlobStore` never exposes objects at a stable public URL. The only | ||
| * read path is `getSignedUrl`, which mints a time-limited URL for one | ||
| * object. There is no method to make an object public, and adapters MUST | ||
| * NOT configure their underlying bucket/container for public read access. | ||
| */ | ||
| export interface BlobStore { | ||
| /** | ||
| * Write `data` to `key`, creating or overwriting the object. Resolves | ||
| * once the write is durable. | ||
| */ | ||
| put( | ||
| key: string, | ||
| data: Uint8Array, | ||
| opts: { contentType: string; contentLength?: number }, | ||
| ): Promise<void>; | ||
|
|
||
| /** | ||
| * Read the full contents of the object at `key`. Rejects if no object | ||
| * exists at that key. | ||
| */ | ||
| get(key: string): Promise<Uint8Array>; | ||
|
|
||
| /** | ||
| * Mint a time-limited, signed URL for reading the object at `key`. The | ||
| * URL expires after `expiresInSeconds` and must not be usable | ||
| * afterward. This is the only way callers outside the engine (e.g. a | ||
| * browser rendering an attachment) ever read blob contents — attachments | ||
| * are served via signed URLs, never a public path. | ||
| */ | ||
| getSignedUrl(key: string, expiresInSeconds: number): Promise<string>; | ||
|
Comment on lines
+41
to
+48
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift Bound and validate signed URL lifetimes.
🤖 Prompt for AI Agents |
||
|
|
||
| /** | ||
| * Delete the object at `key`. Deleting a key that does not exist is a | ||
| * no-op, not an error. | ||
| */ | ||
| delete(key: string): Promise<void>; | ||
|
|
||
| /** Whether an object currently exists at `key`. */ | ||
| exists(key: string): Promise<boolean>; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| /** | ||
| * `InboundEmailProvider` — the seam for inbound mail arriving via provider | ||
| * webhooks. | ||
| * | ||
| * See `src/providers/README.md` for the pattern this fits into. Per | ||
| * CHARTER.md §2/§4, inbound mail arrives via **push webhooks**, never IMAP | ||
| * polling — "no daemons, no polling loops" applies to inbound mail first | ||
| * and foremost. This is where Gmail-push-via-Pub/Sub plugs in today, and | ||
| * where later providers (Postmark inbound, SES inbound, etc.) plug in | ||
| * without the engine changing: regardless of provider, the engine only | ||
| * ever sees a `NormalizedInboundEmail`. | ||
| */ | ||
|
|
||
| /** A normalized attachment reference. Bytes live in the `BlobStore`, not inline. */ | ||
| export interface NormalizedInboundAttachment { | ||
| filename: string; | ||
| contentType: string; | ||
| /** Size in bytes. */ | ||
| size: number; | ||
| /** | ||
| * Key into a `BlobStore` where the attachment's bytes have already been | ||
| * written by the provider adapter. Attachments are never carried inline | ||
| * in `NormalizedInboundEmail` — the adapter is responsible for writing | ||
| * bytes to the `BlobStore` (with a correctly tenant/conversation- | ||
| * namespaced key, per `BlobStore`'s key-namespacing contract) before | ||
| * producing this reference. | ||
| */ | ||
| contentRef: string; | ||
| } | ||
|
|
||
| /** | ||
| * The provider-agnostic shape the engine consumes for every inbound | ||
| * email, regardless of which provider webhook produced it. | ||
| * | ||
| * `inReplyTo` and `references` are carried through unmodified from the | ||
| * inbound message's headers for the threading engine to consume — this | ||
| * interface only normalizes and transports them; it does not interpret | ||
| * them. Per CHARTER.md §2 ("Threading authority lives on the outbound | ||
| * side"), these inbound headers are not trusted as the authority for | ||
| * threading — the engine's outbound-Message-ID signed-reply-token scheme | ||
| * is. See HT-8 for that spec; this type does not re-specify it. | ||
| */ | ||
| export interface NormalizedInboundEmail { | ||
| /** The `Message-ID` of the inbound message, as received. */ | ||
| messageId: string; | ||
|
|
||
| /** The `In-Reply-To` header, if present, verbatim. */ | ||
| inReplyTo?: string; | ||
|
|
||
| /** The `References` header, split into individual message-ids, verbatim order preserved. */ | ||
| references: string[]; | ||
|
|
||
| from: string; | ||
| to: string[]; | ||
| cc: string[]; | ||
| subject: string; | ||
|
|
||
| /** When the provider recorded/delivered the message (not a header-parsed date). */ | ||
| receivedAt: Date; | ||
|
|
||
| /** Plain-text body, if the message provided one. */ | ||
| text?: string; | ||
|
|
||
| /** HTML body, if the message provided one. */ | ||
| html?: string; | ||
|
|
||
| /** | ||
| * Raw headers as received, lower-cased keys, for any header the engine | ||
| * needs beyond the fields already normalized above. Multi-value headers | ||
| * are joined per the provider adapter's convention; consumers that need | ||
| * exact multi-value semantics should not rely on this bag for those | ||
| * headers. | ||
| */ | ||
| headers: Record<string, string>; | ||
|
|
||
| attachments: NormalizedInboundAttachment[]; | ||
| } | ||
|
|
||
| /** | ||
| * Provider for turning one inbound-mail provider's webhook delivery into | ||
| * the engine's normalized shape. One implementation per provider (Gmail | ||
| * push/Pub/Sub, Postmark inbound, SES inbound, ...). | ||
| */ | ||
| export interface InboundEmailProvider { | ||
| /** | ||
| * Verify that `request` is an authentic webhook delivery from this | ||
| * provider (signature/token/shared-secret check, as the provider | ||
| * requires). MUST be called — and MUST resolve `true` — before | ||
| * `parseWebhook` is trusted to run against `request`'s body; | ||
| * implementations of `parseWebhook` may assume the caller has already | ||
| * verified the request and are not required to re-verify internally. | ||
| * | ||
| * Async by contract: the first adapter (Gmail push) verifies a Google | ||
| * OIDC JWT, which may require fetching/refreshing signing certificates. | ||
| * Adapters whose check is purely synchronous simply return a resolved | ||
| * promise. | ||
| */ | ||
| verifySignature(request: Request): Promise<boolean>; | ||
|
|
||
| /** | ||
| * Parse and normalize one webhook delivery into a | ||
| * `NormalizedInboundEmail`. Rejects if the payload cannot be parsed as a | ||
| * valid message for this provider. Any attachment bytes present in the | ||
| * payload are written to a `BlobStore` by the implementation before | ||
| * this resolves, so the returned attachments carry `contentRef`s rather | ||
| * than inline bytes. | ||
| */ | ||
| parseWebhook(request: Request): Promise<NormalizedInboundEmail>; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| /** | ||
| * Barrel for the platform provider interfaces (see `src/providers/README.md`). | ||
| * Engine modules import provider types from here — never from an individual | ||
| * provider file directly, and never from a platform SDK. | ||
| */ | ||
|
|
||
| export type { | ||
| EnqueueOptions, | ||
| QueueMessage, | ||
| QueueHandlerResult, | ||
| QueueMessageHandler, | ||
| QueueProvider, | ||
| } from "./queue"; | ||
|
|
||
| export type { HandlerRef, SchedulerProvider } from "./scheduler"; | ||
|
|
||
| export type { BlobStore } from "./blob"; | ||
|
|
||
| export type { | ||
| NormalizedInboundAttachment, | ||
| NormalizedInboundEmail, | ||
| InboundEmailProvider, | ||
| } from "./inbound-email"; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| /** | ||
| * `QueueProvider` — the seam for at-least-once background work. | ||
| * | ||
| * See `src/providers/README.md` for the pattern this fits into. First | ||
| * adapter target (CHARTER.md §4): Vercel Queues. | ||
| * | ||
| * ## Delivery model | ||
| * | ||
| * Serverless queue consumers are **push-delivered**, not pulled: the | ||
| * platform invokes an HTTP handler with a queued message, rather than the | ||
| * engine running a loop that polls for work (the charter's "no daemons, no | ||
| * polling loops" principle). This interface models that shape directly — | ||
| * there is no `dequeue`/`poll` method. The handler side of the contract is | ||
| * `QueueMessageHandler`, invoked by adapter glue that receives the | ||
| * platform's webhook/invocation and adapts it into a `QueueMessage`. | ||
| * | ||
| * ## At-least-once and idempotency | ||
| * | ||
| * Every implementation of this interface delivers **at least once**: a | ||
| * message may be redelivered after a successful handler run (e.g. if the | ||
| * platform's ack arrives late, or after a retry racing a late ack) and | ||
| * will be redelivered after a failed or timed-out run. Handlers MUST be | ||
| * idempotent — safe to process the same `QueueMessage.id` (or the same | ||
| * `dedupeKey`, when supplied at enqueue time) more than once without | ||
| * duplicating side effects. This interface does not — and cannot — | ||
| * guarantee exactly-once delivery. | ||
| */ | ||
|
|
||
| /** Options controlling how a message is enqueued. */ | ||
| export interface EnqueueOptions { | ||
| /** | ||
| * Delay delivery by this many seconds after enqueue. Omit or `0` for | ||
| * immediate (best-effort) delivery. | ||
| */ | ||
| delaySeconds?: number; | ||
|
|
||
| /** | ||
| * Caller-supplied idempotency key. Implementations SHOULD suppress | ||
| * duplicate enqueues that share the same `topic` and `dedupeKey` within | ||
| * the platform's dedupe window, so that a retried enqueue call (e.g. | ||
| * after a caller timeout) does not produce duplicate work. This is a | ||
| * best-effort de-duplication aid, not a substitute for idempotent | ||
| * handlers — see the at-least-once note above. | ||
| */ | ||
| dedupeKey?: string; | ||
| } | ||
|
|
||
| /** | ||
| * A message as delivered to a consumer. `T` is the payload shape the | ||
| * producer enqueued. | ||
| */ | ||
| export interface QueueMessage<T> { | ||
| /** Provider-assigned unique id for this delivery attempt's message. */ | ||
| id: string; | ||
|
|
||
| /** The topic/queue name this message was enqueued on. */ | ||
| topic: string; | ||
|
|
||
| /** The payload as originally enqueued. */ | ||
| payload: T; | ||
|
|
||
| /** | ||
| * How many times delivery of this message has been attempted, starting | ||
| * at `1` for the first delivery. Handlers can use this to implement | ||
| * their own retry-count-aware logic (e.g. escalate to dead-letter after | ||
| * N attempts) independent of what the platform's own retry policy does. | ||
| */ | ||
| attempts: number; | ||
|
Comment on lines
+52
to
+68
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win Make The module contract requires deduplication using the same 🤖 Prompt for AI Agents |
||
|
|
||
| /** When this message was originally enqueued (producer-observed time). */ | ||
| enqueuedAt: Date; | ||
| } | ||
|
|
||
| /** | ||
| * The outcome a handler returns after processing a `QueueMessage`. | ||
| * Modeled as an explicit result rather than throw/catch so that retry | ||
| * intent (and backoff) is a typed decision the handler makes, not an | ||
| * accident of which exceptions happen to propagate. | ||
| */ | ||
| export type QueueHandlerResult = | ||
| | { kind: "ack" } | ||
| | { kind: "retry"; backoffSeconds?: number } | ||
| | { kind: "deadLetter"; reason: string }; | ||
|
|
||
| /** | ||
| * Consumer contract: a handler processes one `QueueMessage` and returns a | ||
| * `QueueHandlerResult` describing what should happen next. | ||
| * | ||
| * - `ack` — processing succeeded; the message is done and will not be | ||
| * redelivered (subject to the at-least-once caveat above). | ||
| * - `retry` — processing failed in a way that should be retried; | ||
| * `backoffSeconds`, if given, is a hint for how long to wait before the | ||
| * next attempt. Omit it to defer to the provider's default backoff. | ||
| * - `deadLetter` — processing failed in a way that should NOT be retried | ||
| * (e.g. payload is permanently malformed); `reason` is recorded for | ||
| * operator visibility. | ||
| * | ||
| * A handler that throws is treated as equivalent to `retry` by adapters, | ||
| * but handlers SHOULD prefer returning `retry`/`deadLetter` explicitly so | ||
| * retry intent is visible in the return type rather than inferred from an | ||
| * uncaught exception. | ||
| */ | ||
| export type QueueMessageHandler<T> = ( | ||
| message: QueueMessage<T>, | ||
| ) => Promise<QueueHandlerResult>; | ||
|
|
||
| /** | ||
| * Provider for enqueueing at-least-once background work. See the module | ||
| * doc comment above for the delivery and idempotency model. | ||
| */ | ||
| export interface QueueProvider { | ||
| /** | ||
| * Enqueue `payload` on `topic` for later, at-least-once delivery to | ||
| * whatever consumer is registered for that topic. | ||
| * | ||
| * Enqueue itself is fire-and-forget from the caller's perspective: this | ||
| * resolves once the message is durably accepted by the provider, not | ||
| * once it has been processed. | ||
| */ | ||
| enqueue<T>( | ||
| topic: string, | ||
| payload: T, | ||
| opts?: EnqueueOptions, | ||
| ): Promise<void>; | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Keep
contentLengthconsistent with the bytes.Because
datais fully materialized,data.byteLengthis authoritative. An independent optional length can make stored metadata disagree with the actual object and withNormalizedInboundAttachment.size; reject mismatches or derive the value internally.🤖 Prompt for AI Agents