Helpthread · zaridan · Jul 16, 2026 · Jul 16, 2026 · Jul 16, 2026
@@ -64,10 +64,10 @@ Ordered, applied to each received message. Idempotent by step 1, so a whole re-r
    `appendThread` uses for outbound idempotency (sending.md §3a). A fresh insert means we
    own processing; a **conflict** means a concurrent or prior delivery already owns it, so
    we **stop and return that row's outcome** — a terminal `stored`/`suppressed` row is a
-   completed replay, an in-flight `received` row is another worker's claim (do not
-   double-process). A non-atomic read-then-insert would let two concurrent deliveries of
-   the same key both pass a dedup check and both create a conversation; the unique-key
-   claim is what closes that race.
+   completed replay, an in-flight `received` row **whose lease has not lapsed** is another
+   worker's claim (do not double-process; §4's lease). A non-atomic read-then-insert would
+   let two concurrent deliveries of the same key both pass a dedup check and both create a
+   conversation; the unique-key claim is what closes that race.
 2. **Parse.** `parseInboundEmail(raw) → ParsedEmail` (invariant #1). A message that cannot
    be parsed at all is a ledger `failed`/dead-letter case (§4), never a guess.
 3. **Loop/auto-responder gate (§5).** A suppressed message is recorded `suppressed` and
@@ -119,6 +119,56 @@ separately durable. It is the inbound mirror of the outbound get-or-insert in se
 §3a, keyed on `(mailboxId, providerMessageId)` rather than `(conversationId,
 idempotencyKey)`.
 
+**The claim carries a lease, so a crash mid-unit is reclaimed, not stranded (HT-45).** The
+previous paragraph's "the retry redoes the whole unit cleanly" only holds if a retry's
+§3-step-1 claim is actually willing to re-claim a `received` row that never made it to the
+step-5 commit — a hard process crash (SIGKILL / OOM / redeploy) between the claim committing
+`received` and that commit (or the catch-block's `markFailed`) otherwise strands the row at
+`received` forever: nothing ever transitions it to `failed`, so an ordinary re-delivery finds
+a `received` row and — correctly, per this section's own "in-flight, do not double-process"
+rule — refuses to touch it, on every subsequent redelivery, permanently. The delivery ledger's
+`claimed_until` column is what breaks that permanence: every successful claim (fresh insert,
+or a `failed`/`received` reclaim) stamps a lease `leaseMs` into the future, and a `received`
+row becomes reclaimable — by the ordinary §3-step-1 claim path, no separate sweep — once
+`claimed_until IS NULL OR claimed_until < now()`. A single row-locked `UPDATE` performs the
+reclaim, so two concurrent reclaim attempts on the same lapsed lease can never both win — the
+same atomicity this ledger already relies on for the `failed`-row reclaim and that
+`ConversationStore.claimThreadForDelivery` (sending.md) relies on for the outbound lease. The
+retry that actually performs the reclaim is whatever next calls into ingest for this key: a
+redelivered provider notification, or — since a stuck `received` row also blocks this
+mailbox's transport cursor from advancing (gmail-push.md §4) — the transport's own history
+replay re-fetching and re-ingesting the same still-un-advanced message, which recurs on every
+subsequent reconcile run for as long as the cursor cannot pass it, bounded above by that
+transport's own periodic maintenance sweep (gmail-push.md §6) even with no new mail at all.
+
+**A lease is advisory, not exclusive — so every commit is fenced.** Nothing stops a
+slow-but-still-alive owner from finishing its work and committing *after* another worker has
+already reclaimed its lapsed lease — the previous paragraph's reclaim exists precisely because
+a crashed owner is indistinguishable from a merely slow one until the lease lapses. Committing
+that late write unconditionally would reintroduce the same corruption the lease closes: two
+live owners, two commits, two conversations for one email. So `attempts` doubles as a claim
+generation: every successful claim returns the row's current `attempts`, the caller carries
+that number for as long as it processes the delivery, and every ledger-write method
+(`markStoredInTx`, `markSuppressed`, `markFailed`, `markDeadLetter`) requires that same number
+back and fences its `UPDATE` on `status = 'received' AND attempts = $claimedAttempts`. A
+`received`-lease reclaim bumps `attempts` (next paragraph), and any `markFailed`/`markDeadLetter`
+bumps it too, so a stale owner's write always matches zero rows and is rejected — the same
+optimistic-concurrency shape the outbound queue adapter (`src/providers/adapters/
+postgres-queue/index.ts`) already uses. A rejected write reports `in-progress`, never a forced
+`failed`/`dead-letter` outcome that would just collide with whichever generation now legitimately
+owns the row.
+
+**The reclaim counts toward the retry budget too.** A `received`-lease reclaim also bumps
+`attempts` — a lapsed lease is itself evidence of an abandoned attempt (the owner crashed,
+OOM'd, or otherwise never reached a recorded outcome), which is exactly what a message that
+hard-crashes the ingest process on every attempt looks like. Without this, such a message would
+retry forever: it never reaches the `failed`/`dead-letter` catch paths that are the only other
+place `attempts` increments, so `MAX_INGEST_ATTEMPTS` would never engage and the mailbox's
+reconcile cursor would stay wedged behind it permanently — the very symptom this section exists
+to close, recurring instead of stranded. The pipeline checks the post-reclaim `attempts` against
+`MAX_INGEST_ATTEMPTS` before spending another parse/store cycle on a message already proven to
+keep crashing, and dead-letters it immediately once the budget is exhausted.
+
 **At-least-once, with honest partial-failure handling.** Ingest can still fail partway —
 an unparseable message, a blob write that succeeds then a transaction that aborts, an
 `append→deleted` whose fallback-create then fails. The pipeline mirrors the outbound
@@ -222,3 +272,17 @@ engine's existing store/keyring fakes — no cloud required:
 - A verifiable own-message loop → `suppressed`, nothing created; a message that merely
   *claims* our `From` without a verifiable correlation → **ingested**, not dropped.
 - `append→deleted` → falls back to a fresh conversation, mail never lost.
+- A simulated crash (a delivery claimed, then never marked `stored`/`failed`) → while its
+  lease still holds, re-delivery reports `in-progress` and touches nothing (indistinguishable
+  from a genuinely concurrent in-flight claim); once the lease has lapsed, re-delivery
+  reclaims and fully reprocesses it — exactly one conversation, ledger ends `stored` (§4's
+  lease). Two concurrent re-deliveries of the same lapsed row → exactly one reclaim wins,
+  same as the fresh-key concurrent-claim case above.
+- A stale owner that outlives its lease and only THEN tries to commit, after another worker
+  has already reclaimed the lapsed lease → the fenced write is rejected (`LeaseLostError`),
+  reported as `in-progress`; the reclaiming worker's own commit is the one that lands, and no
+  duplicate conversation is created.
+- A message that hard-crashes the ingest process on every attempt (never reaching a recorded
+  `failed`/`dead-letter` outcome, only ever a lapsed lease) → the reclaim's own `attempts`
+  bump still exhausts `MAX_INGEST_ATTEMPTS`, converging to `dead-letter` the same as a message
+  that always throws — not retried forever.
@@ -53,6 +53,7 @@ describe('migrate', () => {
       { id: 11, name: 'gmail_watch_state' },
       { id: 12, name: 'inbound_deliveries' },
       { id: 13, name: 'queue_jobs' },
+      { id: 14, name: 'inbound_delivery_lease' },
     ])
   })
 
@@ -76,6 +77,7 @@ describe('migrate', () => {
       { id: 11 },
       { id: 12 },
       { id: 13 },
+      { id: 14 },
     ])
   })
 

@@ -679,6 +679,41 @@ CREATE UNIQUE INDEX queue_jobs_topic_dedupe_key ON queue_jobs (topic, dedupe_key
 CREATE INDEX queue_jobs_ready_idx ON queue_jobs (topic, run_after) WHERE dead_lettered_at IS NULL;
 `
 
+/**
+ * Migration 014 — `inbound_deliveries.claimed_until`, the inbound delivery
+ * lease (HT-45; `src/store/inbound-deliveries.ts`, specs/mail/inbound-
+ * ingestion.md §4).
+ *
+ * Closes the never-drop gap HT-37 shipped without: a process crash (SIGKILL
+ * / OOM / redeploy) between `InboundDeliveryStore.claim` committing
+ * `'received'` and the ingest pipeline's step-5 store transaction (or the
+ * catch-block `markFailed`) stranded the delivery at `'received'` forever —
+ * `claim()` reclaimed a `'failed'` row but had no notion of a `'received'`
+ * row's claim ever going stale, so re-delivery just replayed the same stuck
+ * `'in-progress'` outcome, and (HT-41's cursor coupling) could block the
+ * mailbox's reconcile cursor from ever advancing past it.
+ *
+ * This is the inbound mirror of migration 003's `threads.claimed_until`
+ * (the outbound send lease `ConversationStore.claimThreadForDelivery` reads
+ * and writes): a nullable lease timestamp, `NULL` or in the past meaning
+ * "free to claim." Unlike migration 003, no new index is added — every
+ * lookup here is still by the existing `(mailbox_id, provider_message_id)`
+ * unique key (`InboundDeliveryStore.claim`'s own get-or-insert), never a
+ * batch scan over `claimed_until`, so there is no query this column needs
+ * to speed up.
+ *
+ * A pre-existing `'received'` row from before this migration has
+ * `claimed_until IS NULL` — `InboundDeliveryStore.claim`'s reclaim check
+ * treats `NULL` as an already-expired lease (see that module's doc comment),
+ * so any delivery already stranded at `'received'` in production becomes
+ * immediately reclaimable on its next claim() call, not just newly-stranded
+ * ones — a deliberate, desirable side effect of the `NULL`-is-free
+ * semantics, not a special backfill case.
+ */
+const MIGRATION_014_INBOUND_DELIVERY_LEASE = `
+ALTER TABLE inbound_deliveries ADD COLUMN claimed_until timestamptz;
+`
+
 /**
  * Every migration, in the order they must apply. `id` is the sole ordering
  * key (ascending) — array position is not relied upon, so re-sorting this
@@ -746,6 +781,11 @@ const MIGRATIONS: Migration[] = [
     name: 'queue_jobs',
     sql: MIGRATION_013_QUEUE_JOBS,
   },
+  {
+    id: 14,
+    name: 'inbound_delivery_lease',
+    sql: MIGRATION_014_INBOUND_DELIVERY_LEASE,
+  },
 ]
 
 /**

@@ -306,6 +306,131 @@ describe('ingestInboundMessage', () => {
     expect(await countRows(db, 'threads')).toBe(1)
   })
 
+  // --- HT-45: a crash strands a delivery at 'received'; the lease closes it. ---
+
+  it('a delivery still within its lease reports in-progress and is NOT reprocessed (a genuinely concurrent claim, not a crash)', async () => {
+    const { db, deps, mailboxId } = await freshDeps()
+    const raw = inboundDelivery(mailboxId, 'provider-msg-1', freshCustomerRaw())
+    // Claim directly against the ledger, as a concurrent ingestInboundMessage
+    // call's own step 1 would — and never mark it, simulating that call
+    // still being genuinely in flight.
+    const stuck = await deps.inboundDeliveryStore.claim(mailboxId, 'provider-msg-1', 30_000)
+    expect(stuck.claimed).toBe(true)
+
+    const outcome = await ingestInboundMessage(raw, deps)
+
+    expect(outcome).toMatchObject({ kind: 'in-progress', deliveryId: stuck.delivery.id })
+    expect(await countRows(db, 'conversations')).toBe(0)
+  })
+
+  it("a delivery stranded at 'received' by a simulated crash (claimed, never marked) is reclaimed and reprocessed once its lease expires", async () => {
+    const { db, deps, mailboxId } = await freshDeps()
+    const raw = inboundDelivery(mailboxId, 'provider-msg-1', freshCustomerRaw())
+
+    // Simulate the crash this ticket closes: claim the delivery (exactly
+    // ingestInboundMessage's own step 1) but never run parse/store/mark —
+    // the window between claim() committing 'received' and step 5's store
+    // transaction (or the catch-block markFailed), if the process died
+    // right there.
+    const stuck = await deps.inboundDeliveryStore.claim(mailboxId, 'provider-msg-1', 30_000)
+    expect(stuck.claimed).toBe(true)
+    await db.query(
+      "UPDATE inbound_deliveries SET claimed_until = now() - interval '1 second' WHERE id = $1",
+      [stuck.delivery.id],
+    )
+
+    // Nothing has processed this message yet: no conversation exists, the
+    // ledger row is still 'received'.
+    expect(await countRows(db, 'conversations')).toBe(0)
+
+    // Re-delivery (a redelivered push notification, or the reconcile sweep
+    // re-listing the same stuck message because the cursor never advanced
+    // past it — HT-41) calls ingestInboundMessage again for the SAME key.
+    // With the lease lapsed, this must reclaim and fully reprocess it, not
+    // report 'in-progress' forever.
+    const outcome = await ingestInboundMessage(raw, deps)
+
+    expect(outcome).toMatchObject({ kind: 'stored', deliveryId: stuck.delivery.id })
+    expect(await countRows(db, 'conversations')).toBe(1)
+    expect(await countRows(db, 'threads')).toBe(1)
+
+    const ledgerRows = await db.query<{ status: string }>(
+      'SELECT status FROM inbound_deliveries WHERE id = $1',
+      [stuck.delivery.id],
+    )
+    expect(ledgerRows[0].status).toBe('stored')
+  })
+
+  it('two concurrent re-deliveries of a lease-expired stranded row resolve to exactly one conversation (the reclaim itself is claim-safe)', async () => {
+    const { db, deps, mailboxId } = await freshDeps()
+    const raw = inboundDelivery(mailboxId, 'provider-msg-1', freshCustomerRaw())
+    const stuck = await deps.inboundDeliveryStore.claim(mailboxId, 'provider-msg-1', 30_000)
+    expect(stuck.claimed).toBe(true)
+    await db.query(
+      "UPDATE inbound_deliveries SET claimed_until = now() - interval '1 second' WHERE id = $1",
+      [stuck.delivery.id],
+    )
+
+    const [a, b] = await Promise.all([
+      ingestInboundMessage(raw, deps),
+      ingestInboundMessage(raw, deps),
+    ])
+
+    expect([a.kind, b.kind]).toContain('stored')
+    expect(await countRows(db, 'conversations')).toBe(1)
+    expect(await countRows(db, 'threads')).toBe(1)
+  })
+
+  // --- HT-45 review fix (should-fix #2): a message that always crashes
+  // (never reaches a recorded failed/dead-letter outcome, only ever a lapsed
+  // lease) must still converge to dead-letter, the same as one that always
+  // throws — not retry forever. ------------------------------------------
+
+  it('a delivery whose lease keeps lapsing (simulating a crash-poison message) converges to dead-letter once the reclaim budget is exhausted', async () => {
+    const { db, deps, mailboxId } = await freshDeps()
+    const raw = inboundDelivery(mailboxId, 'provider-msg-1', freshCustomerRaw())
+    const stuck = await deps.inboundDeliveryStore.claim(mailboxId, 'provider-msg-1', 30_000)
+    expect(stuck.claimed).toBe(true)
+
+    // Simulate MAX_INGEST_ATTEMPTS - 1 prior lease-expiry reclaims (each one
+    // a crash) by setting attempts directly to what that many real reclaims
+    // would have produced, then lapsing the lease one more time — the next
+    // claim's own reclaim bumps attempts the rest of the way to the budget.
+    await db.query(
+      "UPDATE inbound_deliveries SET attempts = $2, claimed_until = now() - interval '1 second' WHERE id = $1",
+      [stuck.delivery.id, MAX_INGEST_ATTEMPTS - 1],
+    )
+
+    const outcome = await ingestInboundMessage(raw, deps)
+
+    // The reclaim's own bump already carried attempts to MAX_INGEST_ATTEMPTS
+    // (the budget check reads that post-reclaim value); markDeadLetter's
+    // unconditional `attempts = attempts + 1` (same as every other caller)
+    // then carries it one further, to MAX_INGEST_ATTEMPTS + 1 — dead-lettering
+    // is still recorded as an accumulated attempt, same as the ordinary
+    // parse/store failure path.
+    expect(outcome).toMatchObject({
+      kind: 'dead-letter',
+      deliveryId: stuck.delivery.id,
+      attempts: MAX_INGEST_ATTEMPTS + 1,
+    })
+    // Dead-lettered before ever parsing/storing — no conversation created.
+    expect(await countRows(db, 'conversations')).toBe(0)
+
+    const ledgerRows = await db.query<{ status: string; attempts: number }>(
+      'SELECT status, attempts FROM inbound_deliveries WHERE id = $1',
+      [stuck.delivery.id],
+    )
+    expect(ledgerRows[0]).toMatchObject({
+      status: 'dead-letter',
+      attempts: MAX_INGEST_ATTEMPTS + 1,
+    })
+
+    // A further re-delivery must NOT auto-retry a dead-lettered message.
+    const again = await ingestInboundMessage(raw, deps)
+    expect(again).toMatchObject({ kind: 'dead-letter', attempts: MAX_INGEST_ATTEMPTS + 1 })
+  })
+
   // --- spec §8: a partial failure → failed → retried → stored. -------------
 
   it('a partial failure in step 5 (the store+ledger transaction aborts) → failed, then a retry → stored, with no orphaned/duplicate conversation', async () => {