Repository navigation
feat(transcript): durable upload with retry + async read/gzip (v0.6.0) - #48
Merged
Merged
Conversation
Transcript uploads were fire-and-forget with no persistence: any failure
(network blip, 5xx, process exit before fetch completed) silently dropped
the transcript forever. This change makes the upload durable and moves
the heavy I/O off the event loop, without making the Stop hook wait.
logger.js:
- New logs/transcripts/pending/ directory.
- stageTranscriptForUpload: sanitize the JSONL and write a snapshot to
pending/<transcript_id>. Long sessions re-stage on every Stop,
overwriting the previous snapshot so the pending copy always holds
the latest content.
- uploadPendingTranscript: reads the pending snapshot, gzips, POSTs.
On 2xx the pending file is unlinked; on any failure it is left for
the next session to retry. Fire-and-forget preserved.
- uploadPendingTranscript is now async and uses fs/promises.readFile +
promisified zlib.gzip, so the Stop hook no longer blocks the event
loop while compressing multi-MB transcripts.
- retryFailedTranscripts: scans pending/ on SessionStart and fires an
upload for every leftover. Added to logger.js exports.
session_start.js:
- SessionStart now calls retryFailedTranscripts() alongside the
existing retryFailedLogs() call when the user has opted in.
plugin.json: 0.5.1 -> 0.6.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Caution Review failedPull request was closed or merged during review WalkthroughPlugin version incremented to 0.6.0. Transcript upload refactored to use two-step staging pipeline: Changes
Sequence DiagramsequenceDiagram
participant Hook as Session/Hook
participant Stage as Staging Layer
participant FS as Pending Directory
participant Upload as Upload Service
participant HTTP as HTTP Endpoint
Hook->>Stage: stageTranscriptForUpload(path)
Stage->>FS: Create sanitized snapshot
FS-->>Stage: Pending path
Stage-->>Hook: Return pending path
Hook->>Upload: uploadPendingTranscript(pendingPath)
Upload->>FS: Read staged file
FS-->>Upload: File contents
Upload->>Upload: gzipAsync compress
Upload->>HTTP: POST compressed data
alt Upload Success (2xx)
HTTP-->>Upload: Success response
Upload->>FS: Delete pending file
FS-->>Upload: Deleted
else Upload Failure
HTTP-->>Upload: Error response
Upload->>FS: Retain pending file
FS-->>Upload: File retained
Upload-->>Hook: Log error, will retry
end
Hook->>Hook: Session restart
Hook->>Upload: retryFailedTranscripts()
Upload->>FS: Scan pending directory
FS-->>Upload: List of pending files
Upload->>Upload: Retry uploads for each file
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
jhunlee-dev
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Transcript uploads are the most critical payload this plugin ships — but they were also the fragile piece. Before this PR:
transferTranscriptwas fire-and-forget with no persistence. Any failure (transient network, 5xx from ingest, process tear-down before the fetch resolved) silently lost the transcript forever.fs.readFileSync+zlib.gzipSyncon a multi-MB JSONL) blocked the hook's event loop for tens to hundreds of milliseconds on every Stop.This PR keeps fire-and-forget (we don't want to block the user's next turn) but makes the transfer durable and moves the heavy I/O off the event loop.
Behavior changes
Durable upload with retry. New
logs/transcripts/pending/directory.stageTranscriptForUploadsanitizes the JSONL and writes the result topending/<transcript_id>before any network attempt. Long sessions re-stage on every Stop, overwriting the previous snapshot so pending always holds the latest content.uploadPendingTranscriptreads pending, gzips, POSTs. On 2xx -> unlink pending. On any failure -> pending stays, log line sayskept pending for next session.retryFailedTranscripts()runs onSessionStart(next to the existingretryFailedLogs) and fires uploads for every pending file.Async I/O.
uploadPendingTranscriptnowawaitsfs/promises.readFileand a promisifiedzlib.gzip, so the Stop hook stops blocking its event loop while compressing. Call sites still don'tawaitthe function, preserving the fire-and-forget contract the user asked for (no blocking their next turn on network).Version bump.
0.5.1->0.6.0.Failure scenarios, before vs after
Trade-offs / known limitations (intentionally not addressed)
X-Transcript-ID.Happy to tackle streaming + size cap in a follow-up PR.
Test plan
Transcript transferred: <uuid>.jsonl, pending dir empty.kept pending for next session; pending file exists on disk.Retrying N pending transcript(s)appears at SessionStart; successful upload removes the pending file.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores