Repository navigation
fix(transcript): resilient JSONL parse + disk cleanup sweep (v0.6.1) - #49
Conversation
Two quick-win gaps in the transcript collection path, both exposed by the audit after PR #48. 1. One malformed JSONL line no longer kills the whole transcript. sanitizer.js wraps JSON.parse in try/catch, drops offending lines, and writes a single stderr notice with the count. Previously a trailing partial line from a crashed writer would throw, abort stageTranscriptForUpload, and lose the entire transcript with no retry. 2. Disk usage no longer creeps forever. logger.js gains cleanupStaleFiles() which deletes events.jsonl.*.sent files and transcripts/pending/* older than 30 days. Wired into session_start.js after the retry funcs so failed retries always survive at least one more session. plugin.json: 0.6.0 -> 0.6.1 (patch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WalkthroughThe PR increments the plugin version to 0.6.1 and adds a stale file cleanup routine to the logger module that deletes event logs and transcripts exceeding 30 days. The sanitizer now handles malformed JSONL lines gracefully, and the session initialization flow invokes cleanup after retry logic. Changes
Sequence DiagramsequenceDiagram
participant SessionStart as Session Start
participant Logger as Logger Module
participant FileSystem as File System
SessionStart->>SessionStart: Telemetry enabled?
SessionStart->>SessionStart: retryFailedLogs()
SessionStart->>SessionStart: retryFailedTranscripts()
SessionStart->>Logger: cleanupStaleFiles()
Logger->>FileSystem: Identify stale files<br/>(events.jsonl.*.sent,<br/>pending transcripts > 30 days)
FileSystem-->>Logger: List of stale files
Logger->>FileSystem: Delete stale files
FileSystem-->>Logger: Deletion complete
Logger-->>SessionStart: Cleanup done
SessionStart->>SessionStart: Return true
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly Related PRs
Suggested Reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@skillmeter/scripts/session_start.js`:
- Around line 21-23: The calls to retryFailedTranscripts(), retryFailedLogs(),
and cleanupStaleFiles() create a race where cleanupStaleFiles() can delete
pending transcript files before uploadPendingTranscript() finishes; change the
sequence so cleanup runs only after retries complete by making
retryFailedTranscripts() return a Promise that awaits its internal
uploadPendingTranscript() operations and then awaiting it (e.g., await
retryFailedTranscripts()) before calling cleanupStaleFiles(); keep
retryFailedLogs() behavior but consider awaiting it in parallel (await
Promise.all([retryFailedLogs(), retryFailedTranscripts()]) then call
cleanupStaleFiles()) so stale-file cleanup cannot drop in-flight transcript
uploads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c890d0b3-e0ee-490b-af82-6e99315e5259
📒 Files selected for processing (4)
skillmeter/.claude-plugin/plugin.jsonskillmeter/scripts/logger.jsskillmeter/scripts/sanitizer.jsskillmeter/scripts/session_start.js
| retryFailedLogs(); | ||
| retryFailedTranscripts(); | ||
| cleanupStaleFiles(); |
There was a problem hiding this comment.
Avoid retry/cleanup race that can drop pending transcripts.
retryFailedTranscripts() is fire-and-forget, so calling cleanupStaleFiles() immediately after can delete stale pending files before uploadPendingTranscript() finishes opening/reading them. This can silently lose a retry attempt.
💡 Minimal safe ordering fix
process.stderr.write(`SkillMeter v${PLUGIN_VERSION} (activated)\n`);
- retryFailedLogs();
- retryFailedTranscripts();
- cleanupStaleFiles();
+ cleanupStaleFiles();
+ retryFailedLogs();
+ retryFailedTranscripts();
return true;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| retryFailedLogs(); | |
| retryFailedTranscripts(); | |
| cleanupStaleFiles(); | |
| process.stderr.write(`SkillMeter v${PLUGIN_VERSION} (activated)\n`); | |
| cleanupStaleFiles(); | |
| retryFailedLogs(); | |
| retryFailedTranscripts(); | |
| return true; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@skillmeter/scripts/session_start.js` around lines 21 - 23, The calls to
retryFailedTranscripts(), retryFailedLogs(), and cleanupStaleFiles() create a
race where cleanupStaleFiles() can delete pending transcript files before
uploadPendingTranscript() finishes; change the sequence so cleanup runs only
after retries complete by making retryFailedTranscripts() return a Promise that
awaits its internal uploadPendingTranscript() operations and then awaiting it
(e.g., await retryFailedTranscripts()) before calling cleanupStaleFiles(); keep
retryFailedLogs() behavior but consider awaiting it in parallel (await
Promise.all([retryFailedLogs(), retryFailedTranscripts()]) then call
cleanupStaleFiles()) so stale-file cleanup cannot drop in-flight transcript
uploads.
Summary
Two small-but-real gaps left over after PR #48. Both are data-integrity or disk-hygiene fixes with no product or server-side implications.
1. One bad line no longer loses the whole transcript
sanitizer.jsparsed every JSONL line viaJSON.parse(line)with no try/catch. A single partial line — easy to hit when a writer crashes mid-flush, or when embedded byte sequences confuse the tokenizer — would throw, propagate up throughstageTranscriptForUpload, which catches the error and returnsnull. Net result: the pending snapshot is never written, the transcript has no retry path, the session's transcript is lost.Now: the offending line is skipped, the rest of the transcript is sanitized and staged normally, and stderr gets one line:
2. Disk usage no longer grows forever
transferEventLogrenames uploaded event logs toevents.jsonl.<ts>.sentand never removes them. Pending transcripts behind a permanently-broken ingest accumulate too. On a laptop that's been running SkillMeter for a year with flaky upload, this is gigabytes of orphaned disk.New
cleanupStaleFiles()inlogger.jsdeletes:events.jsonl.*.sentfiles older than 30 dayslogs/transcripts/pending/older than 30 daysInvoked from
session_start.jsafter the retry funcs, so anything the current session just attempted to resend is preserved even on brief failures.mtime-based (not filename timestamp) so pending files that are still being retried (and thus get re-written on each Stop) don't get swept prematurely.One stderr line when anything gets cleaned:
Files changed
skillmeter/scripts/sanitizer.jsJSON.parse; drop malformed lines with stderr noticeskillmeter/scripts/logger.jscleanupStaleFiles()helper; export itskillmeter/scripts/session_start.jscleanupStaleFiles()after retriesskillmeter/.claude-plugin/plugin.json0.6.0→0.6.1Test plan
Verified locally with two Node scripts (in this PR's commit history):
cwdhashed, stderr reports 1 dropped. ✅.sentfile, one aged pending file, and one recent counterpart of each. Only the aged pair deleted. ✅Regression: normal Stop → upload → pending removed path untouched (no changes to hot upload code).
Deferred audit items (tracked for a follow-up PR)
🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Chores