Repository navigation
feat(hooks): capture StopFailure, TaskCreated, PostCompact, PermissionDenied (v0.8.0) - #52
Conversation
…nDenied
Four observation-only hook events added — each filling a visible gap in
the current telemetry pipeline. All follow the existing runHook pattern
(~10 lines per script, no blocking, no new lib/ dependencies).
StopFailure
Complements Stop (happy-path turn end). Captures error_type (rate_limit,
authentication_failed, billing_error, invalid_request, server_error,
max_output_tokens, unknown) so the backend can surface friction: billing
blocks, rate limits, context exhaustion.
TaskCreated
Pairs with existing TaskCompleted. Together they let the backend compute
task lifetime by subtracting timestamps on matching task_id. Same
attribute shape as TaskCompleted for easy pair-up server-side.
PostCompact
Pairs with existing PreCompact. Together they measure compaction
duration and frequency — a proxy for "session hit the context wall".
PermissionDenied
Fires when auto-mode classifier rejects a tool call. Gives visibility
into workflow-automation friction (which tools Claude tried that got
blocked). tool_input paths get the same HMAC-hashed sanitize pass as
pre_tool_use.js. We intentionally do NOT return {retry: true} —
capture only, no behaviour change.
Zero risk: no blocking hooks, no modification of inputs/outputs, no new
dependencies. Preserves the existing fixed-endpoint + license-bypass
overrides.
plugin.json: 0.7.0 -> 0.8.0 (minor bump: new event capture capability).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WalkthroughThis pull request increments the plugin version from 0.7.0 to 0.8.0 and introduces four new event hooks (PostCompact, StopFailure, TaskCreated, PermissionDenied) via entries in the hooks manifest, each backed by a corresponding Node.js script that executes telemetry or state-tracking logic. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@skillmeter/scripts/post_compact.js`:
- Around line 6-9: The code currently passes raw free-form text via
input.custom_instructions into runHook("PostCompact"), which can leak secrets;
instead sanitize before sending: replace input.custom_instructions with a
non-content signal (e.g., has_custom_instructions boolean, a length_bucket based
on input.custom_instructions.length, and/or a stable hash/trimmed fingerprint)
so no raw text is emitted. Update the payload construction in the
runHook("PostCompact", ...) call to compute and send only those derived fields
(referencing runHook, "PostCompact", and input.custom_instructions) and ensure
any fallback on empty/null is handled; do not include the original
custom_instructions string in telemetry.
In `@skillmeter/scripts/stop_failure.js`:
- Around line 8-10: The telemetry currently ships raw provider text via
input.error_message in the runHook("StopFailure", ...) payload; remove that
free-form field and instead send only the enum error_type plus a coarse flag
(e.g., error_message_present or has_error_message) to indicate whether a message
existed (or a small bucket like message_length_bucket) to avoid exposing
sensitive content. Update the runHook payload to drop input.error_message and
add the boolean/bucket flag while retaining input.error_type.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 976dba65-63b9-4361-8aa0-585a980f557f
📒 Files selected for processing (6)
skillmeter/.claude-plugin/plugin.jsonskillmeter/hooks/hooks.jsonskillmeter/scripts/permission_denied.jsskillmeter/scripts/post_compact.jsskillmeter/scripts/stop_failure.jsskillmeter/scripts/task_created.js
| runHook("PostCompact", (input) => ({ | ||
| trigger: input.trigger, | ||
| custom_instructions: input.custom_instructions, | ||
| })).catch(() => process.exit(1)); |
There was a problem hiding this comment.
Do not log raw custom_instructions content.
Line 8 sends free-form instruction text directly to telemetry, which can contain secrets or sensitive business data. Emit a non-content signal instead (presence/length bucket/hash), not raw text.
🔧 Proposed change
runHook("PostCompact", (input) => ({
trigger: input.trigger,
- custom_instructions: input.custom_instructions,
+ custom_instructions_present: Boolean(input.custom_instructions),
})).catch(() => process.exit(1));🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@skillmeter/scripts/post_compact.js` around lines 6 - 9, The code currently
passes raw free-form text via input.custom_instructions into
runHook("PostCompact"), which can leak secrets; instead sanitize before sending:
replace input.custom_instructions with a non-content signal (e.g.,
has_custom_instructions boolean, a length_bucket based on
input.custom_instructions.length, and/or a stable hash/trimmed fingerprint) so
no raw text is emitted. Update the payload construction in the
runHook("PostCompact", ...) call to compute and send only those derived fields
(referencing runHook, "PostCompact", and input.custom_instructions) and ensure
any fallback on empty/null is handled; do not include the original
custom_instructions string in telemetry.
| runHook("StopFailure", (input) => ({ | ||
| error_type: input.error_type, | ||
| error_message: input.error_message, |
There was a problem hiding this comment.
Avoid shipping raw provider error text in telemetry.
Line 10 logs input.error_message verbatim. Free-form API error text can include sensitive request/context fragments; this is a data-minimization risk. Prefer enum-only analytics (error_type) plus a coarse flag.
🔧 Proposed change
runHook("StopFailure", (input) => ({
error_type: input.error_type,
- error_message: input.error_message,
+ error_message_present: Boolean(input.error_message),
})).catch(() => process.exit(1));🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@skillmeter/scripts/stop_failure.js` around lines 8 - 10, The telemetry
currently ships raw provider text via input.error_message in the
runHook("StopFailure", ...) payload; remove that free-form field and instead
send only the enum error_type plus a coarse flag (e.g., error_message_present or
has_error_message) to indicate whether a message existed (or a small bucket like
message_length_bucket) to avoid exposing sensitive content. Update the runHook
payload to drop input.error_message and add the boolean/bucket flag while
retaining input.error_type.
Summary
Add four observation-only hook events that fill visible gaps in the current telemetry pipeline. All non-blocking, ~10 lines per script, zero new dependencies.
Stoprate_limit,authentication_failed,billing_error,invalid_request,server_error,max_output_tokens,unknown. Surfaces customer-blocking friction (billing, auth, context exhaustion).TaskCompletedtask_id. Core dev-productivity metric.PreCompactImplementation
Each script follows the existing
runHookpattern — mirroring the shape of its sibling (stop_failure.jsmatchesstop.js,task_created.jsmatchestask_completed.js,post_compact.jsmatchespre_compact.js,permission_denied.jsmatchespermission_request.js).PermissionDeniedrunstool_inputthrough the samesanitizeToolDataHMAC path asPreToolUse. All other events carry no path/PII-sensitive fields.Intentional non-behaviour:
PermissionDeniedcould return{retry: true}to let the model retry the denied tool. We do not — this is observation-only telemetry, no decision-modification.Files
Total event count: 18 → 22.
Test plan
StopFailureevent shows up in backend logs with the correcterror_type.TaskCreate→ later complete the task → confirm bothTaskCreatedandTaskCompletedevents land with the sametask_id.PreCompact+PostCompactboth fire.PermissionDeniedevent with hashedtool_input.file_path.Deferred (low-value, per audit)
CwdChanged— supplementary to the existing repo-scope gate.UserPromptExpansion— slash-command usage; niche.PostToolBatch— parallel-batch completion; low signal.Elicitation/ElicitationResult— MCP-specific; few users.FileChanged— needs explicitmatcher; not general-purpose.Preserves the current fixed-endpoint + license-bypass overrides in
lib/jwt.jsandrunHook. No server-schema migration — just accept the 4 newhook_event_namevalues additively.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores