Skip to content

feat: unified gitleaks sanitizer + wire redaction across all upload paths; dedup shared helpers - #91

Merged
minarc merged 3 commits into
mainfrom
feat/unified-gitleaks-sanitizer
Jul 9, 2026
Merged

minarc merged 3 commits into
mainfrom
feat/unified-gitleaks-sanitizer

Conversation

@minarc

@minarc minarc commented Jul 9, 2026 •

Copy link
Copy Markdown
Contributor

What & why

Previously secret/PII redaction ran in exactly one place (the SessionStart harness block), so transcripts, prompts, assistant messages, and tool outputs shipped raw. A scan of a real 20k-line transcript found the OS username ($HOME path) in ~80% of lines, plus emails and tokens in the clear — while plugin.json claimed "secrets redacted, paths hashed". This PR unifies detection on a gitleaks-derived rule table and routes every upload path through one sanitization boundary, then consolidates the duplicated helper code the audit surfaced.

Commits (3, each independently green)

1. refactor: consolidate file I/O + JWT-expiry helpers into lib/io.js

  • New leaf module lib/io.js (safeReadJson, atomicWriteJson, readStdinJson, findGitRoot) — Node built-ins only, no import cycles.
  • ~8 read+JSON.parse+swallow copies, the atomic writer, stdin reading, and git-root discovery now delegate to it.
  • settings.saveTelemetryOptIn writes atomically; two divergent JWT-expiry checks collapse into one parametrized jwt.isJwtExpired(token, {skewSeconds, treatMissingAsExpired}) (semantics preserved).

2. refactor: consolidate Bearer-token POST into lib/http.js

  • Three drifted copies of the activate/refresh/exchange fetch skeleton → http.postBearerJson; each caller keeps its own status-code branching and timeout.

3. feat: unify secret/PII sanitizer on gitleaks rules and wire it across all upload paths

  • lib/rules.js: single gitleaks-derived rule table (MIT — see NOTICE) with keyword pre-filter, Shannon-entropy gate, stopword allowlist; replaces the two hand-rolled Tier-1/Tier-2 arrays.
  • lib/sanitize.js: one redactString loop; scrubDeep walks values and keys, HMAC-hashes path-bearing keys (incl. nested) and the $HOME/username prefix; containsTier1→containsSecret; POLICY_VERSION 2.0.0.
  • logger.js: central sanitizeEventData catch-all over every hook payload + per-event _sanitization tally; fixed fields win key collisions.
  • Transcript pipeline (sanitizeLine) scrubs the whole line, not just cwd. Tool hooks return raw fields; the central boundary hashes paths + redacts content (command is content-scrubbed so its shape is preserved).
  • Fix: anchored the auth key-pattern so author/authored_by aren't over-redacted. Bounded regex spans + memoized home-dir hash for hot-path cost. README/plugin.json privacy claims updated; bump to 0.21.0.

Verification

  • node --test → 49/49 pass (new test/sanitize.test.js, test/lib.test.js).
  • Real 20,646-line transcript scrub → 0 $HOME / email / token hits, all lines valid JSON, ~1.4s (off the hot path).
  • Live hook smoke: secret→[REDACTED_SECRET], email→[EMAIL], nested file_path→HMAC, command shape preserved.
  • claude plugin validate skillmeter ✔. Each of the 3 commits loads independently.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Improved privacy protection for prompts, transcripts, and telemetry by automatically redacting secrets/emails and hashing sensitive path data.
    • Added clearer opt-in telemetry handling and updated project metadata to the latest release version.
  • Bug Fixes

    • Prevented unredacted transcript uploads when privacy salt data is unavailable.
    • Made login and license-related requests more reliable with shared request handling and safer JSON reading.
  • Documentation

    • Updated privacy and harness guidance to better explain what is redacted and how sensitive data is handled.

minarc and others added 3 commits July 9, 2026 13:23
Extract the duplicated `read + JSON.parse + swallow` readers, the atomic
JSON writer, stdin reading, and git-root discovery into a new leaf module
(lib/io.js, Node built-ins only, so any module can import it without cycles),
and collapse the two divergent JWT-expiry checks into one parametrized helper.

- lib/io.js: safeReadJson / atomicWriteJson / readStdinJson / findGitRoot
- credstore, settings, paths, repo-scope, user_prompt_expansion_signin delegate
- settings.saveTelemetryOptIn now writes atomically (0o600)
- jwt.isJwtExpired(token, {skewSeconds, treatMissingAsExpired}); credstore's
  isLicenseTokenExpired delegates (semantics preserved: missing token -> expired,
  5-min skew)
- test/lib.test.js locks io + expiry behavior

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoEUHYQTz1Tx61FcteE979
The activate/refresh/exchange calls shared an identical fetch skeleton
(Bearer header + JSON body + AbortSignal timeout) with three copies that had
drifted only in timeout and status handling. Extract the skeleton into
lib/http.js#postBearerJson; each caller keeps its own status-code branching
and body parsing (return-null-with-log vs throw), which genuinely differ.

- lib/http.js: postBearerJson(url, bearer, body, {timeoutMs})
- license-activation refreshExpiredJwt / trySilentGhActivate and signin
  exchangeForLicense delegate the fetch; per-status behavior unchanged

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoEUHYQTz1Tx61FcteE979
… all upload paths

Previously secret/PII redaction ran in exactly one place (the SessionStart
harness block), so transcripts, prompts, assistant messages, and tool outputs
shipped raw. A scan of a real transcript found the OS username ($HOME path) in
~80% of lines plus emails and tokens in the clear. This unifies detection and
routes every upload path through one boundary.

- lib/rules.js: single gitleaks-derived rule table (MIT, see NOTICE) with
  keyword pre-filter, Shannon-entropy gate, and stopword allowlist; replaces the
  two hand-rolled Tier-1/Tier-2 detector arrays
- lib/sanitize.js: one redactString loop; scrubDeep walks values AND keys, HMAC-
  hashes path-bearing keys (incl. nested) and the $HOME/username prefix;
  containsTier1 -> containsSecret; POLICY_VERSION 2.0.0
- logger.js: central sanitizeEventData catch-all over every hook payload, with a
  per-event _sanitization tally; fixed fields win key collisions
- transcript pipeline (sanitizeLine) now scrubs the whole line, not just cwd
- tool hooks return raw fields; central boundary hashes paths + redacts content
  (command is scrubbed for content so its shape is preserved, not wholesale-hashed)
- fix: anchor the auth key-pattern so author/authored_by aren't over-redacted
- regex spans bounded + home-dir hash memoized to keep the hot path cheap
- README/plugin.json privacy claims updated; bump to 0.21.0
- test/sanitize.test.js added; harness meta rename

Verified: node --test 49/49; real 20k-line transcript scrub leaves 0 $HOME/
email/token hits with valid JSON; plugin validate passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoEUHYQTz1Tx61FcteE979
@coderabbitai

coderabbitai Bot commented Jul 9, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR replaces tiered secret detection with a curated rule-table redaction engine (Gitleaks-derived), centralizes sanitization at the hook boundary via sanitizeEventData/containsSecret, extracts shared JSON/stdin/git-root I/O and HTTP POST helpers, updates credential/settings/transfer/harness code to use them, and refreshes docs and tests.

Changes

Secret/PII sanitization rewrite and centralization

Layer / File(s) Summary
Rule-table redaction engine
skillmeter/scripts/lib/rules.js, skillmeter/scripts/lib/sanitize.js, skillmeter/test/sanitize.test.js
New RULES/STOPWORDS/placeholders define secret and PII patterns; sanitize.js is rewritten around entropy/stopword-gated redaction, home-path HMAC hashing, scrubDeep, and sanitizeEventData(data, hashSalt) returning {policyVersion, secrets, pii, ids}, validated by a new test suite.
Harness fail-closed detection update
skillmeter/scripts/harness.js, skillmeter/test/harness.test.js
Harness name/plugin detection switches from containsTier1 to containsSecret, aliases findRepoRoot to imported findGitRoot, updates comments to "secret/PII" wording, and updates related test assertions.
Hook scripts centralized boundary
skillmeter/scripts/logger.js, skillmeter/scripts/permission_denied.js, skillmeter/scripts/permission_request.js, skillmeter/scripts/post_tool_use.js, skillmeter/scripts/post_tool_use_failure.js, skillmeter/scripts/pre_tool_use.js, skillmeter/scripts/session_start.js
runHook in logger.js now sanitizes eventData centrally via sanitizeEventData and reads stdin via readStdinJson; individual hook scripts pass raw tool_input/tool_response/harness instead of pre-sanitizing.
Shared I/O and JWT helpers
skillmeter/scripts/lib/io.js, skillmeter/scripts/lib/jwt.js, skillmeter/scripts/credstore.js, skillmeter/scripts/lib/paths.js, skillmeter/scripts/lib/repo-scope.js, skillmeter/scripts/lib/settings.js, skillmeter/test/lib.test.js
New lib/io.js provides safeReadJson, atomicWriteJson, readStdinJson, findGitRoot; isJwtExpired gains options; credstore.js, paths.js, repo-scope.js, settings.js adopt these helpers, dropping local implementations.
Shared HTTP helper adoption
skillmeter/scripts/lib/http.js, skillmeter/scripts/lib/license-activation.js, skillmeter/scripts/signin.js, skillmeter/scripts/user_prompt_expansion_signin.js
New postBearerJson helper centralizes bearer POST requests, replacing manual fetch calls in license activation and sign-in flows.
Unconditional transcript sanitization
skillmeter/scripts/lib/transfer.js
stageTranscriptForUpload now always sanitizes the transcript instead of staging raw content when no hash salt exists.
Docs and manifest updates
skillmeter/.claude-plugin/plugin.json, skillmeter/NOTICE, skillmeter/README.md
Version bumped to 0.21.0, new NOTICE attributes Gitleaks-derived patterns, and README privacy/harness sections describe the updated sanitizer.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

Suggested reviewers: jhunlee-dev

Poem

A rabbit hops through rules and gates,
Redacting secrets, dodging fates,
Hashing homes with HMAC glee,
One boundary now sets us free.
🐇✨ Tokens hushed, paths concealed,
All the burrow's helpers healed!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: unified Gitleaks-based redaction across upload paths plus shared helper deduplication.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/unified-gitleaks-sanitizer

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
skillmeter/scripts/lib/http.js (1)

21-31: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Add a default for the opts param.

Destructuring { timeoutMs } with no default will throw a raw TypeError if a future caller omits the options object, rather than a clear error about a missing timeout.

♻️ Proposed defensive default
-function postBearerJson(url, bearer, body, { timeoutMs }) {
+function postBearerJson(url, bearer, body, { timeoutMs } = {}) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skillmeter/scripts/lib/http.js` around lines 21 - 31, The postBearerJson
helper currently destructures { timeoutMs } without a default, so callers that
omit the options object will throw a raw TypeError. Update postBearerJson to
give the options parameter a default empty object, and keep using timeoutMs
inside the fetch/AbortSignal.timeout flow so missing opts fails gracefully
instead of during destructuring.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@skillmeter/README.md`:
- Around line 228-229: The privacy docs in README are out of sync with the
sanitizer behavior: `command` is not wholesale HMAC-hashed and `repo_root` /
`repo_remote_org` should not be grouped with path-bearing fields. Update the
documentation around the sanitizer description to match the actual handling
implemented for `command` and the repo metadata fields, using the existing
privacy/sanitization section as the place to correct the wording.

---

Nitpick comments:
In `@skillmeter/scripts/lib/http.js`:
- Around line 21-31: The postBearerJson helper currently destructures {
timeoutMs } without a default, so callers that omit the options object will
throw a raw TypeError. Update postBearerJson to give the options parameter a
default empty object, and keep using timeoutMs inside the
fetch/AbortSignal.timeout flow so missing opts fails gracefully instead of
during destructuring.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 84c550af-10d9-43d2-8026-f7706ea49ea6

📥 Commits

Reviewing files that changed from the base of the PR and between a50d085 and 59df3f5.

📒 Files selected for processing (27)
  • skillmeter/.claude-plugin/plugin.json
  • skillmeter/NOTICE
  • skillmeter/README.md
  • skillmeter/scripts/credstore.js
  • skillmeter/scripts/harness.js
  • skillmeter/scripts/lib/http.js
  • skillmeter/scripts/lib/io.js
  • skillmeter/scripts/lib/jwt.js
  • skillmeter/scripts/lib/license-activation.js
  • skillmeter/scripts/lib/paths.js
  • skillmeter/scripts/lib/repo-scope.js
  • skillmeter/scripts/lib/rules.js
  • skillmeter/scripts/lib/sanitize.js
  • skillmeter/scripts/lib/settings.js
  • skillmeter/scripts/lib/transfer.js
  • skillmeter/scripts/logger.js
  • skillmeter/scripts/permission_denied.js
  • skillmeter/scripts/permission_request.js
  • skillmeter/scripts/post_tool_use.js
  • skillmeter/scripts/post_tool_use_failure.js
  • skillmeter/scripts/pre_tool_use.js
  • skillmeter/scripts/session_start.js
  • skillmeter/scripts/signin.js
  • skillmeter/scripts/user_prompt_expansion_signin.js
  • skillmeter/test/harness.test.js
  • skillmeter/test/lib.test.js
  • skillmeter/test/sanitize.test.js

Comment thread skillmeter/README.md
Comment on lines +228 to +229
- **Path-bearing tool fields** (`file_path`, `path`, `command`, …) and the
`cwd` / `repo_root` / `repo_remote_org` fields are HMAC-hashed wholesale.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Align the privacy docs with the sanitizer behavior: command is scrubbed as content, not wholesale-hashed, and repo_root / repo_remote_org shouldn’t be listed with the path-bearing fields.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skillmeter/README.md` around lines 228 - 229, The privacy docs in README are
out of sync with the sanitizer behavior: `command` is not wholesale HMAC-hashed
and `repo_root` / `repo_remote_org` should not be grouped with path-bearing
fields. Update the documentation around the sanitizer description to match the
actual handling implemented for `command` and the repo metadata fields, using
the existing privacy/sanitization section as the place to correct the wording.

@minarc
minarc requested a review from TheStarkor July 9, 2026 05:08
@minarc
minarc merged commit 3e3ef6b into main Jul 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants