Skip to content

feat: implement secret scanner, redaction utility, and update docs #1313 - #1321

Merged
Shashankss1205 merged 3 commits into
CodeGraphContext:mainfrom
DakshtaMethwani:security/1313-redact-secrets
Aug 13, 2026
Merged

feat: implement secret scanner, redaction utility, and update docs #1313#1321
Shashankss1205 merged 3 commits into
CodeGraphContext:mainfrom
DakshtaMethwani:security/1313-redact-secrets

Conversation

@DakshtaMethwani

Copy link
Copy Markdown
Contributor

Pull Request Description

Overview

This PR addresses the security vulnerability outlined in Issue #1313. Previously, when CodeGraphContext (CGC) indexed a codebase, string literals containing plain-text secrets (API keys, tokens, passwords) were stored verbatim in the graph database (KuzuDB/Neo4j/FalkorDB) as node metadata. This posed a leakage risk if .cgc bundles were shared or if an LLM accessed the graph via MCP.

This implementation introduces runtime regex/entropy scanning, a warning mechanism, and an optional redaction system.

Proposed Changes

  • Secret Detection Engine: Implemented string literal scanning against common token entropy patterns and regular expressions (e.g., sk-, ghp_, Bearer, AWS keys).
  • Index-time Warnings: The system now emits a non-blocking warning to the console if potential secrets are detected during repository scanning.
  • Redaction Support: Added a new configuration flag CGC_REDACT_SECRETS. When active, any string matching a secret pattern is replaced with [REDACTED] prior to DB persistence.
  • Security Documentation: Updated the security guide and bundle export documentation to advise users on how string literals are handled.

⚙️ Configuration

CGC_REDACT_SECRETS is set via environment variable and defaults to false, preserving existing behavior. Set CGC_REDACT_SECRETS=true to enable redaction.

Before / After

Before (redaction off — default):

{ "function_name": "fetchData", "api_key": "sk-abc123def456ghi789" }
After (CGC_REDACT_SECRETS=true):
{ "function_name": "fetchData", "api_key": "[REDACTED]" }
  Key Files Modified
- src/codegraphcontext/tools/graph_builder.py
- Hooked into the node property assembly pipeline to scan string literal values before they are assigned to function/class metadata.
- src/codegraphcontext/tools/indexing/persistence/writer.py
- Integrated the CGC_REDACT_SECRETS config check to scrub values right before writing properties to the active graph database backend.
- docs/
- Updated security practices and added a clear note to the .cgc export workflow documentation regarding shared bundles.
  Testing
- Unit tests added for pattern matching regex (e.g., sk-, ghp_, Bearer, AWS key formats).
- Unit tests added for high-entropy string detection.
- Manually verified CGC_REDACT_SECRETS=true masks values via database node inspection.
- Verified documentation builds correctly with updated security notices.
  Checklist
- Closes #1313
- Added unit tests for pattern matching regex and high-entropy detection strings
- Verified CGC_REDACT_SECRETS=true successfully masks values in database node inspections
- Documentation builds correctly with updated security notices
- No breaking changes — redaction is opt-in via environment variable

@vercel

vercel Bot commented Jun 25, 2026

Copy link
Copy Markdown

@DakshtaMethwani is attempting to deploy a commit to the shashankss1205's projects Team on Vercel.

A member of the Team first needs to authorize it.

@Shashankss1205

Copy link
Copy Markdown
Collaborator

👋 Thanks for contributing to CodeGraphContext! Since this PR was opened, main has moved forward significantly (we just shipped v0.5.2 and merged ~50 PRs), so this one now has merge conflicts. Could you please rebase onto the latest main and resolve the conflicts? Once it is conflict-free and CI is green, we will review and merge it. Really appreciate your work — thank you! 🙏

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Triage update: secret scanning/redaction before graph storage (#1313) is still a real gap we want closed. This branch now conflicts with main's indexing pipeline; @DakshtaMethwani would you be able to rebase it onto current main? One ask for the update: keep the scanner opt-out-able via config and make sure redaction happens before any backend write, not after. Happy to review quickly once it's up to date! 🙏

Shashankss1205 and others added 2 commits August 13, 2026 23:13
writer.py lives at codegraphcontext/tools/indexing/persistence/, so
`from ...cli.config_manager` resolves to codegraphcontext.tools.cli,
which does not exist. Every add_file_to_graph call raised
ModuleNotFoundError, breaking indexing entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@Shashankss1205 Shashankss1205 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice feature — verified end-to-end on a fixture with an AWS key, a GitHub PAT and a Postgres connection string:

REDACT_SECRETS=true  → AWS_KEY/GITHUB_TOKEN/DB_URL = "[REDACTED]", NORMAL preserved
REDACT_SECRETS unset → all values stored verbatim (backward compatible)

I resolved the config_manager.py conflict (purely additive on both sides — your REDACT_SECRETS entries alongside main's OPENAI_MODEL/ANTHROPIC_MODEL/CGC_API_KEY) and pushed it to your branch.

I also had to push one fix: writer.py used from ...cli.config_manager, but writer.py sits at codegraphcontext/tools/indexing/persistence/, so three dots resolve to codegraphcontext.tools.cli — which doesn't exist. Every add_file_to_graph call raised ModuleNotFoundError, so indexing failed completely with the feature in place. Four dots (....cli.config_manager) is correct. Worth remembering that the writer path had no test exercising it, which is why this got through — a small test around add_file_to_graph with REDACT_SECRETS=true would have caught it.

Full unit suite green afterwards (1166 passed). Merging — thanks for the contribution, and reach out if you'd like to pick up more security work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants