Skip to content

Categorize Code Quality "not enabled" upload failure as user-error#3972

Open
henrymercer wants to merge 2 commits into
mainfrom
henrymercer/code-quality-user-error-categorization
Open

Categorize Code Quality "not enabled" upload failure as user-error#3972
henrymercer wants to merge 2 commits into
mainfrom
henrymercer/code-quality-user-error-categorization

Conversation

@henrymercer

Copy link
Copy Markdown
Contributor

When a repository requests Code Quality analysis but it is not enabled, the analyze/finish job fails and is reported with status=failure, when it should be user-error: this is a customer configuration problem, not a CodeQL Action bug. We saw a burst of ~290 such failures from a single org, all surfacing this miscategorization.

The root cause is the code-quality SARIF upload (PUT /repos/:owner/:repo/code-quality/analysis). On a 403 it returns the message Code quality is not enabled for this repository. Please enable code quality in the repository settings.. wrapApiConfigurationError only converts a 403 into a ConfigurationError when isEnablementError matches a known pattern, and none of the existing patterns covered the Code Quality wording. So the error stayed a plain Error, and getActionsStatus mapped it to failure.

This change adds a /Code Quality is not enabled/i pattern to isEnablementError, mirroring the existing /Code Scanning is not enabled/i entry. The error now becomes a ConfigurationError and is correctly categorized as user-error. The case-insensitive flag tolerates the observed lowercase wording. The existing enablement-error unit test in src/api-client.test.ts is extended with the Code Quality message. The change is intentionally surgical, with no broader error-handling refactor.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Documentation/test-only and telemetry categorization only; no change to upload behavior or control flow beyond how an already-failing case is labelled.

Which use cases does this change impact?

Workflow types:

  • Managed - Impacts users with dynamic workflows (Default Setup, Code Quality, ...).

Products:

  • Code Quality - The changes impact analyses when analysis-kinds: code-quality.

Environments:

  • Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.
  • GHES - Impacts CodeQL workflows on GitHub Enterprise Server.

How did/will you validate this change?

  • Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files). The existing enablement-error test is extended with the Code Quality message.

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Rollback - Change can only be disabled by rolling back the release or releasing a new version with a fix. The blast radius is limited to how an already-failing upload is categorized in telemetry.

How will you know if something goes wrong after this change is released?

  • Telemetry - The status report report_status.status for these Code Quality enablement failures should move from failure to user-error.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

When a repository requests Code Quality analysis but it is not enabled,
the SARIF upload to `PUT /repos/:owner/:repo/code-quality/analysis`
returns a 403 with the message "Code quality is not enabled for this
repository...". `isEnablementError` did not recognize this wording, so
`wrapApiConfigurationError` left it as a plain Error and the analyze job
was reported with `status=failure` instead of `user-error`.

Add a `/Code Quality is not enabled/i` pattern so the error becomes a
ConfigurationError and `getActionsStatus` categorizes it as `user-error`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the size/XS Should be very easy to review label Jun 23, 2026
@henrymercer henrymercer marked this pull request as ready for review June 23, 2026 15:40
@henrymercer henrymercer requested a review from a team as a code owner June 23, 2026 15:40
Copilot AI review requested due to automatic review settings June 23, 2026 15:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

  • Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Pull request overview

Updates the CodeQL Action’s API error classification so that “Code Quality not enabled” SARIF upload failures are treated as configuration/user errors (rather than action failures), aligning telemetry status with the actual root cause.

Changes:

  • Extend isEnablementError detection to match the Code Quality enablement 403 message.
  • Extend the existing wrapApiConfigurationError enablement-error unit test to include the Code Quality wording (and casing variants).
Show a summary per file
File Description
src/api-client.ts Adds a new enablement-error regex so Code Quality “not enabled” 403s wrap as ConfigurationError.
src/api-client.test.ts Expands enablement-error test cases to include the Code Quality message (including casing transforms).
lib/entry-points.js Generated artifact update (content excluded from diff; not reviewed).

Copilot's findings

Files excluded by content exclusion policy (1)
  • lib/entry-points.js
  • Files reviewed: 2/3 changed files
  • Comments generated: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XS Should be very easy to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants