Harden docs-review substitution suggestions#118
Merged
Conversation
Require a pre-output checklist before creating suggestion blocks so Elastic substitution syntax is never included in apply-ready review suggestions. Co-authored-by: GPT-5.5 <gpt-5.5@openai.com> Made-with: Cursor
4 tasks
theletterf
added a commit
that referenced
this pull request
May 5, 2026
Updates the pinned gh-aw extension version from v0.71.1 to v0.71.4 (latest release at time of writing) and recompiles all 10 agentic workflow lock files. The bump is driven by two upstream fixes that landed in v0.71.2: 1. **github/gh-aw#28672** (PR github/gh-aw#28738) — `shared/apm.md` checkout step fails in private caller repos with metadata-only token. Without this fix, the docs-content-internal install of the docs-quality-sweep orchestrator (and of any other gh-aw workflow imported via APM) fails before the agent starts because `actions/checkout` cannot fetch the private caller repo. v0.71.2 adds the missing `contents: read` permission to the shared APM job. 2. **github/gh-aw#28691** (PR github/gh-aw#28736) — Safe-output sanitizer escapes template delimiters inside GitHub `suggestion` blocks, breaking Elastic substitution syntax such as `{{fleet-server}}` when an author clicked **Commit suggestion**. The sanitizer now preserves fenced code-block contents verbatim, so the docs-review prompt no longer needs the defensive guidance added in PRs #116 and #118 to avoid emitting suggestion blocks whenever the line includes `{{...}}`. `gh-aw-docs-review.md` is updated accordingly: the dedicated "substitution syntax pre-output checklist" section is removed, and the surrounding sentence about "passing the pre-output checklist below" / "protected substitution syntax" is simplified back to the plain "prefer apply-ready suggestions when they map cleanly" rule. The recently-hardened lower-priority guidance (avoid stale-content nits, etc.) is left in place. Version pin updated in five places to keep local development, CI lint, the weekly auto-recompile job, and the workflow APM imports in sync: - `Makefile` - `scripts/run-gh-aw-compile.sh` - `.github/workflows/check-aw-updates.yml` - `.github/workflows/pre-commit.yml` - `imports:` field of every gh-aw-*.md that uses the shared APM workflow (frontmatter, applies-to, openings, style, review) `make compile` produces 0 errors, 0 warnings against v0.71.4. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
theletterf
added a commit
that referenced
this pull request
May 5, 2026
…re noop (#133) The openings sweep noop'd on its first real run on elastic/docs-content despite the skill being available, because the agent stochastically invoked the skill with a non-exact form the first time: 11:33:22 ✗ skill(docs-page-opening-optimizer) Skill not found 11:33:27 ● noop "docs-page-opening-optimizer skill unavailable..." 11:33:33 ✓ skill(skill: docs-page-opening-optimizer) succeeded The prompt's "abort with noop on skill failure" rule fired on the first non-exact attempt, before the agent retried with the correct `skill(skill: …)` form and got results. By the time the exact invocation succeeded, the noop was already committed. Frontmatter, applies-to, and style produced fix-issues fine in the same orchestrator run. The difference: the frontmatter sweep prompt explicitly tolerates partial skill failures ("if one fails, report only the other"); applies-to and style happened to land on the exact form first try. Openings was the unlucky one. Lift the docs-review pattern (PRs #117/#118) where skill availability is only declared after a confirmed exact-form failure. Apply to the three sweeps that had the strict abort-on-first-failure rule: - openings (was hit by this in real run 25373751367) - applies-to (would hit it next time the agent picks the wrong form) - style (same) Each now spells out the procedure: try the exact form, retry once on failure, only noop after the second exact-form failure. A single non-exact failure is not sufficient evidence. The frontmatter prompt already had a softer fallback ("merge what works, note any failure in Notes"); coherence and staleness use different mechanisms (>50% MCP failure threshold and full-repo deterministic respectively); typos doesn't import any skill. None of those need this change. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
{{...}}substitution syntax.Test plan
make compilesuccessfully with 0 errors and 0 warnings..github/workflows/gh-aw-docs-review.md.Made with Cursor