fix(tests): fail CI when Gradle produces no test results#21066
Conversation
Gradle 9 silently skips test tasks (NO-SOURCE) and exits 0 when testClassesDirs isn't wired to registered Test tasks. This went undetected for 16 consecutive master commits after the Gradle 9 upgrade because the job reported SUCCESS despite running zero tests. Add a post-make guard in all four CI test runner scripts that checks for JUnit XML output. If no XML files exist after the Gradle invocation, the job fails with a clear error message instead of silently passing. Verified locally: - With testClassesDirs fix: Gradle produces XML, guard passes - Without testClassesDirs fix: Gradle skips (NO-SOURCE), guard fails Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Skipping CI for Draft Pull Request. |
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@janisz 🙏 Please feel free to throw this away and solve it in a better way :) |
🚀 Build Images ReadyImages are ready for commit 3255d69. To use with deploy scripts: export MAIN_IMAGE_TAG=4.12.x-144-g3255d69c9e |
1 participant
Description
Gradle 9 silently skips test tasks (
NO-SOURCE) and exits 0 whentestClassesDirsisn't wired to registeredTesttasks. This went undetected for 16 consecutive master commits after the Gradle 9 upgrade (#20295) because the job reportedSUCCESSdespite running zero Groovy tests.The root cause was fixed in #20429 (
testClassesDirs/classpathwiring), but there was no defense-in-depth to catch a recurrence. This PR adds a post-makeguard in all four CI test runner scripts that checks for JUnit XML output after the Gradle invocation. If no XML files exist, the job fails with a clear error instead of silently passing.User-facing documentation
Testing and quality
Automated testing
How I validated my change
Verified locally with Gradle 9.5.1:
testClassesDirsfix in place:./gradlew testDeploymentCheckruns tests, producesbuild/test-results/testDeploymentCheck/TEST-DeploymentCheck.xml. Guard passes.testClassesDirsfix removed:./gradlew testDeploymentCheckreportsNO-SOURCE, exits 0, produces no XML. Guard correctly fails with:ERROR: No test results (JUnit XML) produced. Gradle may have skipped tests (NO-SOURCE).Also verified against 30 Prow CI runs on master that the duration/result pattern matches exactly: all 16 pre-fix runs showed
NO-SOURCE(43-84 min,SUCCESS), all 9 post-fix runs executed tests (135-152 min).