Integrates Codecov for PHP code coverage#842
Conversation
Adds Codecov integration to the PHP test workflow in GitHub Actions. Enables automated code coverage collection using Xdebug for a dedicated matrix job. Configures Codecov through `codecov.yml` to set project and patch coverage thresholds, ignore irrelevant files, and manage PR comments. This provides automated visibility into code coverage changes within pull requests, helping maintain code quality.
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the Core SVNIf you're a Core Committer, use this list when committing to GitHub Merge commitsIf you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
There was a problem hiding this comment.
Pull request overview
Adds Codecov-based PHP coverage reporting to the CI pipeline, so pull requests can surface project/patch coverage signals and upload a Clover report from a dedicated coverage job.
Changes:
- Add
codecov.ymlconfiguration for coverage statuses (project/patch), ignored paths, and PR comment behavior. - Extend the PHP test GitHub Actions workflow with a dedicated “with coverage” matrix entry that enables Xdebug coverage.
- Upload
tests/logs/clover.xmlto Codecov from the coverage job.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
codecov.yml |
Introduces Codecov configuration (coverage status thresholds, ignore rules, PR comment setting). |
.github/workflows/test.yml |
Adds a coverage-enabled matrix run and uploads the Clover report to Codecov. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
.github/workflows/test.yml
Outdated
| - name: Upload code coverage report | ||
| if: ${{ matrix.coverage }} | ||
| uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de | ||
| with: | ||
| file: tests/logs/clover.xml | ||
| flags: phpunit | ||
| token: ${{ secrets.CODECOV_TOKEN }} |
There was a problem hiding this comment.
codecov/codecov-action is configured with token: ${{ secrets.CODECOV_TOKEN }}. Secrets are not available to pull_request workflows from forks, so the coverage matrix job will fail for external contributors unless you either omit the token (for tokenless uploads via the Codecov GitHub App / OSS mode) or gate the upload step to only run when a token is available (e.g., same-repo PRs / pushes) or set the action to not fail CI when the token is missing.
There was a problem hiding this comment.
@nimesh-xecurify can we get your feedback on this? what would be your approach?
There was a problem hiding this comment.
(for tokenless uploads via the Codecov GitHub App / OSS mode)
I believe removing the token would be the way. It does require installing the Codecov GitHub App across the repo. plugin-check implements the last suggested approach - set the action to not fail CI when the token is missing.
@jeffpaul we could use your opinion on this. Would it be possible to omit the token (OSS Mode)?
There was a problem hiding this comment.
The Codecov app is set up on https://github.com/WordPress/ai so doing the same here seems fine to me, though I don't have repo admin access to click the necessary buttons
Configures the Codecov upload step to not fail the CI pipeline if an error occurs during coverage data submission. This ensures that essential build and test jobs can pass even if there are transient issues with Codecov integration or network connectivity, preventing unnecessary CI failures.
|
Hi @dd32, would you be able to add the Codecov GitHub app to the two-factor GitHub repository? Thanks for your time! |
|
@nimesh-xecurify The Codecov app is already installed for all repo's
|
.github/workflows/test.yml
Outdated
| command: npm run env start | ||
| command: | | ||
| if [[ ${{ matrix.coverage == true }} == true ]]; then | ||
| npm run env start -- --xdebug=coverage |
There was a problem hiding this comment.
Could we simplify this and always run the coverage reporting? In my experience the overhead is negligible and there is arguably no reason we wouldn't want coverage reported in CI runs even if they're not sent to Codecov.
There was a problem hiding this comment.
The condition avoids enabling Xdebug across all ~27 matrix jobs since it can slow PHP execution 2–5x and adds coverage output noise to every job's log. Only one combination uploads to Codecov anyway. That said, if the overhead is acceptable, happy to simplify.
.github/workflows/test.yml
Outdated
| - latest | ||
| - trunk | ||
| - '6.8' | ||
| coverage: [false] |
There was a problem hiding this comment.
Question: does this need to use the array syntax? If so, could we stick to the - ... approach used in other lists? See https://stackoverflow.com/a/33136212
There was a problem hiding this comment.
Good catch, written inline since it's a single value. Will update to match the rest of the matrix.
Configures the PHP test workflow to generate code coverage data consistently. Uploads the coverage report to Codecov for the PHP 8.3 / latest WordPress test matrix. Removes the dedicated `coverage` matrix variable, streamlining the workflow definition. Adds a Codecov badge to the README to visualize current coverage status.
masteradhoc
left a comment
masteradhoc
left a comment
There was a problem hiding this comment.
Thanks @nimesh-xecurify - looks great! Thanks for the fast response after the feedback of @kasparsd.
The codecov readme has still a unknown label, i guess this will be updated as soon as the first run was uploaded to codecov. looks good to merge!
What?
Adds Codecov integration to the PHP test workflow in GitHub Actions. Enables automated code coverage collection using Xdebug for a dedicated matrix job. Configures Codecov through
codecov.ymlto set project and patch coverage thresholds, ignore irrelevant files, and manage PR comments.Why?
This provides automated visibility into code coverage changes within pull requests, helping maintain code quality.
Changelog Entry
Fixes #832