Skip to content

Get tests passing again in all repositories #5810

Closed

Description

@danielbachhuber

We have a lot of automated test failures right now. Let's get them passing again!

Done is:

  • Automated tests are passing in all repositories.

Activity

  1. added this to the 2.9.0 milestone on Jul 12, 2023
  2. swissspidy commented on Jul 12, 2023

    @swissspidy
    Member

    Now that wp-cli/.github#61 has been merged, let's re-run some tests to see if they still fail now.

    Some are now already passing again, so that's great 🎉

    Note: Some packages (e.g. widget-command, package-command, maintenance-mode-command, language-command, embed-command) haven't run tests in a while, so we need to push some changes to trigger a run

    ➡️ #58673 is a common issue across many packages. It will be resolved hopefully before RC. No action needed from our side!

    wp-cli/scaffold-command

    Still some failures:

    • Scenario: Install WordPress from trunk (in general)
    • Scenario: Install WordPress from trunk (on PHP 5.6 + WP latest)
      • Needs a @require-php-7.0 tag

    wp-cli/profile-command

    • Scenario: Error when SAVEQUERIES is defined to false

    wp-cli/db-command

    • Scenario: Clean up a WordPress install without dropping its database entirely but tables with prefix
  3. wojsmol commented on Jul 12, 2023

    @wojsmol
    Contributor

    Note: Some packages (e.g. widget-command, package-command, maintenance-mode-command, language-command, embed-command) haven't run tests in a while, so we need to push some changes to trigger a run

    For this re-enabiling a testing workflow will be enough.

  4. swissspidy commented on Jul 12, 2023

    @swissspidy
    Member

    Note: Some packages (e.g. widget-command, package-command, maintenance-mode-command, language-command, embed-command) haven't run tests in a while, so we need to push some changes to trigger a run

    For this re-enabiling a testing workflow will be enough.

    We tested it earlier and I was not able to re-run tests after re-enabling the workflows because the runs were too old (more than 90 days). See for example https://github.com/wp-cli/maintenance-mode-command/actions/workflows/testing.yml

  5. wojsmol commented on Jul 12, 2023

    @wojsmol
    Contributor

    @swissspidy After re-enabling https://github.com/wp-cli/maintenance-mode-command/blob/5d51793771c3d927ef9c501b527ea42d30c9838c/.github/workflows/testing.yml#L9-L10 should work Manual run is not allowed for testing workflow.

    To at least receive notification when a repo is stale we can use https://github.blog/2023-06-05-announcing-the-stale-repos-action/ in a wp-cli/dashbord repo

  6. swissspidy commented on Jul 12, 2023

    @swissspidy
    Member

    @wojsmol That's not what I meant, I was referring to this button which is missing in those cases:

    Screenshot 2023-07-12 at 19 39 13

    But yeah I'll just wait for the next test to be triggered.

  7. swissspidy commented on Jul 12, 2023

    @swissspidy
    Member

    wp-cli/core-command (run)

    • 001 Example: | trunk | When I run wp core update --version=trunk
    • Example: | nightly | When I run wp core update --version=nightly
    • 003 Scenario: Installing latest nightly build should skip cache

    These should all get @require-php-7.0

    wp-cli/automated-tests


    I think that's it. We can go through all of them again once that bug in core is fixed.

  8. swissspidy commented on Jul 12, 2023

    @swissspidy
    Member

    wp-cli/search-replace-command (run)

    • 001 Scenario: Run on all tables matching string with wildcard
      • This one seems to cause an exception on trunk. Needs further investigating.
  9. swissspidy commented on Jul 13, 2023

    @swissspidy
    Member

    This fix in trunk should resolve most failures now: https://core.trac.wordpress.org/changeset/56226

  10. danielbachhuber commented on Jul 13, 2023

    @danielbachhuber
    MemberAuthor

    Tagged some releases and submitted an update to the bundle wp-cli/wp-cli-bundle#559

  11. swissspidy commented on Jul 13, 2023

    @swissspidy
    Member

    So far it's looking really good!

    search-replace-command still needs looking into: https://github.com/wp-cli/search-replace-command/actions/runs/5538237598/jobs/10118012114

    In the automated-tests repo embed-command is now failing when trying to do wp embed provider match https://asciinema.org/a/140798. Tests passed in the embed-command repo itself though yesterday. But I don't know why.

    asciinema.org is not an allowlisted oEmbed provider in WordPress and does not oEmbed meta tags to its web pages. Maybe they just removed that recently? In that case we would need to update tests to use different URLs.

    media-command: the tests were disabled, I just re-enabled the workflow.

    This test appears to be failing as it tries to upgrade to trunk on PHP 5.6:

    https://github.com/wp-cli/media-command/blob/0334b288f123861714029c843d1ea9410ebb7f27/features/media-regenerate.feature#L1214-L1226

    Any ideas?

  12. danielbachhuber commented on Jul 13, 2023

    @danielbachhuber
    MemberAuthor

    search-replace-command still needs looking into: https://github.com/wp-cli/search-replace-command/actions/runs/5538237598/jobs/10118012114

    Seems to be some collision with a new core option in trunk. I tweaked the test with wp-cli/search-replace-command#185

    In the automated-tests repo embed-command is now failing when trying to do wp embed provider match https://asciinema.org/a/140798. Tests passed in the embed-command repo itself though yesterday. But I don't know why.

    Yeah, let's swap with a different provider.

    This test appears to be failing as it tries to upgrade to trunk on PHP 5.6:

    https://github.com/wp-cli/media-command/blob/0334b288f123861714029c843d1ea9410ebb7f27/features/media-regenerate.feature#L1214-L1226

    We can skip the test because it's only applicable to PHP 5.6: wp-cli/media-command#179

  13. swissspidy commented on Jul 13, 2023

    @swissspidy
    Member
  14. swissspidy commented on Jul 13, 2023

    @swissspidy
    Member

    The dashboard is all green now! 🎉

  15. danielbachhuber commented on Jul 13, 2023

    @danielbachhuber
    MemberAuthor

    Thanks for your work on this, @swissspidy ! 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions