Test Chat Summary: 18th June 2025

On Wednesday, June 18, 2025 at 07:00 PM GMT+3, <test-chat> started in #core-test facilitated by @krupajnanda. The agenda can be found here.

1. Attendance 

@krupajnanda, @azharderaiya, @sirlouen, @dvpatel, @lumiblog, @narenin, @muddassirnasim, @oglekler, @ravigadhiyawp,  and @dilip2615.

2. Looking for Volunteers

  • July 2: Test Chat Meeting Facilitator: Volunteer Needed 
  • July 2: Test Chat Meeting Recap Notes: Volunteer Needed

3. Announcements 📣

4. Test Team Updates

5. Questions/Blocker

@sirlouen noted that most people in the #core team appeared to be completely unaware of the Test team’s work. He pointed out that even though @oglekler had been running test tables for a long time, this contribution was not widely known.

@krupajnanda acknowledged the lack of awareness and clarified that this was one of the reasons she had raised several points during earlier discussions. However, she mentioned that due to time constraints, she had not received many answers.

@oglekler emphasised the need for a structured plan outlining the team’s activities and goals. She suggested we also needed a proper content plan and proposed checking with the Core Dev Blog team to determine what kind of testing-related content could be shared there. She highlighted that once such content was published, the team could request amplification for a broader reach.

@krupajnanda proposed creating a “Month in Test” summary, similar to our existing weekly updates but offering a higher-level overview. She also suggested:

  • Reporting on the number of tickets resolved,
  • Tracking the onboarding of new contributors,
  • Hosting monthly video meetings (e.g., on Zoom or Google Meet),
  • Documenting updates,
  • And continuing patch testing work, which SirLouen had already been involved in.

@sirlouen shared that the current testing activities during release parties often involved repetitive tasks that did not effectively uncover issues in new features. He stated that contributors often performed the same basic actions, which could easily be covered by automated E2E (End-to-End) tests. He added that resources could be used more wisely by focusing on deeper feature-specific testing.

He proposed that:

  • Ideal test cases could be documented in developer notes before release parties.
  • The BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. Testing PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party could be enhanced to provide contextual testing instructions.
  • The plugin could be improved to engage testers more effectively, taking inspiration from Apple TestFlight.

@oglekler agreed that they needed automation for routine tasks, but stressed that human testers brought creativity and unpredictability that machines could not replicate. She felt the release parties served a dual purpose; verifying package integrity and fostering community spirit, and should not be discarded.

@krupajnanda added that documentation already existed (such as Help Test Docs), which outlined testing steps. However, she agreed that testers should go beyond ticking off checklists and explore newly introduced features to ensure functionality.

@sirlouen clarified that his intention was not to eliminate release parties but to make them more effective by focusing efforts on meaningful testing. He suggested creating a bullet list of tasks typically performed during release parties to help define what could be automated and what required human input.

@oglekler remarked that while most testers were simply updating via plugins, some were doing more extensive work, like testing version upgrades. She acknowledged that parties still had value but agreed that more targeted testing should be encouraged.

@krupajnanda proposed shifting focus from triage to testing 6.8.2 tickets starting next week. @oglekler responded that she would review the tickets, or possibly @sirlouen would do so first. 

@sirlouen added that most 6.8.2 tickets had likely already been tested, as they had been merged into trunk earlier. He said he would still monitor bug scrubs for any relevant tickets.

@krupajnanda concluded that they would assess whether any tickets required special attention; if not, they would continue with triage as usual.

6. Call for Testers/Visibility

7. Open Floor

There was no issue to discuss. 

8. Next Meeting 🗓

The next meeting will be on Wednesday, July 2, 2025 at 07:00 PM GMT+3, held on #core-test!

#meeting-notes

Thank you, @krupajnanda, for the peer review of this post. 

Are you interested in helping write Test chat summaries like this one? Volunteer at the start of the next <test-chat> and earn some props.

#test-team

Test Team Chat Summary: 3 August 2021

The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., here.

Announcement: Test Team Reps nominations

Announcement starts here.

@hellofromtonya mentioned that two testers were nominated for this position to work together, both of them accepted the nomination, @hellofromtonya @boniu91

Team had no objections to those nominations

@hellofromtonya published and shared previously prepared introductions of newly nominated Test Team Reps

Discussion: What is needed for Testers to make their contribution easier and better

The discussion starts here.

@francina shared her list, people present on the meeting agreed:

  • Clear testing instructions in tickets
  • Clear, unified, synched instructions to setup different testing environments
  • Streaming to go through tests together and learn by doing
  • A roadmap and action plan to automated testing

@lucatume added, that’s not clear for him:

  • Where and how he could contribute in a best way. He’s willing to help with what he’s the best in: coding.
  • It’s also not clear where we should collect discussions about the tickets and other important things.

@hellofromtonya replied to the second point, saying that:

  • If the discussion is related to ticket, we should collect them in ticket. If the discussion happens on Slack, it’s a good practice to link it inside the ticket.
  • Slack is great for adding collaboration, seeking guidance, Team chats, etc.

@lucatume expanded the previous (first) point and for the most effective contribution, the following things should be clear:

  • A clear indication of what tickets will need test code, not manual testing.
  • A clear path to how test code can be contributed. If tools are required, what tools are required.
  • A definition of “good” and “bad” friction.

@hellofromtonya answered, that tickets with needs-unit-tests keyword are the ones for the PHPUnit side of things. We don’t have anything for the e2e tests though.

@lucatume suggested adding needs-e2e-tests keyword and the Team agreed that’d be useful.

@francina mentioned, that the current test setup is Jest + Puppeteer. We need developers to set up a tool for the Team that will make creating e2e tests easy.

Team agreed that it’s not possible to have 100% of automated tests, with no human review.

@francina shared what the Team needs right now to kick off the e2e testing:

  • Skilled QA engineers to setup up the infrastructure
  • Documentation to teach people to write testing specs
  • People to write the tests
  • Automated test engineers to review and maintain

@hellofromtonya said, that most likely, the bottleneck will be the group of automation test engineers to do the review, tuning, and maintenance work.

@lucatume explained how the “autogenerating” of e2e tests looks like (here)

@hellofromtonya confirmed the workflow:

  • Human does the manual test
  • Tooling records those steps
  • Tooling converts the steps into code

Mai mentioned, that generated code can be worse in terms of quality than the one written by a human. Team agreed that it’ll need to be refined and maintained. @hellofromtonya shared improved workflow:

  • Human tester does the manual testing steps
  • Tooling records those steps
  • Tooling converts the steps into code
  • Test code is attached to the ticket
  • Code is reviewed by a human is skilled in e2e tests and the thing under test
  • Once approved, a coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. committer commits the test code into the project
  • Skilled e2e test humans maintain the tests (including tuning and refinement)

Team agreed to start a pilot initiative:

  • Build a prototype
  • Start small with a handful of impactful e2e tests
  • Get those tests stable
  • Learn
  • Iterate

@lucatume will start creating the prototype
@francina offered reviewing the test handbooks

#build-test-tools, #test-team