Skip to content

Speculative fix for hour_of_code_starwars flaky test - #51278

Merged
molly-moen merged 6 commits into
stagingfrom
molly/star_wars_flaky_test
Apr 13, 2023
Merged

Speculative fix for hour_of_code_starwars flaky test#51278
molly-moen merged 6 commits into
stagingfrom
molly/star_wars_flaky_test

Conversation

@molly-moen

@molly-moen molly-moen commented Apr 11, 2023

Copy link
Copy Markdown
Contributor

iPhone_hour_code_code_starwars has been flaky. I was able to find two failure cases--one where the page was not reloading as expected (saucelabs link) and one where it seemed like we weren't switching droplet to text mode (saucelabs link). I think what was actually happening in that case was the page was finally loading while or after we tried to switch modes, so the mode switch didn't take.

When the page didn't reload, we were stuck on the continue modal from the previous page, and therefore could not find the run button to run. I think this is happening because we are checking for the reloaded page before the page has had a chance to refresh the screen, so we see the previous level, which looks to us like a loaded page. To avoid this issue I added a check to wait until the page is loaded. Brendan saw a similar problem in a different test and fixed it this way

While looking into the second issue before realizing it was the same as the first issue, I ran the script in our existing logic, and it passed for both droplet block and text mode, so it seems like that check is no longer valid. I updated the text mode check to be the opposite of the block mode check; that the gutter display is none. This should just help us in the future to accurately determine if droplet is in text mode or not.

Link

Testing story

Tested this change on saucelabs and it is passing. This test is pretty intermittently flaky, and takes a while to run so I only ran it 3 times and it passed every time.

PR Checklist:

  • Tests provide adequate coverage
  • Privacy and Security impacts have been assessed
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@molly-moen molly-moen changed the title Speculative fix for hour_of_code_star_wars flaky test Speculative fix for hour_of_code_starwars flaky test Apr 12, 2023
@molly-moen
molly-moen requested a review from a team April 12, 2023 22:37
@molly-moen
molly-moen marked this pull request as ready for review April 12, 2023 22:38
@molly-moen
molly-moen merged commit 4cff35a into staging Apr 13, 2023
@molly-moen
molly-moen deleted the molly/star_wars_flaky_test branch April 13, 2023 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants