Fix flaky CI tests - #299
Merged
Merged
Conversation
The PyScript page bootstrap (Pyodide boot + micropip package install) is a cold-start cost paid only by the first test to visit /pyscript/. That test waited 30s for the server-rendered placeholder but only the default 10s for the rendered component, so slow CI runners intermittently timed out. Give the rendered component the same generous 30s timeout as the placeholder, consistent with ReactPy core's PyScript test fixtures. Refs: #297
Archmonger
marked this pull request as ready for review
September 14, 2026 06:03
The test timeout change is not user-facing, so it does not warrant a changelog entry.
Two classes of intermittent failures were reproduced under load: 1. Timeout-based flakes (test_form_async_events, test_component_use_user_data, test_component_use_user_data_with_default, test_component_session_exists, test_error_synchronous_only_operation, test_component_params): the shared page's default 10s Playwright timeout was too short for async state updates, form mutations, and page loads on slow/loaded runners. Raise it to 30s to match ReactPy core's test fixture timeout. 2. test_url_router_navigation_state (assert uuid1 == uuid2): SPA navigation preserves use_state, but the WebSocket reconnects under load, causing ReactPy to re-mount the component and reset hook state. Stabilize the navigation token via a module-level store keyed per component, and hoist the route definitions to module scope so route elements have stable identity.
Archmonger
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The scheduled
Test/python-sourceCI jobs fail intermittently due to a handful of flaky tests. This PR makes the suite robust by addressing the two root causes.Root causes & fixes
1. Timeout-based flakes
The shared browser page used the default 10s Playwright timeout (
page.set_default_timeout(10000)), which was too short for the full component set running under one shared page + server on slow/loaded CI runners. This caused intermittentPage.wait_for_selector: Timeout 10000ms exceededin:test_form_async_eventstest_component_use_user_datatest_component_use_user_data_with_defaulttest_component_session_existstest_error_synchronous_only_operationtest_component_paramstest_pyscript_0_hello_world(PyScript cold-start — already addressed)Fix: raise the default timeout to 30s, matching ReactPy core's test fixture timeout (
DisplayFixture(timeout=30)).2. Router navigation-state flake
test_url_router_navigation_statefailed intermittently withAssertionError: assert uuid1 == uuid2. SPA navigation preservesuse_state, but the WebSocket can reconnect under load, which causes ReactPy to re-mount the component and reset hook state (losing thenext_pagenavigation token).Fix (test app):
main()on every render).next_pagenavigation token in a module-level per-component store so it survives both navigation and transient reconnects.Verification
test_componentssuite (86 tests) passes 10 consecutive runs locally with zero flakes (previously reproducedtest_form_async_events,test_url_router_navigation_state, and the user-data/session/db flakes under load).rufflint/format andpyrighttype-checks pass.python-sourcematrix jobs (16 combos), formatting, types, JS, docs, and CodeQL pass on the PR CI.Checklist
By submitting this pull request I agree that all contributions comply with this project's open source license(s).