Skip to content

ci(docs-infra): increase wait for SW on localhost to avoid CI flakes - #29953

Closed
gkalpak wants to merge 2 commits into
angular:masterfrom
gkalpak:build-aio-fix-pwa-test-2
Closed

gkalpak wants to merge 2 commits into
angular:masterfrom
gkalpak:build-aio-fix-pwa-test-2

Conversation

@gkalpak

@gkalpak gkalpak commented Apr 17, 2019

Copy link
Copy Markdown
Member

The local server is unoptimized (e.g. no compression), so wait longer for the SW to be loaded and registered to allow Lighthouse reliably detect it, especially on slower environments (e.g. CI).

Fixes #29910.

@mary-poppins

Copy link
Copy Markdown

You can preview a23f677 at https://pr29953-a23f677.ngbuilds.io/.

@gkalpak
gkalpak force-pushed the build-aio-fix-pwa-test-2 branch from a23f677 to 8c848d7 Compare April 17, 2019 07:17
@mary-poppins

Copy link
Copy Markdown

You can preview 8c848d7 at https://pr29953-8c848d7.ngbuilds.io/.

@gkalpak
gkalpak force-pushed the build-aio-fix-pwa-test-2 branch 2 times, most recently from 88d587e to 0a1d519 Compare April 17, 2019 07:35
@mary-poppins

Copy link
Copy Markdown

You can preview 88d587e at https://pr29953-88d587e.ngbuilds.io/.

@mary-poppins

Copy link
Copy Markdown

You can preview 0a1d519 at https://pr29953-0a1d519.ngbuilds.io/.

@mary-poppins

Copy link
Copy Markdown

You can preview f30670a at https://pr29953-f30670a.ngbuilds.io/.

@gkalpak gkalpak added area: build & ci Related the build and CI infrastructure of the project comp: docs-infra effort1: hours action: merge The PR is ready for merge by the caretaker target: patch This PR is targeted for the next patch release risk: low labels Apr 17, 2019
@ngbot ngbot Bot modified the milestone: needsTriage Apr 17, 2019
The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).

Related: GoogleChrome/lighthouse#5527 (comment)

Fixes angular#29910
@gkalpak
gkalpak force-pushed the build-aio-fix-pwa-test-2 branch from f30670a to 17ea000 Compare April 17, 2019 09:42
@mary-poppins

Copy link
Copy Markdown

You can preview 17ea000 at https://pr29953-17ea000.ngbuilds.io/.

@gkalpak
gkalpak marked this pull request as ready for review April 17, 2019 09:48
@gkalpak
gkalpak requested review from a team April 17, 2019 09:48
function waitLongerForSw(config) {
console.info(`Will wait longer for ServiceWorker (${LONG_WAIT_FOR_SW_DELAY}ms)...`);
const passes = config.passes || (config.passes = []);
passes.push({passName: 'defaultPass', pauseAfterLoadMs: LONG_WAIT_FOR_SW_DELAY});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What other implications are there here? I'd expect local scores to be artificially inflated because in these 5s the app loads completely.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT, there are no other implications. (The score is 100% anyway as long as the SW is detected in time and the failing tests when it is not detected are boolean (does it have a SW? does it work offline? etc).)

Other categories (e.g. performance) don't seem to be inflated due to this setting (e.g. performance is still ~45% on localhost). This makes sense, because pauseAfterLoadMs does not affect performance-specific metrics (such as first-meaningful-paint).

benlesh pushed a commit that referenced this pull request Apr 17, 2019
…29953)

The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).

Related: GoogleChrome/lighthouse#5527 (comment)

Fixes #29910

PR Close #29953
@benlesh benlesh closed this in ca9b3ee Apr 17, 2019
benlesh pushed a commit that referenced this pull request Apr 17, 2019
…29953)

The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).

Related: GoogleChrome/lighthouse#5527 (comment)

Fixes #29910

PR Close #29953
@gkalpak
gkalpak deleted the build-aio-fix-pwa-test-2 branch April 17, 2019 19:33
gkalpak added a commit to gkalpak/angular that referenced this pull request Apr 19, 2019
In angular#29953, the wait period for SW on localhost was increased to avoid CI
flakes for the PWA score tests.

This commit expands the fix to non-localhost origins to avoid flakes in
the `aio_monitoring` job, when CircleCI VMs/network are slow.
(For reference, example failures: [289127], [289238])

[289127]: https://circleci.com/gh/angular/angular/289127
[289238]: https://circleci.com/gh/angular/angular/289238
benlesh pushed a commit that referenced this pull request Apr 19, 2019
#29988)

In #29953, the wait period for SW on localhost was increased to avoid CI
flakes for the PWA score tests.

This commit expands the fix to non-localhost origins to avoid flakes in
the `aio_monitoring` job, when CircleCI VMs/network are slow.
(For reference, example failures: [289127], [289238])

[289127]: https://circleci.com/gh/angular/angular/289127
[289238]: https://circleci.com/gh/angular/angular/289238

PR Close #29988
benlesh pushed a commit that referenced this pull request Apr 19, 2019
#29988)

In #29953, the wait period for SW on localhost was increased to avoid CI
flakes for the PWA score tests.

This commit expands the fix to non-localhost origins to avoid flakes in
the `aio_monitoring` job, when CircleCI VMs/network are slow.
(For reference, example failures: [289127], [289238])

[289127]: https://circleci.com/gh/angular/angular/289127
[289238]: https://circleci.com/gh/angular/angular/289238

PR Close #29988
BioPhoton pushed a commit to BioPhoton/angular that referenced this pull request May 21, 2019
BioPhoton pushed a commit to BioPhoton/angular that referenced this pull request May 21, 2019
…ngular#29953)

The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).

Related: GoogleChrome/lighthouse#5527 (comment)

Fixes angular#29910

PR Close angular#29953
BioPhoton pushed a commit to BioPhoton/angular that referenced this pull request May 21, 2019
angular#29988)

In angular#29953, the wait period for SW on localhost was increased to avoid CI
flakes for the PWA score tests.

This commit expands the fix to non-localhost origins to avoid flakes in
the `aio_monitoring` job, when CircleCI VMs/network are slow.
(For reference, example failures: [289127], [289238])

[289127]: https://circleci.com/gh/angular/angular/289127
[289238]: https://circleci.com/gh/angular/angular/289238

PR Close angular#29988
@angular-automatic-lock-bot

Copy link
Copy Markdown

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot Bot locked and limited conversation to collaborators Sep 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: build & ci Related the build and CI infrastructure of the project cla: yes effort1: hours risk: low target: patch This PR is targeted for the next patch release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(docs-infra): test_aio/test_aio_local jobs occasionally fail due to low PWA score

4 participants