Skip to content

chore(build): split browsers between providers - #5504

Closed
marclaval wants to merge 1 commit into
angular:masterfrom
marclaval:splitBrowsers
Closed

chore(build): split browsers between providers#5504
marclaval wants to merge 1 commit into
angular:masterfrom
marclaval:splitBrowsers

Conversation

@marclaval

@marclaval marclaval commented Nov 30, 2015

Copy link
Copy Markdown
Contributor

This PR splits browsers between Sauce Labs (SL) and Browserstack (BS) in order to minimize the number of flakes and failures.
Monitoring the last 160 builds of the project shows that BS is much better with MacOS, while SL is with Windows and Linux.

On top of that, some browsers are only available by some providers:

  • SL: Android4.1, Android4.2, Android5
  • BS: Windows Phone

So the proposed split is:

  • SL: all Android, all IE, Edge, Chrome, Firefox
  • BS: all Safari, all iOS, Windows Phone

Appendix

Failures per browser and per provider:

Browser SL BS
Android4.1 2 N/A
Android4.2 6 N/A
Android4.3 4 8
Android4.4 2 5
Android5 8 N/A
Chrome 0 9
Edge 5 6
Firefox 0 7
IE10 0 4
IE11 0 5
IE11mobile N/A 6
IE9 0 5
Safari7 0 1
Safari8 7 1
Safari9 4 1
iOS7 6 1
iOS8 19 3
iOS9 N/A N/A

Flakes per browser and per provider:

Browser SL BS
Android4.1 1 N/A
Android4.2 4 N/A
Android4.3 4 2
Android4.4 4 4
Android5 3 N/A
Edge 3 5
Firefox 4 10
IE10 0 10
IE11 1 12
IE11mobile N/A 11
IE9 32 43
Safari7 1 1
Safari8 12 0
Safari9 1 0
iOS7 21 3
iOS8 13 7
iOS9 N/A N/A

Flakes per browser and per provider excluding the most frequent one that will be solved by #5493:

Browser SL BS
Android4.1 1 N/A
Android4.2 3 N/A
Android4.3 3 2
Android4.4 1 4
Android5 0 N/A
Edge 3 4
Firefox 3 10
IE10 0 8
IE11 0 12
IE11mobile N/A 11
IE9 18 5
Safari7 1 0
Safari8 8 0
Safari9 1 0
iOS7 1 0
iOS8 12 0
iOS9 N/A N/A

@marclaval marclaval mentioned this pull request Nov 30, 2015
Closed
@marclaval marclaval added the action: review The PR is still awaiting reviews from at least one requested reviewer label Nov 30, 2015
@marclaval marclaval added this to the beta-00 milestone Nov 30, 2015
@IgorMinar

IgorMinar commented Dec 2, 2015

Copy link
Copy Markdown
Contributor

nice analysis. thanks

@IgorMinar

IgorMinar commented Dec 2, 2015

Copy link
Copy Markdown
Contributor

in the future please use build(sauce/bs) as the commit message prefix

@IgorMinar IgorMinar added pr_state: LGTM action: merge The PR is ready for merge by the caretaker labels Dec 2, 2015
@jelbourn

jelbourn commented Dec 2, 2015

Copy link
Copy Markdown
Contributor

Closed via 2a4ac63

@jelbourn jelbourn closed this Dec 2, 2015
@marclaval
marclaval deleted the splitBrowsers branch December 11, 2017 10:17
@angular-automatic-lock-bot

angular-automatic-lock-bot Bot commented Sep 13, 2019

Copy link
Copy Markdown

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot Bot locked and limited conversation to collaborators Sep 13, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

@IgorMinar IgorMinar

Labels

action: merge The PR is ready for merge by the caretaker action: review The PR is still awaiting reviews from at least one requested reviewer cla: yes

Projects

None yet

Milestone

beta.0

Development

Successfully merging this pull request may close these issues.

4 participants

@marclaval @IgorMinar @jelbourn @googlebot