Skip to content

Conversation

@olix0r
Copy link
Member

@olix0r olix0r commented Nov 18, 2021

  • Avoid building the binary target, since it slows down the build
  • Use --skip-clean to use the cached build
  • Fetch a pre-build tarpaulin binary (saves ~5m)
  • Mark flakey tests as flakey

* Avoid building the binary target, since it slows down the build
* Use --skip-clean to use the cached build
* Fetch a pre-build tarpaulin binary (saves ~5m)
* Mark flakey tests as flakey
@olix0r olix0r requested a review from a team November 18, 2021 01:01
}

// XXX This test is flakey when running coverage tests.
#[cfg_attr(not(feature = "flaky_tests"), ignore)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe the flakey_tests feature is disabled when running the normal CI test job, as well, so these tests will now never run on CI. should we have a separate cfg to disable them only under coverage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure these tests can actually be flakey in ci too :/ I don't have a good catalog of flakey test failures, but they show up in normal CI runs from time to time. So, I'd really love to figure out how to test this stuff better in general...

I guess we could add a very_flaky_test feature for now, though.

@olix0r
Copy link
Member Author

olix0r commented Nov 18, 2021

@hawkw please take another look -- we add a flakey-in-coverage feature which includes these tests. I've updated integration tests to run all tests by default; and in doing so I've found a few tests that never pass for me (now marked as #[ignore])

Copy link
Contributor

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, lgtm!

the fact that so many of the "flaky" tests are just consistently broken is pretty sad, we should definitely fix those (after determining if they're even still relevant?) but the coverage change is 👍

@olix0r olix0r merged commit 2f19d9c into main Nov 18, 2021
@olix0r olix0r deleted the ver/coverage branch November 18, 2021 22:51
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Nov 24, 2021
This release includes no user-facing changes.

The proxy's outbound HTTP router has been refactored to support upcoming
changes.

---

* build(deps): bump serde_json from 1.0.70 to 1.0.71 (linkerd/linkerd2-proxy#1383)
* ci: Run dependabot updates at 2AM PST (linkerd/linkerd2-proxy#1384)
* build(deps): bump tower from 0.4.10 to 0.4.11 (linkerd/linkerd2-proxy#1385)
* ci: Fix coverage workflow (linkerd/linkerd2-proxy#1382)
* outbound: Use per-route services in routing stack (linkerd/linkerd2-proxy#1380)
* build(deps): bump tracing-subscriber from 0.3.1 to 0.3.2 (linkerd/linkerd2-proxy#1387)
* build(deps): bump libc from 0.2.107 to 0.2.108 (linkerd/linkerd2-proxy#1386)
* build(deps): bump futures from 0.3.17 to 0.3.18 (linkerd/linkerd2-proxy#1388)
alpeb pushed a commit to linkerd/linkerd2 that referenced this pull request Nov 25, 2021
This release includes no user-facing changes.

The proxy's outbound HTTP router has been refactored to support upcoming
changes.

---

* build(deps): bump serde_json from 1.0.70 to 1.0.71 (linkerd/linkerd2-proxy#1383)
* ci: Run dependabot updates at 2AM PST (linkerd/linkerd2-proxy#1384)
* build(deps): bump tower from 0.4.10 to 0.4.11 (linkerd/linkerd2-proxy#1385)
* ci: Fix coverage workflow (linkerd/linkerd2-proxy#1382)
* outbound: Use per-route services in routing stack (linkerd/linkerd2-proxy#1380)
* build(deps): bump tracing-subscriber from 0.3.1 to 0.3.2 (linkerd/linkerd2-proxy#1387)
* build(deps): bump libc from 0.2.107 to 0.2.108 (linkerd/linkerd2-proxy#1386)
* build(deps): bump futures from 0.3.17 to 0.3.18 (linkerd/linkerd2-proxy#1388)
@olix0r olix0r restored the ver/coverage branch November 29, 2021 16:42
@olix0r olix0r deleted the ver/coverage branch November 29, 2021 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants