You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add tests with enabled=false to report as skipped #49
changed the title [-]Add tests with enabled=false to report as skipped[/-][+][TestNG] Add tests with enabled=false to report as skipped[/+]on Apr 19, 2017
Disabled test should be marked as pending or "unknown" (Allure 2 version). Skipped tests in testNG lexicon are tests that was not executed due to failure of depend-on tests or configuration methods. TestNG is not executing disabled tests at all and not keeping any record of them in the test results. In Allure 1 AllureTestListener I added functionality to mark disabled tests as "pending" ("work in progress") for test coverage tracking. It is preferable to label those tests as "Disabled" but there is no such status currently in Allure report.
@ehborisov
Hi.
Is there any way to implement this?
It will help a lot.
'Cos use case is the following: we have 1 acceptance test (that connection is established). If this check fails - no sense to continue because it is an entry point for each and every test so we do not proceed with the rest of tests.
And we may see the following trend - e.g. 50 test -> 51 tests -> 1 failed test -> 51 -> 1 failed test and it is very confusing.
changed the title [-][TestNG] Add tests with enabled=false to report as skipped[/-][+]Add tests with enabled=false to report as skipped[/+]on Nov 12, 2018
Activity
[-]Add tests with enabled=false to report as skipped[/-][+][TestNG] Add tests with enabled=false to report as skipped[/+]braimanm commentedon Jul 27, 2017
Disabled test should be marked as pending or "unknown" (Allure 2 version). Skipped tests in testNG lexicon are tests that was not executed due to failure of depend-on tests or configuration methods. TestNG is not executing disabled tests at all and not keeping any record of them in the test results. In Allure 1 AllureTestListener I added functionality to mark disabled tests as "pending" ("work in progress") for test coverage tracking. It is preferable to label those tests as "Disabled" but there is no such status currently in Allure report.
slipwalker commentedon Mar 19, 2018
@ehborisov If I got it right, currently Allure2 report doesn't track disabled tests (tests with enabled=false)? Am I right?
ehborisov commentedon Mar 25, 2018
@slipwalker yes, there isn't any easy way to keep track of those tests. They aren't being executed so test listener doesn't know anything about them.
tarasmytlovych commentedon Apr 11, 2018
@ehborisov
Hi.
Is there any way to implement this?
It will help a lot.
'Cos use case is the following: we have 1 acceptance test (that connection is established). If this check fails - no sense to continue because it is an entry point for each and every test so we do not proceed with the rest of tests.
And we may see the following trend - e.g. 50 test -> 51 tests -> 1 failed test -> 51 -> 1 failed test and it is very confusing.
allure-cucumber-jvm-adaptor, version:1.6.4
allure-cucumber-plugin, version: 1.0.2
AlexanderPoleschuk commentedon May 17, 2018
According to documentation: http://testng.org/doc/documentation-main.html#ignore-tests the most appropriate way to handle enabled=false case is to add another status called Ignored.
This will store only disabled tests and distinguish it from really skipped tests.
[-][TestNG] Add tests with enabled=false to report as skipped[/-][+]Add tests with enabled=false to report as skipped[/+]display disabled tests in the report (fixes #49, via #275)