You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
TestNg Setup/Before errors should NOT be separate test cases in report #643
Describe the bug
Allure generates individual test case from testNG setup/before methods too if they break. Attached project to reproduce: bug_reports.zip
To Reproduce
Steps to reproduce the behavior:
unzip Attached project
execute mvn clean test
execute cd target and allure serve
Check generated report. (I attached screenshot too)
It contains one additional test "myBeforeClass" which is broken. This confuses testers because 1-the number of tests may be varying in the test class 2-it links a method which may be totally unknown for them (if its located in parent or static class).
3-Also It does not add too much value because the same stack can be found linked to the "real" test.
Expected behavior
I would NOT expect to see the setup/before hook's excpetion as independent tests in the report. I suggest deleting the onConfigurationFailure() method from io/qameta/allure/testng/AllureTestNg.java.
Follow steps to verify:
11. Open attached project
12. comment or delete method onConfigurationFailure() in io/qameta/allure/testng/AllureTestNg.java
13. repeat steps 2,3
14. See that in this case only the real broken test is shown in the list. And we can also find the error starting from this.
BTW: the comment in code also indicates that configuration error should not be separate tests
Screenshots
If applicable, add screenshots to help explain your problem.
5b. if uploaded to allureEE it generated strange orphan tests.
==expected images===
Desktop (please complete the following information):
OS: [MAC]
Browser [chrome]
Version [e.g. 22]
Smartphone (please complete the following information):
Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]
Additional context
Add any other context about the problem here.
Reacted by SerhiiMaksymiv, Lyubov Ilyushchenko and Amr Mahmoud
Activity
lilyushchenko commentedon Jan 25, 2023
Still actual, kind of the same issue raised here as well #623
Is there a chance that there will be a fix?
tomkeiev commentedon Jun 26, 2023
seems it's going to be merged soon in #920