Hi @Atos1337 I reproduced the pipeline, its finished with the report creation, but in console it finishes with the following text (I've trained on the smallest piece of data and model is not accurate) with build errors

Please let me know if it's normal behaviour (probably we need to document it)

rg.antlr.v4.codegen.model.RuleFunctionTest > testGetNameByFuzzer1 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6890
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer2]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer3]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer4]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName10]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName11]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName12]: FAILURE

org.antlr.v4.codegen.model.RuleFunctionTest > testGetName12 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6810
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName13]: SUCCESS
Test summary: FAILURE (1079 tests, 30 successes, 1049 failures, 0 skipped)

1079 tests completed, 1049 failed

> Task :utbot-junit-contest:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':utbot-junit-contest:test'.
> There were failing tests. See the report at: file:///home/zaleslaw/IdeaProjects/UTBotJava/utbot-junit-contest/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 12s
36 actionable tasks: 4 executed, 32 up-to-date

Hi @Atos1337 I reproduced the pipeline, its finished with the report creation, but in console it finishes with the following text (I've trained on the smallest piece of data and model is not accurate) with build errors

Please let me know if it's normal behaviour (probably we need to document it)

rg.antlr.v4.codegen.model.RuleFunctionTest > testGetNameByFuzzer1 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6890
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer2]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer3]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer4]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName10]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName11]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName12]: FAILURE

org.antlr.v4.codegen.model.RuleFunctionTest > testGetName12 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6810
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName13]: SUCCESS
Test summary: FAILURE (1079 tests, 30 successes, 1049 failures, 0 skipped)

1079 tests completed, 1049 failed

> Task :utbot-junit-contest:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':utbot-junit-contest:test'.
> There were failing tests. See the report at: file:///home/zaleslaw/IdeaProjects/UTBotJava/utbot-junit-contest/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 12s
36 actionable tasks: 4 executed, 32 up-to-date

@amandelpie it's OK, because UTBot can generate tests that should be failed. So in this case, there are no build errors, there are just failed generated tests.