Description

I'm trying to add allure-cucumber to an existing codebase. This codebase does not use RSpec, and test failures are triggered as explicit raise "error message" calls.
The problem: if a test fails, the failing step is correctly reported as failed, but the test itself is reported as broken.

The cause: I've narrowed this down to the on_test_case_finished method. Unlike on_test_step_finished, this method additionally calls ResultUtils.status for failed tests, which checks the failure exception against RSpec::Expectations::ExpectationNotMetError. Without RSpec, my exception is a standard RuntimeError, therefore the condition fails, and the test is labeled as broken.

I would like to clarify if this behavior is expected.

Expected Behavior

Both the step and test itself are assigned the same status - either failed or broken.

Alternative Solutions

For now I've added an override for the status method, which resolves my problem:

module Allure
  module ResultUtils
    class << self
      def status(exception)
        exception.is_a?(StandardError) ? Allure::Status::FAILED : Allure::Status::BROKEN
      end
    end
  end
end