QUnit.todo() with a failure are reported as failling #111
Description
I had the use case of a test being properly implemented but the feature code being incomplete and failing. When running the tests with the QUnit web interface, the test is not erroring out and is properly labeled todo.
However when running karma, the test is wrongfully reported as failing:
SUMMARY:
✖ 1 test failed
FAILED TESTS:
ext.graph.visualEditor
✖ ve.ui.TableWidget
Chrome 66.0.3359 (Linux 0.0.0)
Died on test #4 at
....
TypeError: Cannot read property '0' of undefined
at VeUiTableWidget.ve.ui.TableWidget.onInsertColumn // expected failure
...
at runTest (node_modules/qunit/qunit/qunit.js:1530:30)
at Test.run (node_modules/qunit/qunit/qunit.js:1516:6)
at node_modules/qunit/qunit/qunit.js:1728:12
at advanceTaskQueue (node_modules/qunit/qunit/qunit.js:1129:6)
at advance (node_modules/qunit/qunit/qunit.js:1110:4)
In lib/adapter.js I adjusted runner.testDone to print the test and testResult to the console which yields:
LOG: Object{
name: 've.ui.TableWidget',
module: 'ext.graph.visualEditor',
skipped: false,
todo: true,
failed: 1,
passed: 3,
total: 4,
LOG: Object{success: false, errors: ['Died on test #4 ...
The testResult is set by runner.log which mark it with success = false. However in runner.testDone, the todo flag should be used to invert the result. Potentially:
--- orig.js 2018-06-26 22:31:26.880087441 +0200
+++ new.js 2018-06-26 22:32:36.218186993 +0200
@@ -8,6 +8,11 @@
time: new Date().getTime() - timer
}
+ // For QUnit.todo(): flip success report
+ if ( test.todo === true ) {
+ result.success = !testResult.success
+ }
+
if (result.description.indexOf('global failure') !== -1) {
return
}
Environment:
- karma 2.0.2
- karma-qunit 2.0.1
- qunit 2.6.0