Looks fantastic, thank you! Once we have all tests marked up, perhaps it makes sense to use the lack of an annotation to mean "should pass"?

@hdgarrood Can you show some examples of multiple errors or compilation errors?

Once we have all tests marked up, perhaps it makes sense to use the lack of an annotation to mean "should pass"?

I'm not sure, we do already have a way of indicating this based on what directory the file is in; that is, passing vs failing. I do think it would be a good idea to make sure that every failing test does fail with a specific error though; perhaps a good way to achieve this would be to require that every test inside the failing directory has at least one shouldFailWith declaration?

Can you show some examples of multiple errors or compilation errors?

I've put 3 examples in already, I'm going to start going through and adding more now. Come to think of it I'm not sure if there are any that should produce more than 1 error, we'll see in a moment.

Quite a few of these no longer testing what they are supposed to test since 0.7! I think this is all good now, though.

Also, some next steps from here:

• add tests for suggestions (I think we should have a real data type for suggestions too, not just strings).
• add tests for warnings
• work out how to run all the tests in the manual directory automatically. Perhaps the shouldFailWith mechanism is enough, but we can presumably also embed specific examples inside the Haskell source if we need a more elaborate check.

Brilliant! I've wanted this for so long and never got round to implementing it. Thanks!