@@ -18,25 +18,48 @@ The easiest way to run tests is to say "make". This runs all
1818the tests.
1919
2020 *** t0000-basic.sh ***
21- * ok 1: .git/objects should be empty after git- init in an empty repo.
22- * ok 2: .git/objects should have 256 subdirectories.
23- * ok 3: git-update-index without --add should fail adding.
21+ ok 1 - .git/objects should be empty after git init in an empty repo.
22+ ok 2 - .git/objects should have 3 subdirectories.
23+ ok 3 - success is reported like this
2424 ...
25- * ok 23: no diff after checkout and git-update-index --refresh.
26- * passed all 23 test(s)
27- *** t0100-environment-names.sh ***
28- * ok 1: using old names should issue warnings.
29- * ok 2: using old names but having new names should not issue warnings.
30- ...
31-
32- Or you can run each test individually from command line, like
33- this:
34-
35- $ sh ./t3001-ls-files-killed.sh
36- * ok 1: git-update-index --add to add various paths.
37- * ok 2: git-ls-files -k to show killed files.
38- * ok 3: validate git-ls-files -k output.
39- * passed all 3 test(s)
25+ ok 43 - very long name in the index handled sanely
26+ # fixed 1 known breakage(s)
27+ # still have 1 known breakage(s)
28+ # passed all remaining 42 test(s)
29+ 1..43
30+ *** t0001-init.sh ***
31+ ok 1 - plain
32+ ok 2 - plain with GIT_WORK_TREE
33+ ok 3 - plain bare
34+
35+ Since the tests all output TAP (see http://testanything.org) they can
36+ be run with any TAP harness. Here's an example of parallel testing
37+ powered by a recent version of prove(1):
38+
39+ $ prove --timer --jobs 15 ./t[0-9]*.sh
40+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms
41+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms
42+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms
43+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms
44+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms
45+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===
46+
47+ prove and other harnesses come with a lot of useful options. The
48+ --state option in particular is very useful:
49+
50+ # Repeat until no more failures
51+ $ prove -j 15 --state=failed,save ./t[0-9]*.sh
52+
53+ You can also run each test individually from command line, like this:
54+
55+ $ sh ./t3010-ls-files-killed-modified.sh
56+ ok 1 - git update-index --add to add various paths.
57+ ok 2 - git ls-files -k to show killed files.
58+ ok 3 - validate git ls-files -k output.
59+ ok 4 - git ls-files -m to show modified files.
60+ ok 5 - validate git ls-files -m output.
61+ # passed all 5 test(s)
62+ 1..5
4063
4164You can pass --verbose (or -v), --debug (or -d), and --immediate
4265(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
@@ -198,15 +221,101 @@ This test harness library does the following things:
198221 - If the script is invoked with command line argument --help
199222 (or -h), it shows the test_description and exits.
200223
201- - Creates an empty test directory with an empty .git/objects
202- database and chdir(2) into it. This directory is 't/trash directory'
203- if you must know, but I do not think you care.
224+ - Creates an empty test directory with an empty .git/objects database
225+ and chdir(2) into it. This directory is 't/trash
226+ directory.$test_name_without_dotsh', with t/ subject to change by
227+ the --root option documented above.
204228
205229 - Defines standard test helper functions for your scripts to
206230 use. These functions are designed to make all scripts behave
207231 consistently when command line arguments --verbose (or -v),
208232 --debug (or -d), and --immediate (or -i) is given.
209233
234+ Do's, don'ts & things to keep in mind
235+ -------------------------------------
236+
237+ Here are a few examples of things you probably should and shouldn't do
238+ when writing tests.
239+
240+ Do:
241+
242+ - Put all code inside test_expect_success and other assertions.
243+
244+ Even code that isn't a test per se, but merely some setup code
245+ should be inside a test assertion.
246+
247+ - Chain your test assertions
248+
249+ Write test code like this:
250+
251+ git merge foo &&
252+ git push bar &&
253+ test ...
254+
255+ Instead of:
256+
257+ git merge hla
258+ git push gh
259+ test ...
260+
261+ That way all of the commands in your tests will succeed or fail. If
262+ you must ignore the return value of something (e.g. the return
263+ value of export is unportable) it's best to indicate so explicitly
264+ with a semicolon:
265+
266+ export HLAGH;
267+ git merge hla &&
268+ git push gh &&
269+ test ...
270+
271+ Don't:
272+
273+ - exit() within a <script> part.
274+
275+ The harness will catch this as a programming error of the test.
276+ Use test_done instead if you need to stop the tests early (see
277+ "Skipping tests" below).
278+
279+ - Break the TAP output
280+
281+ The raw output from your test may be interpreted by a TAP harness. TAP
282+ harnesses will ignore everything they don't know about, but don't step
283+ on their toes in these areas:
284+
285+ - Don't print lines like "$x..$y" where $x and $y are integers.
286+
287+ - Don't print lines that begin with "ok" or "not ok".
288+
289+ TAP harnesses expect a line that begins with either "ok" and "not
290+ ok" to signal a test passed or failed (and our harness already
291+ produces such lines), so your script shouldn't emit such lines to
292+ their output.
293+
294+ You can glean some further possible issues from the TAP grammar
295+ (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
296+ but the best indication is to just run the tests with prove(1),
297+ it'll complain if anything is amiss.
298+
299+ Keep in mind:
300+
301+ - Inside <script> part, the standard output and standard error
302+ streams are discarded, and the test harness only reports "ok" or
303+ "not ok" to the end user running the tests. Under --verbose, they
304+ are shown to help debugging the tests.
305+
306+
307+ Skipping tests
308+ --------------
309+
310+ If you need to skip all the remaining tests you should set skip_all
311+ and immediately call test_done. The string you give to skip_all will
312+ be used as an explanation for why the test was skipped. for instance:
313+
314+ if ! test_have_prereq PERL
315+ then
316+ skip_all='skipping perl interface tests, perl not available'
317+ test_done
318+ fi
210319
211320End with test_done
212321------------------
@@ -222,9 +331,9 @@ Test harness library
222331There are a handful helper functions defined in the test harness
223332library for your script to use.
224333
225- - test_expect_success <message> <script>
334+ - test_expect_success [<prereq>] <message> <script>
226335
227- This takes two strings as parameter, and evaluates the
336+ Usually takes two strings as parameter, and evaluates the
228337 <script>. If it yields success, test is considered
229338 successful. <message> should state what it is testing.
230339
@@ -234,7 +343,14 @@ library for your script to use.
234343 'git-write-tree should be able to write an empty tree.' \
235344 'tree=$(git-write-tree)'
236345
237- - test_expect_failure <message> <script>
346+ If you supply three parameters the first will be taken to be a
347+ prerequisite, see the test_set_prereq and test_have_prereq
348+ documentation below:
349+
350+ test_expect_success TTY 'git --paginate rev-list uses a pager' \
351+ ' ... '
352+
353+ - test_expect_failure [<prereq>] <message> <script>
238354
239355 This is NOT the opposite of test_expect_success, but is used
240356 to mark a test that demonstrates a known breakage. Unlike
@@ -243,6 +359,16 @@ library for your script to use.
243359 success and "still broken" on failure. Failures from these
244360 tests won't cause -i (immediate) to stop.
245361
362+ Like test_expect_success this function can optionally use a three
363+ argument invocation with a prerequisite as the first argument.
364+
365+ - test_expect_code [<prereq>] <code> <message> <script>
366+
367+ Analogous to test_expect_success, but pass the test if it exits
368+ with a given exit <code>
369+
370+ test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
371+
246372 - test_debug <script>
247373
248374 This takes a single argument, <script>, and evaluates it only
@@ -275,6 +401,85 @@ library for your script to use.
275401 Merges the given rev using the given message. Like test_commit,
276402 creates a tag and calls test_tick before committing.
277403
404+ - test_set_prereq SOME_PREREQ
405+
406+ Set a test prerequisite to be used later with test_have_prereq. The
407+ test-lib will set some prerequisites for you, e.g. PERL and PYTHON
408+ which are derived from ./GIT-BUILD-OPTIONS (grep test_set_prereq
409+ test-lib.sh for more). Others you can set yourself and use later
410+ with either test_have_prereq directly, or the three argument
411+ invocation of test_expect_success and test_expect_failure.
412+
413+ - test_have_prereq SOME PREREQ
414+
415+ Check if we have a prerequisite previously set with
416+ test_set_prereq. The most common use of this directly is to skip
417+ all the tests if we don't have some essential prerequisite:
418+
419+ if ! test_have_prereq PERL
420+ then
421+ skip_all='skipping perl interface tests, perl not available'
422+ test_done
423+ fi
424+
425+ - test_external [<prereq>] <message> <external> <script>
426+
427+ Execute a <script> with an <external> interpreter (like perl). This
428+ was added for tests like t9700-perl-git.sh which do most of their
429+ work in an external test script.
430+
431+ test_external \
432+ 'GitwebCache::*FileCache*' \
433+ "$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
434+
435+ If the test is outputting its own TAP you should set the
436+ test_external_has_tap variable somewhere before calling the first
437+ test_external* function. See t9700-perl-git.sh for an example.
438+
439+ # The external test will outputs its own plan
440+ test_external_has_tap=1
441+
442+ - test_external_without_stderr [<prereq>] <message> <external> <script>
443+
444+ Like test_external but fail if there's any output on stderr,
445+ instead of checking the exit code.
446+
447+ test_external_without_stderr \
448+ 'Perl API' \
449+ "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
450+
451+ - test_must_fail <git-command>
452+
453+ Run a git command and ensure it fails in a controlled way. Use
454+ this instead of "! <git-command>" to fail when git commands
455+ segfault.
456+
457+ - test_might_fail <git-command>
458+
459+ Similar to test_must_fail, but tolerate success, too. Use this
460+ instead of "<git-command> || :" to catch failures due to segv.
461+
462+ - test_cmp <expected> <actual>
463+
464+ Check whether the content of the <actual> file matches the
465+ <expected> file. This behaves like "cmp" but produces more
466+ helpful output when the test is run with "-v" option.
467+
468+ - test_when_finished <script>
469+
470+ Prepend <script> to a list of commands to run to clean up
471+ at the end of the current test. If some clean-up command
472+ fails, the test will not pass.
473+
474+ Example:
475+
476+ test_expect_success 'branch pointing to non-commit' '
477+ git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
478+ test_when_finished "git update-ref -d refs/heads/invalid" &&
479+ ...
480+ '
481+
482+
278483Tips for Writing Tests
279484----------------------
280485
0 commit comments