Skip to content
This repository was archived by the owner on Apr 30, 2021. It is now read-only.

Commit da4bb4f

Browse files
gerphgerph
committed
Update the Makefile to invoke the examples; add a levels concept to tests.
The Makefile's default target has been updated so that it now requires that the 'system' level tests build. This is one of 3 new targets that have been added: - test_level_unittests, which should cause the unit tests to run. We don't have any yet, but some will be added shortly. - test_level_integration, which causes the integration tests to run. These are the tests in the 'tests' directory, which exercise the external interface in a test-like environment to see that it functions as desired. This level is dependant on the unittests passing before it will be run. - test_level_system, which causes the examples to be run. These examples are the sorts of programs that a user might write, and which we expect to be invoked from the command and to do their job. This level is dependant on the integration tests passing before it will be run. In CI, the individual test targets (rather than the levels) would probably be invoked, with each one still dependant on the earlier ones in stages (or maybe just run them all in parallel, as they're all very fast).
1 parent c089d8f commit da4bb4f

1 file changed

Lines changed: 26 additions & 4 deletions

File tree

‎Makefile‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Common User Targets:
22
#
33
# make tests
4-
# - Sxecute the tests
4+
# - Execute the tests
55
#
66
# make shell
77
# - Set up the environment and drop to a shell (re-uses existing shells if they
@@ -40,12 +40,34 @@ NOTICE = @notice() { printf "\n${COL_NOTICE}+++ %s${COL_RESET}\n" "$$@"; } && no
4040
GOOD = @notice() { printf "\n${COL_GOOD}+++ %s${COL_RESET}\n" "$$@"; } && notice
4141

4242

43-
tests: test_testable
44-
${NOTICE} "Running tests"
43+
tests: test_level_system
44+
${GOOD} "All tests passed"
45+
46+
# 'test_level_*' targets run all the tests up to that level
47+
test_level_unittests: unittests
48+
test_level_integration: test_level_unittests integrationtests
49+
test_level_system: test_level_integration systemtests
50+
51+
# Unit tests test individual parse of a small unit.
52+
unittests: test_testable
53+
${NOTICE} "Running unit tests"
54+
${GOOD} "Unit tests passed (we don't have any yet)"
55+
56+
# Integration tests check the integration of those units.
57+
integrationtests: test_testable
58+
${NOTICE} "Running integration tests"
4559
@# Note: We cd into the tests directory, so that we are testing the installed version, not
4660
@# the version in the repository.
4761
${ACTIVATE} && cd tests && python -munittest -v ${TEST_MODULES}
48-
${GOOD} "Tests passed"
62+
${GOOD} "Integration tests passed"
63+
64+
# System tests check that the way that a user might use it works.
65+
systemtests: test_testable
66+
${NOTICE} "Running system tests"
67+
@# We only run 1000 runs; just enough that we get to see that it's running the tests.
68+
${ACTIVATE} && examples/run_all_examples.py --runs 1000
69+
${GOOD} "System tests passed"
70+
4971

5072
venv: venv/successful-${PYTHON_TOOL}
5173

0 commit comments

Comments
 (0)