Skip to content

Fix test parallelism failures - #360

Merged
mpictor merged 2 commits into
stepcode:masterfrom
jepler:test-parallelism-failures
Aug 22, 2017
Merged

Fix test parallelism failures#360
mpictor merged 2 commits into
stepcode:masterfrom
jepler:test-parallelism-failures

Conversation

@jepler

@jepler jepler commented Aug 16, 2017

Copy link
Copy Markdown
Contributor

These two changes fix the overt and subtle problem I identified with parallel ctest. After applying these two changes locally, I was able to complete 24 consecutive 'ctest' invocations each with a parallelism of 24 (on an 4C/8T system). Typical end of successful test output:

100% tests passed, 0 tests failed out of 242

Label Time Summary:
cpp_schema_build       =  76.07 sec (44 tests)
cpp_schema_gen         =  25.17 sec (22 tests)
cpp_schema_rw          =  39.02 sec (78 tests)
cpp_schema_specific    =   8.50 sec (13 tests)
cpp_unit_stepcore      =   4.68 sec (8 tests)
exchange_file          =   0.93 sec (33 tests)
parser                 =   1.03 sec (2 tests)
unitary_schemas        =   0.25 sec (31 tests)

Total Test time (real) =   8.34 sec

Closes: #359

@mpictor

mpictor commented Aug 19, 2017

Copy link
Copy Markdown
Member

I like having the date & time in the version string, so what about only disabling that for testing?
i.e.

if(SC_ENABLE_TESTING)
  set( date_time_string "NA - disabled for testing")
else ()
  # existing logic goes here
endif()

jepler added 2 commits August 21, 2017 08:31
As analyzed in #359, if the header contains the current time, it will
be updated while running the testsuite; this, in turn, causes multiple
cmake processes to attempt to update targets like lib/libexpress.so.2.0.0
at the same time, causing test failures.
In #359 I identify a race condition between multiple parallel invocations
of cmake, which can arise naturally during ctests.  Now that the file
contents will not change without an intervening git commit, it is
sufficient to ensure that the parallel invocations use distinct temporary
file names with high probability.
@jepler

jepler commented Aug 21, 2017

Copy link
Copy Markdown
Contributor Author

@mpictor as requested, I've updated this branch so that the timestamp is only disabled when SC_ENABLE_TESTING.

@mpictor
mpictor merged commit 81a0362 into stepcode:master Aug 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

why many tests fail when run in parallel

2 participants