Skip to content

Add Jenkins workflow and optimise testing mechanism - #6400

Closed
jboskovic wants to merge 1 commit into
cppcheck-opensource:mainfrom
jboskovic:add-coverage
Closed

Add Jenkins workflow and optimise testing mechanism#6400
jboskovic wants to merge 1 commit into
cppcheck-opensource:mainfrom
jboskovic:add-coverage

Conversation

@jboskovic

Copy link
Copy Markdown

Add Jenkins workflow and optimise testing mechanism

@firewave

Copy link
Copy Markdown
Collaborator

Thanks for your contribution. But this is not we change we would accept as this is clearly for a specific downstream implementation which is is not used in our repo. Also we are doing Coverage is a different way (but it is currently temporally disabled because of an upstream issue).

I do like the approach of collecting coverage for each separate test. This is something I wanted to do myself in the past in a different project.

The title does not describe the actual changes.

Also this is not portable and more of a hack.

Also you can achieve this without changing a single line of code. The compiler flags are supposed to be passed to make via CXXFLAGS. And you can simply execute testrunner for each test from a script which prepares and specifies the environment. Some recently introduced parameter allow that. Please take a look at test/scripts/testrunner-single.sh. Extending the loop body at line 16 should be sufficient. There is a downsize though because all tests (and related data) are loaded at each start-up so you might still require the flush. Instantiating the tests only when they are actually executed is being in tracked in https://trac.cppcheck.net/ticket/12080 and will hopefully be addressed within the year.

Still some additional comments in case we will keep using this locally.

Comment thread Makefile
endif
endif

ifeq ($(COVERAGE), 1)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is file is generated via dmake so changes should be applied via the tool otherwise you might lose them.

Comment thread Makefile
endif

ifeq (g++, $(findstring g++,$(CXX)))
override CXXFLAGS += -std=gnu++0x -pipe

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this change. Might be unnecessary to do that. We have some anachronisms in the file as well as some shortcomings. A bigger cleanup is pending review in #4968.

Comment thread test/fixture.cpp
const char* coverage = std::getenv("COVERAGE");
if (coverage) {
std::string testDir = "./coverage_per_test/" + std::string(testname); // Adjust the path as needed
if (mkdir(testDir.c_str(), 0777) != 0) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big no-no. You should not create files or directory with execute or write permissions if not necessary. That might lead to security vulnerabilities. It should be 0644 at most.

Comment thread test/fixture.cpp
{
teardownTestInternal();

__gcov_flush();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is done unconditionally it would interfere with other coverage executions. So this should be dependent on the environment you set up earlier.

@firewave

Copy link
Copy Markdown
Collaborator

No reply in over two weeks - closing.

Please re-open if comments are being addressed and there is still something to apply.

@firewave firewave closed this May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants