-
Notifications
You must be signed in to change notification settings - Fork 349
fix Klocwork errors and criticals from logger sources #6738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix Klocwork errors and criticals from logger sources #6738
Conversation
paulstelian97
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many of these changes are either pointless or outright breaking functionality. I am strongly against them. If we thin out this PR to only the useful changes it would be less than a quarter in size to be honest.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is explicitly bad -- what if we end up with a format specifier inside "time_fmt" after the sprintf? Also unsafe sprintf without length specifier AND having a string as both input and output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Is there anything wrong, with having string as both input and output?
- Please read content of this article: Format String Vulnerability.
Yeah this one may be wrong from me, I'm a bit confused here with this format expansions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes here specifically the new code is the wrong one -- time_fmt is supposed to be a format specifier (computed on line 347 of the original code). You are NOT supposed to escape it here, because it's not accidental but intentional.
Further, your new line 351 passes a format string without arguments (time_fmt will have format specifiers), which INTRODUCES the very vulnerability you're mentioning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@paulstelian97 is correct, this change will break the (deprecated) sof-logger. Good catch thanks.
Static analyzers are not always right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
time_fmt is (safely until proven otherwise) constructed at the line before but Klocwork is not capable of analyzing that. In this case it's only capable of a dumb check. This is a classic case of a false positive.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
time_fmt seems to itself be a format specifier; you're changing semantics here unfortunately.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pointless refactor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not pointless refactor. Calling return on ferror causes a dangling pointer to global_config, which is no longer in scope. And static analysis tools reports that fact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This specific change would also have the exact same dangling pointer because it literally compiles to the same thing if you have at least -O1...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This error is fatal, the tool exits when it happens. KW is not smart enough to see that, this is yet another false positive.
Indeed this change makes absolutely no difference, I hope KW is not so dumb to be deceived by it.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're making a value special without it needing to be special. I don't agree at all with this specific change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, so please propose an alternative solution. What do we do if name is NULL?
Skip printing at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fprintf can print which, while ugly, isn't a real vulnerability. Passing NULL for a string is something that most printf implementations (at least the Linux ones) check for explicitly and won't crash at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably the only reason name could be NULL is because some malloc failed. In that very unlikely case the program should just stop and not print garbage. Please use some kind of assert either here or probably better inside log_vasprintf() (in case KW can understand that)
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pointless refactor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that you do not see the issue is not meaning that it is pointless refactor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is pointless because it fixes absolutely nothing here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error in this return rather comes from line 1048, config->logs_header = &snd;. So on return, we're left with config->logs_header (config is passed here as a ptr from scope higher) and local variable address (now dangling) from local scope.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now the issue is still weird -- why do you set a local variable to NULL? That thing can get optimized out and you still get the same code. If "config" is a global variable (so that this change does become meaningful) disregard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Config is not local variable, it is function argument here int convert(struct convert_config *config);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Function parameters are still local variables actually.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Compiler error (due to typo) and it's a pointless refactor anyway.
After a change: no longer typo, but it still remains pointless if "config" is a local variable.
@paulstelian97 I appreciate your review. Anyway, I have to dismiss it now. This PR is intended to be merged to mtl-002 branch (MeteorLake platform only) and we need to proceed it quickly, The PR for main branch will be issued soon and I hope we can continue the discussion there and find the best possible sollution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@paulstelian97 I appreciate your review. Anyway, I have to dismiss it now.
Don't break sof-logger in any branch. Either it's used and you cannot break it, or it's not used in that branch and breaking it is pointless (and you should stop scanning it).
Static analyzers are not always right. Klocwork itself knows this and has a documented process to mark false positives. Use that KW process instead of randomly patching the code.
marc-hb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some of these changes have clearly not been tested.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change this to some assert. If it started to happen because of some bug or other catastrophe then it may happen again later, shouldn't keep going and leave random and confusing gaps in the logs.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This error is fatal, the tool exits when it happens. KW is not smart enough to see that, this is yet another false positive.
Indeed this change makes absolutely no difference, I hope KW is not so dumb to be deceived by it.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably the only reason name could be NULL is because some malloc failed. In that very unlikely case the program should just stop and not print garbage. Please use some kind of assert either here or probably better inside log_vasprintf() (in case KW can understand that)
|
I just noticed this code does not even compile: https://github.com/thesofproject/sof/actions/runs/3639665715/jobs/6143338384 Yet Klocwork does not mind...? |
|
Converted to draft, fixing typos and warnings. Pushed in hurry as we are in hurry with release. Now, I would like to elaborate on following comments:
And those are related specifically to this case: I assume you know what a variable scope is.
means that it ends function scope. Now, let's go to the case where return statement returns the expression (emphasis mine): This means, that the function scope ends with return statement. This means following: This means, that function arguments passed to func_b, if they are pointer types which are copied by value (in C++ there are references whose lifetime is expanded in this situation!), then those are dangling pointers - a pointers to local variables of a func_a that are outside of the scope during execution of func_b. You pass a is not the same as: Compiler might optimize it if
Also, fact of compiler optimization does not allow developers to break coding rules of the language! Next time you write something not very kind @paulstelian97 please make sure you know what you talk about. |
|
The problem with your example here is that CAN be optimized to generate the exact same code as The C standard doesn't guarantee any local variable actually gets a stack frame allocation (unless you take its address and that address isn't optimized out for some different reason) Also... "dangling pointer" I feel like it doesn't mean what you think it means. version_fd is a field of the thing pointed at by "global_config", which doesn't get automatically freed in any way (global_config doesn't point to anywhere on the stack), as such version_fd remains valid (if it's a pointer in the first place anyway). You can't have dangling pointers if the memory they point to isn't freed (and thus remains valid) -- a pointer to valid memory cannot by definition be a dangling pointer. The return statement ends the execution of the function after the expression has been evaluated (so it knows what value to return). I challenge you to support everything you claim about C with actual quotes from the standard (or from a public copy of said standard).
Well some of the statements you mention are NOT rules of the language (in C specifically -- they would be in other languages but C on purpose leaves some things more vague in order to enable optimizers to be powerful -- for example, you DON'T get a guarantee that local variables are allocated anywhere -- they can be in a register or outright missing if unused) I do apologize if I come off as rude, but I have to point out that C isn't e.g. Java, where the compiled code does mostly reflect the source code unlike in C where it only really does so with -O0. |
Default GCC asserts does not work with -NDEBUG flag and I would like it to work with production code when memory allocation fails. Including sof/debug/panic.h fails also since logger does not have architecture defined (its not a platform so...). |
|
This is a tool, so for error handling you can probably print an error message and call the exit() [EDIT: or abort() might be more appropriate?] function. I have looked at your past commit history, this is your first time in C code (unless I'm missing some old work -- EDIT: I did find a few basically trivial commits such as a missing cast that did get accepted) on this repo. Sorry for making the "welcome" be so rough; I just have to point out that C doesn't behave pretty much at all like Python and there's a LOT of gotchas that seem to have stumped you when sending this pull request. |
c4d6466 to
11e7aeb
Compare
paulstelian97
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still not great, but a bit better than last time. See individual comments.
tools/logger/convert.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now the issue is still weird -- why do you set a local variable to NULL? That thing can get optimized out and you still get the same code. If "config" is a global variable (so that this change does become meaningful) disregard.
tools/logger/logger.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is the only meaningfully good change I can see.
@paulstelian97 what are you talking about? |
Well I did not see the quality of your code anyway. Still, maybe you're accustomed to embedded compilers which... might be of lower quality than compilers that do userspace Linux (or Windows, they aren't much worse on Windows) programs. Anyway you claim that some things are meaningful when my experience says otherwise so I would like you to tell me exactly why those things are meaningful. |
11e7aeb to
2db7e5d
Compare
I'll gladly elaborate on this on Saturday in my free time since we need to finish release asap and production world will not be happy that I prolong the process due to scientific debates :-) |
|
Next iteration:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I'm gonna allow these changes (there's no longer any actively-breaking change, just two changes that prevent crashes and a good chunk of changes that do nothing). They are still meaningless (you trust the tool more than the experts, which is why I can't outright approve these changes) AND have not truly made an attempt to explain why the tool is right over the experts for most of these changes. I will no longer block the merge however.
I suggest you use more well known static analysis tools. Coverity is an example. And when a tool reports an error, I suggest you get a full understanding of what it's saying to be able to evaluate whether it's right or wrong, rather than trust it (I always do that, rather than blindly trust the tools -- I do my best to understand its point and if I fail to understand I just... do more research to try to figure it out, rather than trust that it's right). No tool gets all things right (the best tools will just never miss, but will report errors that aren't errors)
You are not going to allow anything. Your review had been dismissed and this is not SOF main branch but a product release branch. You are not only maintainer here and I think @lgirdwood should do the review. You should certainly be more careful with what you say and do as a maintainer of this project. Further comments on my developer career I will take as an offence. |
|
I will not make comments on your developer career, that I can agree with. And while I don't maintain this branch, I believe that marc does, and Liam isn't the only maintainer that can deal with these branches. I'm cool with Intel guys commenting and reviewing patches on i.MX stable branches just fine, and WILL take into account what they are saying. You don't seem to take stuff into account from me OR from marc, so I strongly doubt Liam will be of a different opinion. The only real difference is he'll be more polite than myself in saying most of these changes are either pointless or wrong. |
2db7e5d to
1d60d3a
Compare
1d60d3a to
8effda0
Compare
Files tools\logger\logger.c and tools\logger\convert.c had been verified for issues and had been fixed. Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
8effda0 to
93586d6
Compare
|
After consultations with team turns out that logger is not part of the release for MTL, closing |
|
I'm closing this PR |
Unacceptable tone @aborisovich. This is not how we work with partners and colleagues. |
This is an interesting C quizz. Please correct me but I think and hope your long and illustrated explanation can be summarized with just these 3 words: "tail call optimization"? I would naively trust compilers to "do the Right Thing" and NOT perform any tail call optimization when the function argument contains any reference to the local stack but maybe that's too naive. C is a terrifying minefield of unintuitive and undefined behaviors so maybe this is yet another one of them? https://stefansf.de/c-quiz/ https://www.bing.com/search?q=linus+torvalds+gcc, etc. Also, maybe most compilers are smart enough but a few are not (or they are buggy) and Klocwork is being very conservative and erring on the "false positive" side? It would suit its role. I'm also a bit sceptical that a (too?) simple intermediate variable trick is enough to defeat tail call optimization, now that seems a bit too naive too? Anyway I see sof-logger is removed in that branch by #6754 and that's OK but you still need to fix the KW process because there will always be other false positives that must be disposed (after very careful team review) and it won't always be possible to change or delete the code to avoid them. Static analyzers always produce false positives which is why they all have a process to ignore some warnings and KW is no exception.
I'm looking forward to Saturday as long as we all focus 100% on the technical questions and stop all personal comments! https://www.sofproject.org/community/ |
|
Interesting experimentations: https://quuxplusone.github.io/blog/2021/01/09/tail-call-optimization/ tl;dr: some compilers perform tail call optimizations but they stop when things "get too complicated", likely out of fear of breaking things (as usual with most optimizations and undefined behaviors) https://godbolt.org/z/vcY3v9 - See assembly generated "live" by a few compilers |
|
BTW different static analyzers find different bugs and the CMake maintainer in Zephyr has submitted a pull request to support any of them generically: This is very promising, looking forward to it! Different static analyzers also emit different false positives that may conflict with each other: write the code one way and analyzer 1 complains, write it differently and now analyzer 5 complains! "Static analyzer whack-a-mole" gets nowhere fast. It's OK to simplify the code to help all analyzers (and humans!) understand it. It's not OK to make it more complicated to silence a single analyzer. That's why many static analyzers offer a way to filter out false positives (after very careful team review). |
Good catch.
|
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com>
|
Making global_config truly global with a minimal number of lines changed in: |
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 327a26b) Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 2dfaee6) Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 327a26b) Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 2dfaee6) Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 327a26b)
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see thesofproject#6858 and thesofproject#6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 2dfaee6)
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not finish, leaving behind a supposedly "global" variable that is actually a confusing global pointer to a struct local to the main() function. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because the main()'s stack lifespan is the same as a global but let's simplify things anyway. Also stop using 'extern' in .c files, use a proper .h file instead. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 327a26b)
Finish the job that commit 5b29dae ("logger: Create global convert_config variable to avoid spaghetti code.") started but did not complete, leaving a confusing mix of globals and locals. This confuses some static analyzer complaining that stack values are being returned, see #6858 and #6738. This is a false positive because convert's() stack lifespan is practically the same as a global but let's simplify things anyway. Signed-off-by: Marc Herbert <marc.herbert@intel.com> (cherry picked from commit 2dfaee6)







Files tools\logger\logger.c and tools\logger\convert.c had been verified for issues and had been fixed.
Signed-off-by: Andrey Borisovich andrey.borisovich@intel.com