-
-
Notifications
You must be signed in to change notification settings - Fork 33.7k
gh-123171: Fix error paths in ast_opt.c for for_iter optimisations #123172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Python/ast_opt.c
Outdated
| if (newval == NULL) { | ||
| return 0; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fold_tuple, this call is immediately followed by make_const which then clears any exception that is not PyExc_KeyboardInterrupt, and then returns 1. So my question is: should we actually clear the error or not in this case as well? (or maybe patch fold_tuple in this case?).
More generally, calls to make_const are not preceeded by newval == NULL checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that make_const does error checking for one of their arguments and callers are relying on that behaviour is not only error prone but it's also very confusing. If we fail to create one of this objects we should propagate the error (think about MemoryError or something worse, we surely don't want to clean that).
In other words, calls to make_const are not preceeded by newval == NULL checks.
I think that's fundamentally flawed. This code was moved from the time this happened in peephole.c (and even before, if you track it down it goes as back as 00ee7ba) and was clearing out all exceptions in the peephole after compilation and special casing KeyboardInterrupt which it's masking real errors and can actually crash if the caller is left in an inconsistent state.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should clean this up in another PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I just wanted to address the test failures (which I think they could be related to this). But I agree that we shouldn't clear errors otherwise. I was more concerned about: "should we fix calls to make_const here or separately?".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that
make_constdoes error checking for one of their arguments and callers are relying on that behaviour is not only error prone but it's also very confusing. If we fail to create one of this objects we should propagate the error (think aboutMemoryErroror something worse, we surely don't want to clean that).In other words, calls to make_const are not preceeded by newval == NULL checks.
I think that's fundamentally flawed. This code was moved from the time this happened in
peephole.c(and even before, if you track it down it goes as back as 00ee7ba) and was clearing out all exceptions in the peephole after compilation and special casingKeyboardInterruptwhich it's masking real errors and can actually crash if the caller is left in an inconsistent state.
Let's think about one concrete case: 1 / 0. There's ZeroDivisionError is being raised.
Should we propagate it? That's a really a complex question.
If you don't mind, I would like to work on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should always propagate. If there is an exception we screwed up somewhere and the code should defend against it otherwise. Reliying on a catch-all it's bad because this is basically ignoring anything inheriting from BaseException, including SystemErrors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here the other problem really is that make_const_tuple it's returning NULL without the error set :(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have made here a big refactor so make_const doesn't eat errors and so exceptions that are explicitly ignored are the only thing that's ignored
|
converting to draft meanwhile I try to address test failures |
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
|
Is this the only way to fix an error (whatever it is)? Currently the complexity of errors handling is hidden in What exactly issues with the current code? I will try to solve them in other way. |
The first issue is that that's very confusing because if you don't know that A reviewer that doesn't know this weird behaviour about The second thing it's that is capturing everything and clearing it (except KeyBoardInterrupt) which isn't granular at all and it will hide other things like
It just surfaces existing complexity and makes the error paths more obvious. It's like if you say that doing instead of it's adding a lot of complexity. Sure there is more code, but you are being explicit about what's being ignored and why (like the @serhiy-storchaka OTOH I don't feel very strongly about this so If you still think you prefer the current code anyway I'm happy to close this but |
|
This is not an antipattern. If there is some code repeated multiple times, you can factor out it into a function. And if this is the same check of argument before every call of the function, it can be moved into that function. This simplifies multiple caller sites and guarantees that the check will be not missed. Such pattern is not common in the public C API, this is why it can be confusing, but it is used for convenience if this is appropriate (see for example Usually it is more correct to check for exceptions types that should be silenced (OverflowError, MemoryError, RecursionError, whatever) instead of exceptions types that should not be silenced (KeyboardInterrupt), but what other exceptions can be raised here? There is a limited set of exceptions that can be raised here, and only KeyboardInterrupt should not be silenced. If we a wrong and missed some type of very rare exception that should be silenced, we do not want to throw it in the face of a user. I do not know any other exception besides KeyboardInterrupt that should not be ignored. So checking for KeyboardInterrupt is not only practical (less code), it is more error-proof. I prefer the current code. |
I don't see this as refactoring common code, the function is called What's more, we already have problems with functions that swallor errors and new functions had to be introduced to fix the confision for users (see
At the very least any of the calls that create Python objects can raise
I disagree, in the first case ( What also worries me more is that there may be new exceptions appearing after refactors that are going to be ignored and the optimisations may not be performed and we will not find out easily unless the tests are triggering those problems as well, which is unlikely. |
|
I'm closing this, I don't feel that passionate about it and I trust @serhiy-storchaka criteria |
Uh oh!
There was an error while loading. Please reload this page.