Skip to content

Conversation

@cool-RR
Copy link
Contributor

@cool-RR cool-RR commented Mar 10, 2020

I recently went over Matplotlib and Pandas, fixing a small mistake in the way that Python 3's exception chaining is used. If you're interested, I can do it here too. I've done it on just one file right now.

The mistake is this: In some parts of the code, an exception is being caught and replaced with a more user-friendly error. In these cases the syntax raise new_error from old_error needs to be used.

Python 3's exception chaining means it shows not only the traceback of the current exception, but that of the original exception (and possibly more.) This is regardless of raise from. The usage of raise from tells Python to put a more accurate message between the tracebacks. Instead of this:

During handling of the above exception, another exception occurred:

You'll get this:

The above exception was the direct cause of the following exception:

The first is inaccurate, because it signifies a bug in the exception-handling code itself, which is a separate situation than wrapping an exception.

Let me know what you think!

@tensorflow-bot tensorflow-bot bot added the size:S CL Change Size: Small label Mar 10, 2020
@gbaned gbaned self-assigned this Mar 11, 2020
@gbaned gbaned requested a review from mrry March 11, 2020 03:12
@mrry
Copy link
Contributor

mrry commented Mar 11, 2020

@cool-RR Thanks for this change! Am I right in thinking that this chaining syntax is only available in Python 3? Would six.raise_from() work in this case?

@martinwicke Do we have a timeline for accepting changes that use Python 3 features?

@cool-RR
Copy link
Contributor Author

cool-RR commented Mar 11, 2020

@mrry You're welcome, yes and yes.

@mrry
Copy link
Contributor

mrry commented Mar 11, 2020

Great! Would you mind switching the PR to use six.raise_from() and then I'd be happy to approve the change?

@cool-RR cool-RR force-pushed the 2020-03-10-raise-from branch from a5b332a to 6209b5d Compare March 11, 2020 15:01
@cool-RR
Copy link
Contributor Author

cool-RR commented Mar 11, 2020

Yep, done. Just curious though, looking here it says that Python 3.5+ is required. What reason is there to avoid native Python 3 syntax like raise foo from bar?

mrry
mrry previously approved these changes Mar 11, 2020
@tensorflow-bot tensorflow-bot bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Mar 11, 2020
@mrry
Copy link
Contributor

mrry commented Mar 11, 2020

Thanks! Yes, I was hoping @martinwicke could clarify our policy on that. Even though we are no longer building PIP packages for Python 2.7, I believe we still have a few non-PIP users that are stuck on that version of Python, which prevents us from adopting new syntax (just yet).

@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Mar 11, 2020
@cool-RR
Copy link
Contributor Author

cool-RR commented Mar 11, 2020

@mrry: I understand. I'll wait with fixing this problem in the entire project until after you've moved to Python 3 completely, so we could switch straight to raise foo from bar. (Assuming I won't be too busy then.) Once you've left Python 2 completely, feel free to send me a message here.

@gbaned gbaned added stat:awaiting tensorflower Status - Awaiting response from tensorflower and removed ready to pull PR ready for merge process labels Mar 19, 2020
@mrry mrry added stat:awaiting response Status - Awaiting response from author and removed stat:awaiting tensorflower Status - Awaiting response from tensorflower labels Apr 10, 2020
@gbaned
Copy link
Contributor

gbaned commented Apr 24, 2020

@cool-RR Any update on this PR, please. Thanks!

@cool-RR
Copy link
Contributor Author

cool-RR commented Apr 24, 2020

@gbaned See my last message.

@tensorflowbutler tensorflowbutler removed the stat:awaiting response Status - Awaiting response from author label Apr 26, 2020
@gbaned
Copy link
Contributor

gbaned commented Apr 27, 2020

@mrry Any update on this PR, please. Thanks!

@mrry
Copy link
Contributor

mrry commented Apr 27, 2020

@gbaned: See @cool-RR's last message. This PR is on hold until we get confirmation that TensorFlow can use Python 3–specific features.

@mrry mrry marked this pull request as draft May 1, 2020 17:28
@gbaned
Copy link
Contributor

gbaned commented Jul 14, 2020

@mrry, @cool-RR This PR is in draft, any update on this? Please. Thanks!

@cool-RR cool-RR marked this pull request as ready for review July 14, 2020 10:36
@cool-RR
Copy link
Contributor Author

cool-RR commented Jul 14, 2020

@gbaned I moved this PR to ready state now. As far as I'm concerned we can move forward, just need confirmation that Python 2 support is dropped in TensorFlow.

Here's a blog post I made where I explained my effort to push this Python 3 syntax in many open-source packages: https://blog.ram.rachum.com/post/621791438475296768/improving-python-exception-chaining-with

There's now a PyLint rule that checks for this syntax.

@gbaned gbaned added the awaiting review Pull request awaiting review label Jul 17, 2020
@gbaned
Copy link
Contributor

gbaned commented Jul 24, 2020

@mrry Can you please take a look on the above comment from @cool-RR. Thanks!

@martinwicke martinwicke self-requested a review August 3, 2020 06:29
@martinwicke martinwicke added the kokoro:force-run Tests on submitted change label Aug 3, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2020
@martinwicke
Copy link
Member

@cool-RR We are soooo close to being able to actually using Python3-only syntax. I'm re-running the tests and I will merge this if I can using the six as is.

@ematejska FYI this would make a good test case whether we can really break Py2.

martinwicke
martinwicke previously approved these changes Aug 3, 2020
@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Aug 3, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2020
@gbaned gbaned added kokoro:force-run Tests on submitted change and removed awaiting review Pull request awaiting review labels Aug 3, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2020
@gbaned gbaned added the kokoro:force-run Tests on submitted change label Aug 3, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2020
@gbaned
Copy link
Contributor

gbaned commented Aug 3, 2020

@cool-RR Can you please check build failures. Thank you!

@gbaned gbaned removed the ready to pull PR ready for merge process label Aug 3, 2020
@cool-RR cool-RR dismissed stale reviews from martinwicke and mrry via 72276a1 August 3, 2020 13:20
@cool-RR cool-RR force-pushed the 2020-03-10-raise-from branch from 6209b5d to 72276a1 Compare August 3, 2020 13:20
@cool-RR
Copy link
Contributor Author

cool-RR commented Aug 3, 2020

@gbaned I rebased and pushed. It's been 4 hours and the CI hasn't started running yet. Is this normal?

@martinwicke martinwicke added the kokoro:force-run Tests on submitted change label Aug 3, 2020
@martinwicke
Copy link
Member

triggered it now

@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2020
@gbaned gbaned added the kokoro:force-run Tests on submitted change label Aug 4, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 4, 2020
@gbaned
Copy link
Contributor

gbaned commented Aug 4, 2020

@cool-RR Still, build failures are appearing, can you please fix those?. Thank you!

@cool-RR
Copy link
Contributor Author

cool-RR commented Aug 4, 2020

Looks like you have a test test_propagates_exception_trace that checks things I don't understand about the exception traceback. Looks like @yifeif wrote it. Yifey, if you have a solution here, let me know. Otherwise this is getting too hairy for me.

@yifeif
Copy link
Contributor

yifeif commented Aug 4, 2020

Took a brief look, and seems like six changed the exception string, and you might need to update the test to reflect that. Here is the current error from CI

AssertionError: '<string>' not found in ['session.py', 'monitored_session.py'] : The exception was raised from an unrecognized file. This unit test probably needs to be updated. Traceback:
[<FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/training/monitored_session_test.py, line 750 in test_propagates_exception_trace>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/training/monitored_session.py, line 1384 in run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/six_archive/six.py, line 703 in reraise>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/training/monitored_session.py, line 1369 in run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py, line 1689 in run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/client/session.py, line 976 in run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/client/session.py, line 1202 in _run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/client/session.py, line 1380 in _do_run>, <FrameSummary file /b/f/w/bazel-out/k8-opt/bin/tensorflow/python/monitored_session_test.runfiles/org_tensorflow/tensorflow/python/client/session.py, line 1407 in _do_call>, <FrameSummary file <string>, line 3 in raise_from>]

Here is where the test was getting the name of the file exception was raised from (where the change needs to be made),

@cool-RR
Copy link
Contributor Author

cool-RR commented Aug 4, 2020

I'm gonna bow out, this is getting too complicated for me.

@cool-RR cool-RR closed this Aug 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes size:S CL Change Size: Small

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants