[in progress] python 2 and 3 compatibility without 2to3 by Carreau · Pull Request #2095 · ipython/ipython

Carreau · 2012-07-04T12:33:11Z

Logical step after #2064 where the discussion started.

Most of the fixes apply by 2to3 when converting the current source tree to python 3 can be applied and are valid python 2.6 and 2.7 (not 2.5 but we don't support it), with minimal changes.

The plan is here to apply theses changes as early as possible ans deactivate them when using 2to3 in order to get a python2 and python3 compatible IPython code.

For what I've seen right now, 2to3 is sometime a little overzealous with print, converting
print 'xxx', to print('xxx', end=' ')(space) but in some place we should have end=''(no space),
and it sometime also change print ('an'+'expression') to print(('an'+'expression')). I detected and fixed some by hand because of failling test, but might have missed a few.

The other 2 fixes I added was getting rid of apply in some places, and convert exec statement to a function.

My guess is the harder part will be unicode strings prefix that is not valid python 3.0 to 3.2 and valid again python 3.3. One of our option might be from __future__ import unicode_literals.

It would be really cool to be able to do python3 ipython.py

Note : This does not merge cleanly right now and is on top of #2064, we can either close #2064 then i'll rebase the all on top of master, or I'll rebase when #2064 is merged.

brutally replace all `exeption <type>, <name>:` by `exception <type> as <name> :` `exception <type>, <type> :` should not be present anywhere in the code anymore, or should be present with explicit tuple as `exception (<type>, <type>)`

2to3 makes a better job than regexp

this should allow a more compatible codebase between python 2.x and 3.x

takluyver · 2012-07-04T15:12:33Z

Thanks, I've been thinking of aiming for this.

I think exec is actually the trickiest bit - the function form won't work under Python 2 if you're specifying the namespaces, as far as I know. The workaround would be to declare our own exec_ function, like six does: https://bitbucket.org/gutworth/six/src/0c712d55e12c/six.py#cl-297 (In fact, we could use six, since our own py3compat module duplicates quite a bit of its functionality).

For unicode strings, I think the approach Django took was to use unicode_literals, and then use b'bytes', 'unicode' and n('native string'), as there are relatively few cases where the latter is needed. So long as your native strings are all ascii, n = str works.

Carreau · 2012-07-04T15:44:35Z

I removed the exec fix, i can push this branch onto the main repository so that everyone can have acces to it.

minrk · 2012-07-04T18:09:27Z

Please do not do this. Maintaining Python3 compatible source in Python 2 is an ugly pain.

takluyver · 2012-07-04T18:26:00Z

I'm not sure it's necessarily much worse, given that we're only
supporting >= 2.6 anyway, and we've already got compatibility
functions around. The compatible source approach is increasingly
popular.

I suggest we just progress one step at a time, as we've done with the
"except/as" syntax. As we get to the trickier changes, we can decide
whether the benefits outweigh the costs.

minrk · 2012-07-04T18:26:42Z

My main objection is the print function, which I really, really hate.

minrk · 2012-07-04T18:35:44Z

Counterpoints to my own argument:

python3 setupegg.py develop would be awesome.
Our current occasional print_function use is inconsistent.

But I still would vastly prefer to not apply print_function everywhere. At least not yet.

takluyver · 2012-07-04T18:40:14Z

My own plan was to apply it first in places where we're using the
print >>foo syntax, because I find that particularly ugly. But in
other cases, the difference is just a couple of parentheses, so I
don't have a problem with using it.

bfroehle · 2012-07-04T20:00:25Z

The extra parenthesis here (and a couple lines below) are not needed.

bfroehle · 2012-07-04T20:22:56Z

I'd definitely be in favor of this. Currently developing IPython for Python 3 is a pain as any change you make requires at least running python3 setup.py build.

As an aside, Python 3 is starting to reach such a critical mass of supported packages at the distribution level (i.e., in Debian Wheezy / Ubuntu Precise) that I'm now finding myself often using Python 3 as my go-to Python intepreter.

takluyver · 2012-07-04T20:31:38Z

That's good to hear. I know Ubuntu is also pushing to port core
applications for Quantal, and with Python 3.3 on the way, it's picking
up speed rapidly.

bfroehle · 2012-07-04T20:34:34Z

Aside: if only matplotlib could get their act together and create a new release...

minrk · 2012-07-04T20:47:04Z

My pattern has been similar to @takluyver's - when more complicated behavior is called for, I add print_function, but not before. I think to do so across all of IPython is a huge net detriment.

takluyver · 2012-07-04T21:01:27Z

Shall we start by doing these things that I think are uncontroversial:

backticks --> repr()
Remove the couple of uses of apply()
Use print_function in modules where we have the print >>foo syntax only.

I suggest we close this PR and open a new one for those changes - this one is unwieldy, and already has a merge conflict.

We could also express a preference that new modules added to the codebase should use print_function.

minrk · 2012-07-04T21:23:31Z

@takluyver's suggestion seems sensible to me. code-wide print_function is a major style change, and code-wide unicode_literals will likely be a big mess (and I expect not worth it, and we should just wait for 3.3 on that one). But this PR includes several helpful little changes, which will be dwarfed by the discussion of others.

Carreau · 2012-07-05T11:56:10Z

ok, closing, and reopening as #2100 with @takluyver sugestion.

fperez · 2012-07-05T17:01:59Z

Closing (I think you forgot to actually close it, @Carreau :)

Carreau added 18 commits June 30, 2012 18:34

conform to pep 3110

28b538a

brutally replace all `exeption <type>, <name>:` by `exception <type> as <name> :` `exception <type>, <type> :` should not be present anywhere in the code anymore, or should be present with explicit tuple as `exception (<type>, <type>)`

fix more exeptions

90c0a4e

2to3 makes a better job than regexp

disable some specific 2to3 fixes whith setup.py

1c49f84

this should allow a more compatible codebase between python 2.x and 3.x

fix english in comment

3f213ee

use print function in magics

367bc5e

use print function in core

692a6f1

use print function in frontends

b4a2a56

use print function in configurabel

2c7e6fd

use print function in extensions

7059650

use print function in parallel

04668c0

use print function in lib

9ead380

use print function in utils

83956b7

use print function in testing

f748063

use print function in external

2733c58

use print function in zmq

98e4377

use print function in docs

8eb3311

use print function in tools

ca8dad6

use 2to3 apply fix, apply is deprecated since 2.3

22d9a77

Carreau added 3 commits July 4, 2012 17:33

use repr fix on core

a2ebc54

disable some 2to3 fix that are not needed

0173bef

use funcattrs fix

30b89d1

bfroehle reviewed Jul 4, 2012
View reviewed changes

Carreau mentioned this pull request Jul 5, 2012

[in progress] python 2 and 3 compatibility without 2to3, second try #2100

Merged

fperez closed this Jul 5, 2012

This was referenced Jul 13, 2012

Apply 2to3 next fix. #2134

Merged

2to3: Apply has_key fixer. #2140

Merged

bfroehle mentioned this pull request Sep 27, 2012

IPEP 4: Python 3 Compatibility #2440

Closed

Uh oh!

Conversation

Carreau commented Jul 4, 2012

Uh oh!

takluyver commented Jul 4, 2012

Uh oh!

Carreau commented Jul 4, 2012

Uh oh!

minrk commented Jul 4, 2012

Uh oh!

takluyver commented Jul 4, 2012

Uh oh!

minrk commented Jul 4, 2012

Uh oh!

minrk commented Jul 4, 2012

Uh oh!

takluyver commented Jul 4, 2012

Uh oh!

bfroehle Jul 4, 2012

Choose a reason for hiding this comment

Uh oh!

bfroehle commented Jul 4, 2012

Uh oh!

takluyver commented Jul 4, 2012

Uh oh!

bfroehle commented Jul 4, 2012

Uh oh!

minrk commented Jul 4, 2012

Uh oh!

takluyver commented Jul 4, 2012

Uh oh!

minrk commented Jul 4, 2012

Uh oh!

Carreau commented Jul 5, 2012

Uh oh!

fperez commented Jul 5, 2012

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants