Skip to content

Commit 93b47cb

Browse files
rockyrocky
committed
Keeping up with the masters
1 parent b52f341 commit 93b47cb

File tree

17 files changed

+703
-244
lines changed

17 files changed

+703
-244
lines changed

‎ChangeLog‎

Lines changed: 413 additions & 101 deletions
Large diffs are not rendered by default.

‎NEWS‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,35 @@
1-
uncompyle6 2.832 2016-09-11 live from NYC!
1+
uncompyle6 2.9.2 2016-10-15
2+
3+
- use source-code line breaks to assist in where to break
4+
in tuples and maps
5+
- Fix Python 1.5 decompyle bugs
6+
- Fix some Python 2.6 and below bugs
7+
- DRY fragments.py code a little
8+
9+
uncompyle6 2.9.1 2016-10-09
10+
11+
- Improved Python 1.5 decompiling
12+
- Handle old-style pre Python 2.2 classes
13+
14+
uncompyle6 2.9.0 2016-10-09
15+
16+
- Use xdis 3.0.0 protocol load_module.
17+
this Forces change in requirements.txt and _pkg_info_.py
18+
- Start Python 1.5 decompiling; another round of work is needed to
19+
remove bugs
20+
- Simpify python 2.1 grammar
21+
- Fix bug with -t ... Wasn't showing source text when -t option was given
22+
- Fix 2.1-2.6 bug in list comprehension
23+
24+
uncompyle6 2.8.4 2016-10-08
25+
26+
- Python 3 disassembly bug fixes
27+
- Python 3.6 fstring bug fixes (from moagstar)
28+
- Python 2.1 disassembly
29+
- COME_FROM suffixes added in Python3
30+
- use .py extension in verification disassembly
31+
32+
uncompyle6 2.8.3 2016-09-11 live from NYC!
233

334
NOTE: this is possibly the last release before a major reworking of
435
control-flow structure detection is done.

‎README.rst‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ Introduction
1111
------------
1212

1313
*uncompyle6* translates Python bytecode back into equivalent Python
14-
source code. It accepts bytecodes from Python version 2.2 to 3.6 or
15-
so, including PyPy bytecode.
14+
source code. It accepts bytecodes from Python version 2.1 to 3.6 or
15+
so, including PyPy bytecode and Dropbox's Python 2.5 bytecode.
1616

1717
Why this?
1818
---------
@@ -45,7 +45,7 @@ Requirements
4545

4646
This project requires Python 2.6 or later, PyPy 3-2.4, or PyPy-5.0.1.
4747
The bytecode files it can read has been tested on Python bytecodes from
48-
versions 2.2-2.7, and 3.2-3.6 and the above-mentioned PyPy versions.
48+
versions 2.1-2.7, and 3.2-3.6 and the above-mentioned PyPy versions.
4949

5050
Installation
5151
------------
@@ -96,22 +96,35 @@ For usage help:
9696
Known Bugs/Restrictions
9797
-----------------------
9898

99-
Python 2 deparsing decompiles and about 90% verifies from Python 2.3.7 to Python
99+
About 90% of the decompilation verifies from Python 2.3.7 to Python
100100
3.4.2 on the standard library packages I have on my system.
101101

102102
(Verification is the process of decompiling bytecode, compiling with a
103103
Python for that byecode version, and then comparing the byetcode
104104
produced by the decompiled/compiled program. Some allowance is made
105-
for inessential differences.)
105+
for inessential differences, but other semantically equivalent
106+
differences are not caught.)
106107

107108
Later distributions average about 200 files. At this point, 2.7
108-
decompilation is better than uncompyle2. A number of bugs have been
109-
fixed.
109+
decompilation is definitely better than uncompyle2. A number of bugs
110+
have been fixed. We now handle more Python bytecodes than the old
111+
decompyle program that handled Python bytecodes ranging from 1.5 to
112+
2.4. There is some work do do on the lower end which is more
113+
difficult for us since we don't have a Python interpreter for versions
114+
1.5, 1.6 or 2.0.
110115

111116
Python 3.5 largely works, but still has some bugs in it.
112117
Python 3.6 changes things drastically by using word codes rather than
113118
byte codes, and that needs to be addressed.
114119

120+
Currently not all Python magic numbers are supported. Specifically in
121+
some versions of Python, notably Python 3.6, the magic number has
122+
changes several times within a version. We support only the released
123+
magic. There are also customized Python interpreters, notably Dropbox,
124+
which use their own magic and encrypt bytcode. With the exception of
125+
the Dropbox's old Python 2.5 interpreter this kind of thing is not
126+
handled.
127+
115128
There is lots to do, so please dig in and help.
116129

117130
See Also

‎__pkginfo__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838
]}
3939
ftp_url = None
4040
install_requires = ['spark-parser >= 1.4.0',
41-
'xdis >= 2.3.0']
41+
'xdis >= 3.1.0']
4242
license = 'MIT'
4343
mailing_list = 'python-debugger@googlegroups.com'
4444
modname = 'uncompyle6'

‎requirements.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
spark-parser >= 1.4.0
2-
xdis >= 2.2.2
1+
# Pick up stuff from setup.py
2+
-e .

‎uncompyle6/bin/uncompile.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ def main_bin():
144144
usage()
145145

146146
if outfile == '-':
147-
if 'do_verify' in options and len(files) == 1:
147+
if 'do_verify' in options and options['do_verify'] and len(files) == 1:
148148
junk, outfile = tempfile.mkstemp(suffix=".pyc",
149149
prefix=files[0][0:-4]+'-')
150150
else:

‎uncompyle6/disas.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,8 @@ def disassemble_file(filename, outstream=None, native=False):
7777
return
7878

7979
filename = check_object_path(filename)
80-
version, timestamp, magic_int, co, is_pypy = load_module(filename)
80+
(version, timestamp, magic_int, co, is_pypy,
81+
source_size) = load_module(filename)
8182
if type(co) == list:
8283
for con in co:
8384
disco(version, con, outstream)

‎uncompyle6/main.py‎

Lines changed: 23 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,19 @@
11
from __future__ import print_function
22
import datetime, os, sys
33

4-
from uncompyle6 import verify, PYTHON_VERSION, IS_PYPY
4+
from uncompyle6 import verify, IS_PYPY
55
from xdis.code import iscode
66
from uncompyle6.disas import check_object_path
77
from uncompyle6.semantics import pysource
88
from uncompyle6.parser import ParserError
9+
from uncompyle6.version import VERSION
910

1011
from xdis.load import load_module
1112

1213
def uncompyle(
13-
version, co, out=None, showasm=False, showast=False,
14+
bytecode_version, co, out=None, showasm=False, showast=False,
1415
timestamp=None, showgrammar=False, code_objects={},
15-
is_pypy=False, magic_int=None):
16+
source_size=None, is_pypy=False, magic_int=None):
1617
"""
1718
ingests and deparses a given code block 'co'
1819
"""
@@ -22,21 +23,26 @@ def uncompyle(
2223
real_out = out or sys.stdout
2324
co_pypy_str = 'PyPy ' if is_pypy else ''
2425
run_pypy_str = 'PyPy ' if IS_PYPY else ''
25-
print('# %sPython bytecode %s%s disassembled from %sPython %s' %
26-
(co_pypy_str, version,
26+
print('# uncompyle6 version %s\n'
27+
'# %sPython bytecode %s%s\n# Disassembled from: %sPython %s' %
28+
(VERSION, co_pypy_str, bytecode_version,
2729
" (%d)" % magic_int if magic_int else "",
28-
run_pypy_str, PYTHON_VERSION),
29-
file=real_out)
30+
run_pypy_str, '\n# '.join(sys.version.split('\n'))),
31+
file=real_out)
3032
if co.co_filename:
3133
print('# Embedded file name: %s' % co.co_filename,
3234
file=real_out)
3335
if timestamp:
3436
print('# Compiled at: %s' % datetime.datetime.fromtimestamp(timestamp),
3537
file=real_out)
38+
if source_size:
39+
print('# Size of source mod 2**32: %d bytes' % source_size,
40+
file=real_out)
3641

3742
try:
38-
pysource.deparse_code(version, co, out, showasm, showast, showgrammar,
39-
code_objects=code_objects, is_pypy=is_pypy)
43+
pysource.deparse_code(bytecode_version, co, out, showasm, showast,
44+
showgrammar, code_objects=code_objects,
45+
is_pypy=is_pypy)
4046
except pysource.SourceWalkerError as e:
4147
# deparsing failed
4248
print("\n")
@@ -55,7 +61,8 @@ def uncompyle_file(filename, outstream=None, showasm=False, showast=False,
5561

5662
filename = check_object_path(filename)
5763
code_objects = {}
58-
version, timestamp, magic_int, co, is_pypy = load_module(filename, code_objects)
64+
(version, timestamp, magic_int, co, is_pypy,
65+
source_size) = load_module(filename, code_objects)
5966

6067

6168
if type(co) == list:
@@ -65,7 +72,8 @@ def uncompyle_file(filename, outstream=None, showasm=False, showast=False,
6572
is_pypy=is_pypy, magic_int=magic_int)
6673
else:
6774
uncompyle(version, co, outstream, showasm, showast,
68-
timestamp, showgrammar, code_objects=code_objects,
75+
timestamp, showgrammar,
76+
code_objects=code_objects, source_size=source_size,
6977
is_pypy=is_pypy, magic_int=magic_int)
7078
co = None
7179

@@ -118,7 +126,10 @@ def _get_outstream(outfile):
118126
elif out_base is None:
119127
outstream = sys.stdout
120128
else:
121-
outfile = os.path.join(out_base, filename) + '_dis'
129+
if filename.endswith('.pyc'):
130+
outfile = os.path.join(out_base, filename[0:-1])
131+
else:
132+
outfile = os.path.join(out_base, filename) + '_dis'
122133
outstream = _get_outstream(outfile)
123134
# print(outfile, file=sys.stderr)
124135

‎uncompyle6/parsers/parse22.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,10 @@ def __init__(self, debug_parser=PARSER_DEFAULT_DEBUG):
1414
def p_misc22(self, args):
1515
'''
1616
_for ::= LOAD_CONST FOR_LOOP
17+
list_iter ::= list_if JUMP_FORWARD
18+
COME_FROM POP_TOP COME_FROM
19+
list_for ::= expr _for designator list_iter CONTINUE JUMP_FORWARD
20+
COME_FROM POP_TOP COME_FROM
1721
'''
1822

1923
class Python22ParserSingle(Python23Parser, PythonParserSingle):

‎uncompyle6/scanners/scanner2.py‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -143,13 +143,19 @@ def unmangle(name):
143143
extended_arg = 0
144144
for offset in self.op_range(0, n):
145145
if offset in jump_targets:
146-
k = 0
147-
for j in jump_targets[offset]:
146+
jump_idx = 0
147+
# We want to process COME_FROMs to the same offset to be in *descending*
148+
# offset order so we have the larger range or biggest instruction interval
149+
# last. (I think they are sorted in increasing order, but for safety
150+
# we sort them). That way, specific COME_FROM tags will match up
151+
# properly. For example, a "loop" with an "if" nested in it should have the
152+
# "loop" tag last so the grammar rule matches that properly.
153+
for jump_offset in sorted(jump_targets[offset], reverse=True):
148154
tokens.append(Token(
149-
'COME_FROM', None, repr(j),
150-
offset="%s_%d" % (offset, k),
155+
'COME_FROM', None, repr(jump_offset),
156+
offset="%s_%d" % (offset, jump_idx),
151157
has_arg = True))
152-
k += 1
158+
jump_idx += 1
153159

154160
op = self.code[offset]
155161
opname = self.opc.opname[op]
@@ -410,7 +416,7 @@ def build_stmt_indices(self):
410416
j = self.prev[s]
411417
while code[j] in self.designator_ops:
412418
j = self.prev[j]
413-
if code[j] == self.opc.FOR_ITER:
419+
if self.version >= 2.1 and code[j] == self.opc.FOR_ITER:
414420
stmts.remove(s)
415421
continue
416422
last_stmt = s
@@ -564,7 +570,8 @@ def detect_structure(self, pos, op):
564570

565571
target = self.get_target(jump_back, self.opc.JUMP_ABSOLUTE)
566572

567-
if code[target] in (self.opc.FOR_ITER, self.opc.GET_ITER):
573+
if (self.version >= 2.0 and
574+
code[target] in (self.opc.FOR_ITER, self.opc.GET_ITER)):
568575
loop_type = 'for'
569576
else:
570577
loop_type = 'while'
@@ -837,7 +844,8 @@ def find_jump_targets(self):
837844
oparg = self.get_argument(offset)
838845

839846
if label is None:
840-
if op in self.opc.hasjrel and op != self.opc.FOR_ITER:
847+
if (op in self.opc.hasjrel and
848+
(self.version < 2.0 or op != self.opc.FOR_ITER)):
841849
label = offset + 3 + oparg
842850
elif self.version == 2.7 and op in self.opc.hasjabs:
843851
if op in (self.opc.JUMP_IF_FALSE_OR_POP,

0 commit comments

Comments
 (0)