Mercurial > p > roundup > code

annotate roundup/anypy/strings.py @ 7766:c65e0a725c88

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix: replace eval with ast.literal_eval; ruff linting strings.py: When reading csv export files, use the safer literal_eval to evaluate the fields. Issue pointed out by ruff linter. Other Cleanup: sort imports replace if, else, if, else with if, elif, else use isinstance with tuples rather than 'isinstance() or isinstance()' test_anypy.py: Added tests for tuples, and booleans. Also added exception handling for malformed strings, booleans, tuples.
author John Rouillard <rouilj@ieee.org>
date Sat, 02 Mar 2024 04:01:22 -0500
parents 417c8ddc98ac
children 4261449081be
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
1 # Roundup represents text internally using the native Python str type.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
2 # In Python 3, these are Unicode strings. In Python 2, these are
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
3 # encoded using UTF-8, and the Python 2 unicode type is only used in a
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
4 # few places, generally for interacting with external modules
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
5 # requiring that type to be used.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
6
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
7 import ast
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
8 import io
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
9 import sys
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
10
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
11 _py3 = sys.version_info[0] > 2
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
12
5418
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
13 if _py3:
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
14 StringIO = io.StringIO
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
15 else:
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
16 StringIO = io.BytesIO
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
17
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
18
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
19 def b2s(b):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
20 """Convert a UTF-8 encoded bytes object to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
21 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
22 return b.decode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
23 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
24 return b
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
25
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
26
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
27 def s2b(s):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
28 """Convert a string object to UTF-8 encoded bytes."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
29 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
30 return s.encode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
31 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
32 return s
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
33
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
34
5492
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
35 def bs2b(s):
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
36 """Convert a string object or UTF-8 encoded bytes to UTF-8 encoded bytes.
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
37 """
5492
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
38 if _py3:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
39 if isinstance(s, bytes):
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
40 return s
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
41 else:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
42 return s.encode('utf-8')
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
43 else:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
44 return s
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
45
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
46
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
47 def s2u(s, errors='strict'):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
48 """Convert a string object to a Unicode string."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
49 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
50 return s
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
51 else:
7755
417c8ddc98ac fix: undo last minute edit and try to get make checkin clean.
John Rouillard <rouilj@ieee.org>
parents: 7230
diff changeset
52 return unicode(s, 'utf-8', errors) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
53
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
54
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
55 def u2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
56 """Convert a Unicode string to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
57 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
58 return u
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
59 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
60 return u.encode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
61
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
62
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
63 def us2u(s, errors='strict'):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
64 """Convert a string or Unicode string to a Unicode string."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
65 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
66 return s
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
67 elif isinstance(s, unicode): # noqa: F821
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
68 return s
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
69 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
70 return unicode(s, 'utf-8', errors) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
71
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
72
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
73 def us2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
74 """Convert a string or Unicode string to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
75 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
76 return u
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
77 elif isinstance(u, unicode): # noqa: F821
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
78 return u.encode('utf-8')
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
79 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
80 return u
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
81
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
82
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
83 def uany2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
84 """Convert a Unicode string or other object to the internal string format.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
85
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
86 Objects that are not Unicode strings are passed to str()."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
87 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
88 return str(u)
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
89 elif isinstance(u, unicode): # noqa: F821
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
90 return u.encode('utf-8')
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
91 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
92 return str(u)
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
93
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
94
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
95 def is_us(s):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
96 """Return whether an object is a string or Unicode string."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
97 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
98 return isinstance(s, str)
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
99 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
100 return isinstance(s, (str, unicode)) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
101
5417
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
102
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
103 def uchr(c):
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
104 """Return the Unicode string containing the given character."""
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
105 if _py3:
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
106 return chr(c)
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
107 else:
7755
417c8ddc98ac fix: undo last minute edit and try to get make checkin clean.
John Rouillard <rouilj@ieee.org>
parents: 7230
diff changeset
108 return unichr(c) # noqa: F821
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
109
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
110 # CSV files used for export and import represent strings in the style
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
111 # used by repr in Python 2; this means that each byte of the UTF-8
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
112 # representation is represented by a \x escape if not a printable
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
113 # ASCII character. When such a representation is interpreted by eval
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
114 # in Python 3, the effect is that the Unicode characters in the
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
115 # resulting string correspond to UTF-8 bytes, so encoding the string
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
116 # as ISO-8859-1 produces the correct byte-string which must then be
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
117 # decoded as UTF-8 to produce the correct Unicode string. The same
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
118 # representations are also used for journal storage in RDBMS
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
119 # databases, so that the database can be compatible between Python 2
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
120 # and Python 3.
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
121
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
122
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
123 def repr_export(v):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
124 """Return a Python-2-style representation of a value for export to CSV."""
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
125 if _py3:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
126 if isinstance(v, str):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
127 return repr(s2b(v))[1:]
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
128 elif isinstance(v, dict):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
129 repr_vals = []
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
130 for key, value in sorted(v.items()):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
131 repr_vals.append('%s: %s' % (repr_export(key),
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
132 repr_export(value)))
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
133 return '{%s}' % ', '.join(repr_vals)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
134 else:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
135 return repr(v)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
136 else:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
137 return repr(v)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
138
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
139
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
140 def eval_import(s):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
141 """Evaluate a Python-2-style value imported from a CSV file."""
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
142 try:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
143 if _py3:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
144 try:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
145 v = ast.literal_eval(s)
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
146 except SyntaxError:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
147 # handle case where link operation reports id a long
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
148 # int ('issue', 5002L, "status") rather than as a
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
149 # string. This was a bug that existed and was fixed
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
150 # before or with v1.2.0
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
151 import re
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
152 v = ast.literal_eval(re.sub(r', ([0-9]+)L,', r', \1,', s))
6531
82f870433b18 issue2551107 - Handle long int in history params
John Rouillard <rouilj@ieee.org>
parents: 6048
diff changeset
153
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
154 if isinstance(v, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
155 return v.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
156 elif isinstance(v, dict):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
157 v_mod = {}
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
158 # ruff: noqa: PLW2901
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
159 for key, value in v.items():
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
160 if isinstance(key, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
161 key = key.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
162 if isinstance(value, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
163 value = value.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
164 v_mod[key] = value
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
165 return v_mod
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
166 else:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
167 return v
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
168 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
169 return ast.literal_eval(s)
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
170
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
171 except (ValueError, SyntaxError) as e:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
172 raise ValueError(
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
173 ("Error %(exception)s trying to parse value '%(value)s'") %
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
174 { 'exception': e, 'value': s})
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
175
Roundup Issue Tracker: http://roundup-tracker.org/