Mercurial > p > roundup > code

annotate roundup/anypy/strings.py @ 8264:09e8d1a4c796

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
docs: clarify wording, fix index, add superseder link Make superseder, messages etc. properties index entries point to the right place. Link to description of using Superseder in the original overview. fix bad wording on boolean properties.
author John Rouillard <rouilj@ieee.org>
date Wed, 08 Jan 2025 11:39:54 -0500
parents 4261449081be
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
1 # Roundup represents text internally using the native Python str type.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
2 # In Python 3, these are Unicode strings. In Python 2, these are
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
3 # encoded using UTF-8, and the Python 2 unicode type is only used in a
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
4 # few places, generally for interacting with external modules
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
5 # requiring that type to be used.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
6
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
7 import ast
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
8 import io
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
9 import sys
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
10
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
11 _py3 = sys.version_info[0] > 2
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
12
5418
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
13 if _py3:
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
14 StringIO = io.StringIO
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
15 else:
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
16 StringIO = io.BytesIO
55f09ca366c4 Python 3 preparation: StringIO.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5417
diff changeset
17
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
18
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
19 def b2s(b):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
20 """Convert a UTF-8 encoded bytes object to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
21 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
22 return b.decode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
23 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
24 return b
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
25
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
26
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
27 def s2b(s):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
28 """Convert a string object to UTF-8 encoded bytes."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
29 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
30 return s.encode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
31 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
32 return s
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
33
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
34
5492
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
35 def bs2b(s):
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
36 """Convert a string object or UTF-8 encoded bytes to UTF-8 encoded bytes.
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
37 """
5492
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
38 if _py3:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
39 if isinstance(s, bytes):
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
40 return s
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
41 else:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
42 return s.encode('utf-8')
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
43 else:
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
44 return s
6b0c542642be blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5418
diff changeset
45
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
46
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
47 def s2u(s, errors='strict'):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
48 """Convert a string object to a Unicode string."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
49 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
50 return s
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
51 else:
7755
417c8ddc98ac fix: undo last minute edit and try to get make checkin clean.
John Rouillard <rouilj@ieee.org>
parents: 7230
diff changeset
52 return unicode(s, 'utf-8', errors) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
53
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
54
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
55 def u2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
56 """Convert a Unicode string to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
57 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
58 return u
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
59 else:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
60 return u.encode('utf-8')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
61
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
62
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
63 def us2u(s, errors='strict'):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
64 """Convert a string or Unicode string to a Unicode string."""
7771
4261449081be chore: fix spacing/indent, ignore rule, reduce if/elif with or
John Rouillard <rouilj@ieee.org>
parents: 7766
diff changeset
65 if _py3 or isinstance(s, unicode): # noqa: F821
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
66 return s
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
67 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
68 return unicode(s, 'utf-8', errors) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
69
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
70
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
71 def us2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
72 """Convert a string or Unicode string to the internal string format."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
73 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
74 return u
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
75 elif isinstance(u, unicode): # noqa: F821
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
76 return u.encode('utf-8')
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
77 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
78 return u
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
79
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
80
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
81 def uany2s(u):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
82 """Convert a Unicode string or other object to the internal string format.
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
83
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
84 Objects that are not Unicode strings are passed to str()."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
85 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
86 return str(u)
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
87 elif isinstance(u, unicode): # noqa: F821
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
88 return u.encode('utf-8')
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
89 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
90 return str(u)
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
91
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
92
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
93 def is_us(s):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
94 """Return whether an object is a string or Unicode string."""
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
95 if _py3:
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
96 return isinstance(s, str)
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
diff changeset
97 else:
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
98 return isinstance(s, (str, unicode)) # noqa: F821
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
99
5417
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
100
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
101 def uchr(c):
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
102 """Return the Unicode string containing the given character."""
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
103 if _py3:
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
104 return chr(c)
c749d6795bc2 Python 3 preparation: unichr.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5416
diff changeset
105 else:
7755
417c8ddc98ac fix: undo last minute edit and try to get make checkin clean.
John Rouillard <rouilj@ieee.org>
parents: 7230
diff changeset
106 return unichr(c) # noqa: F821
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
107
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
108 # CSV files used for export and import represent strings in the style
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
109 # used by repr in Python 2; this means that each byte of the UTF-8
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
110 # representation is represented by a \x escape if not a printable
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
111 # ASCII character. When such a representation is interpreted by eval
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
112 # in Python 3, the effect is that the Unicode characters in the
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
113 # resulting string correspond to UTF-8 bytes, so encoding the string
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
114 # as ISO-8859-1 produces the correct byte-string which must then be
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
115 # decoded as UTF-8 to produce the correct Unicode string. The same
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
116 # representations are also used for journal storage in RDBMS
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
117 # databases, so that the database can be compatible between Python 2
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
118 # and Python 3.
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
119
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
120
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
121 def repr_export(v):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
122 """Return a Python-2-style representation of a value for export to CSV."""
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
123 if _py3:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
124 if isinstance(v, str):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
125 return repr(s2b(v))[1:]
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
126 elif isinstance(v, dict):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
127 repr_vals = []
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
128 for key, value in sorted(v.items()):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
129 repr_vals.append('%s: %s' % (repr_export(key),
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
130 repr_export(value)))
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
131 return '{%s}' % ', '.join(repr_vals)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
132 else:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
133 return repr(v)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
134 else:
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
135 return repr(v)
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
136
6048
81990ac0b013 flak8: move import; whitespace changes; ignore undefined
John Rouillard <rouilj@ieee.org>
parents: 5525
diff changeset
137
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
138 def eval_import(s):
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
139 """Evaluate a Python-2-style value imported from a CSV file."""
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
140 try:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
141 if _py3:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
142 try:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
143 v = ast.literal_eval(s)
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
144 except SyntaxError:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
145 # handle case where link operation reports id a long
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
146 # int ('issue', 5002L, "status") rather than as a
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
147 # string. This was a bug that existed and was fixed
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
148 # before or with v1.2.0
7771
4261449081be chore: fix spacing/indent, ignore rule, reduce if/elif with or
John Rouillard <rouilj@ieee.org>
parents: 7766
diff changeset
149 import re # noqa: PLC0415
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
150 v = ast.literal_eval(re.sub(r', ([0-9]+)L,', r', \1,', s))
6531
82f870433b18 issue2551107 - Handle long int in history params
John Rouillard <rouilj@ieee.org>
parents: 6048
diff changeset
151
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
152 if isinstance(v, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
153 return v.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
154 elif isinstance(v, dict):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
155 v_mod = {}
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
156 # ruff: noqa: PLW2901
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
157 for key, value in v.items():
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
158 if isinstance(key, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
159 key = key.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
160 if isinstance(value, str):
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
161 value = value.encode('iso-8859-1').decode('utf-8')
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
162 v_mod[key] = value
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
163 return v_mod
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
164 else:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
165 return v
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5492
diff changeset
166 else:
7771
4261449081be chore: fix spacing/indent, ignore rule, reduce if/elif with or
John Rouillard <rouilj@ieee.org>
parents: 7766
diff changeset
167 return ast.literal_eval(s)
7766
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
168
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
169 except (ValueError, SyntaxError) as e:
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
170 raise ValueError(
c65e0a725c88 fix: replace eval with ast.literal_eval; ruff linting
John Rouillard <rouilj@ieee.org>
parents: 7755
diff changeset
171 ("Error %(exception)s trying to parse value '%(value)s'") %
7771
4261449081be chore: fix spacing/indent, ignore rule, reduce if/elif with or
John Rouillard <rouilj@ieee.org>
parents: 7766
diff changeset
172 {'exception': e, 'value': s})
Roundup Issue Tracker: http://roundup-tracker.org/