Mercurial > p > roundup > code
annotate roundup/backends/__init__.py @ 5543:bc3e00a3d24b
MySQL backend fixes for Python 3.
With Python 2, text sent to and from MySQL is treated as bytes in
Python. The database may be recorded by MySQL as having some other
encoding (latin1 being the default in some MySQL versions - Roundup
does not set an encoding explicitly, unlike in back_postgresql), but
as long as MySQL's notion of the connection encoding agrees with its
notion of the database encoding, no conversions actually take place
and the bytes are stored and returned as-is.
With Python 3, text sent to and from MySQL is treated as Python
Unicode strings. When the database and connection encoding is latin1,
that means the bytes stored in the database under Python 2 are
interpreted as latin1 and converted from that to Unicode, producing
incorrect results for any non-ASCII characters; furthermore, if trying
to store new non-ASCII data in the database under Python 3, any
non-latin1 characters produce errors.
This patch arranges for both the connection and database character
sets to be UTF-8 when using Python 3, and documents a need to export
and import the database when moving from Python 2 to Python 3 with
this backend.
| author | Joseph Myers <jsm@polyomino.org.uk> |
|---|---|
| date | Sun, 16 Sep 2018 16:19:20 +0000 |
| parents | 22eecc8a8bd4 |
| children | 7f00fc5958ca |
| rev | line source |
|---|---|
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
1 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
6 # |
| 214 | 7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
11 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1190
diff
changeset
|
17 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
18 '''Container for the hyperdb storage backend implementations. |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1190
diff
changeset
|
19 ''' |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
20 __docformat__ = 'restructuredtext' |
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
159
diff
changeset
|
21 |
|
2925
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
22 import sys |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
23 |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
24 # These names are used to suppress import errors. |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
25 # If get_backend raises an ImportError with appropriate |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
26 # module name, have_backend quietly returns False. |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
27 # Otherwise the error is reraised. |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
28 _modules = { |
|
3718
0d561b24ceff
support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents:
3503
diff
changeset
|
29 'mysql': ('MySQLdb',), |
|
5464
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
30 'postgresql': ('psycopg2',), |
|
4269
ab4563e97d22
corrections for python2.3 compatibility:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3941
diff
changeset
|
31 'sqlite': ('pysqlite', 'pysqlite2', 'sqlite3', '_sqlite3', 'sqlite'), |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
32 } |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
33 |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
34 def get_backend(name): |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
35 '''Get a specific backend by name.''' |
|
2925
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
36 vars = globals() |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
37 # if requested backend has been imported yet, return current instance |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4269
diff
changeset
|
38 if name in vars: |
|
2925
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
39 return vars[name] |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
40 # import the backend module |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
41 module_name = 'back_%s' % name |
|
5425
ebd6df1126a2
Python 3 preparation: update __import__ call for relative import.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
42 module = __import__(module_name, vars, level=1) |
|
4687
4960a2c21590
Python 2.3 workarounds cleanup
anatoly techtonik <techtonik@gmail.com>
parents:
4570
diff
changeset
|
43 vars[name] = module |
|
4960a2c21590
Python 2.3 workarounds cleanup
anatoly techtonik <techtonik@gmail.com>
parents:
4570
diff
changeset
|
44 return module |
|
159
764db91c0dea
Checks for ability to import the specific back-end module.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
45 |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
46 def have_backend(name): |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
47 '''Is backend "name" available?''' |
|
1809
bd127cafe3a8
Simplify backend importing, by moving the imports into the backend modules.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1609
diff
changeset
|
48 try: |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
49 get_backend(name) |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
50 return 1 |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
5005
diff
changeset
|
51 except ImportError as e: |
|
5464
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
52 if hasattr(e, 'name'): |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
53 modname = e.name |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
54 else: |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
55 modname = e.args[0][16:] if e.args[0].startswith('No module named ') else None |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
56 |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
57 if modname and (modname in _modules.get(name, (name,))): |
|
22eecc8a8bd4
fixed backend detection for Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5425
diff
changeset
|
58 return 0 |
|
3718
0d561b24ceff
support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents:
3503
diff
changeset
|
59 raise |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
60 return 0 |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
61 |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
62 def list_backends(): |
|
2925
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
63 '''List all available backend names. |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
64 |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
65 This function has side-effect of registering backward-compatible |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
66 globals for all available backends. |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
67 |
|
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
68 ''' |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
69 l = [] |
|
3941
9997b941dd6d
remove some metakit references
Richard Jones <richard@users.sourceforge.net>
parents:
3727
diff
changeset
|
70 for name in 'anydbm', 'mysql', 'sqlite', 'postgresql': |
|
2856
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
71 if have_backend(name): |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
72 l.append(name) |
|
adec352e2ce0
don't try to import all backends in backends.__init__ unless we *want* to
Richard Jones <richard@users.sourceforge.net>
parents:
2812
diff
changeset
|
73 return l |
|
854
3cdfa5d86cec
Added metakit backend to the db tests...
Richard Jones <richard@users.sourceforge.net>
parents:
733
diff
changeset
|
74 |
|
2925
e54c95de4c6a
fix: second call to get_backend() succeeded...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2856
diff
changeset
|
75 # vim: set filetype=python sts=4 sw=4 et si : |
