Mercurial > p > roundup > code
annotate roundup/backends/back_mysql.py @ 5075:04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
There are various 'import *' statements scattered through the code which
are generally not a good thing. These should be fairly safe changes, but
I'll commit them one file at a time to make it easier to track down
issues with a bisect if they crop up later.
| author | John Kristensen <john@jerrykan.com> |
|---|---|
| date | Fri, 10 Jun 2016 18:14:36 +1000 |
| parents | e424987d294a |
| children | e74c3611b138 |
| rev | line source |
|---|---|
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
1 # |
|
1433
8429095241d7
mysql tests will not be run if there is no chance of passing.
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1415
diff
changeset
|
2 # Copyright (c) 2003 Martynas Sklyzmantas, Andrey Lebedev <andrey@micro.lt> |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
3 # |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
4 # This module is free software, and you may redistribute it and/or modify |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
5 # under the same terms as Python, so long as this copyright message and |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
6 # disclaimer are retained in their original form. |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
7 # |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
8 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
9 '''This module defines a backend implementation for MySQL. |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
10 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
11 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
12 How to implement AUTO_INCREMENT: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
13 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
14 mysql> create table foo (num integer auto_increment primary key, name |
|
3934
8c75e8fce5dc
Fix a couple more old instances of "type" instead of "ENGINE" for mysql backend.
Richard Jones <richard@users.sourceforge.net>
parents:
3929
diff
changeset
|
15 varchar(255)) AUTO_INCREMENT=1 ENGINE=InnoDB; |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
16 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
17 ql> insert into foo (name) values ('foo5'); |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
18 Query OK, 1 row affected (0.00 sec) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
19 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
20 mysql> SELECT num FROM foo WHERE num IS NULL; |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
21 +-----+ |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
22 | num | |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
23 +-----+ |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
24 | 4 | |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
25 +-----+ |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
26 1 row in set (0.00 sec) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
27 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
28 mysql> SELECT num FROM foo WHERE num IS NULL; |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
29 Empty set (0.00 sec) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
30 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
31 NOTE: we don't need an index on the id column if it's PRIMARY KEY |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
32 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
33 ''' |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
34 __docformat__ = 'restructuredtext' |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
35 |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
36 from roundup import date, hyperdb, password |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
37 from roundup.backends import rdbms_common |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
38 import MySQLdb |
|
1434
b953750bdc04
trackers on mysql can be initialised
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1433
diff
changeset
|
39 import os, shutil |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
40 from MySQLdb.constants import ER |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
41 import logging |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
42 |
|
4887
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
43 isolation_levels = \ |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
44 { 'read uncommitted': 'READ UNCOMMITTED' |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
45 , 'read committed': 'READ COMMITTED' |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
46 , 'repeatable read': 'REPEATABLE READ' |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
47 , 'serializable': 'SERIALIZABLE' |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
48 } |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
49 |
|
2693
59ea85d47d34
add dbnamestr argument to connection_dict...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2679
diff
changeset
|
50 def connection_dict(config, dbnamestr=None): |
|
59ea85d47d34
add dbnamestr argument to connection_dict...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2679
diff
changeset
|
51 d = rdbms_common.connection_dict(config, dbnamestr) |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
52 if d.has_key('password'): |
|
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
53 d['passwd'] = d['password'] |
|
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
54 del d['password'] |
|
3025
7032b500b7e0
fix port number as int in mysql connection info [SF#1082530]
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
55 if d.has_key('port'): |
|
7032b500b7e0
fix port number as int in mysql connection info [SF#1082530]
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
56 d['port'] = int(d['port']) |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
57 return d |
|
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
58 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
59 def db_nuke(config): |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
60 """Clear all database contents and drop database itself""" |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
61 if db_exists(config): |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
62 kwargs = connection_dict(config) |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
63 conn = MySQLdb.connect(**kwargs) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
64 try: |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
65 conn.select_db(config.RDBMS_NAME) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
66 except: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
67 # no, it doesn't exist |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
68 pass |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
69 else: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
70 cursor = conn.cursor() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
71 cursor.execute("SHOW TABLES") |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
72 tables = cursor.fetchall() |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
73 # stupid MySQL bug requires us to drop all the tables first |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
74 for table in tables: |
|
3603
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
75 command = 'DROP TABLE `%s`'%table[0] |
|
4085
04843a029ea1
Fix some broken logging.
Richard Jones <richard@users.sourceforge.net>
parents:
4060
diff
changeset
|
76 logging.debug(command) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
77 cursor.execute(command) |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
78 command = "DROP DATABASE %s"%config.RDBMS_NAME |
|
4085
04843a029ea1
Fix some broken logging.
Richard Jones <richard@users.sourceforge.net>
parents:
4060
diff
changeset
|
79 logging.info(command) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
80 cursor.execute(command) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
81 conn.commit() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
82 conn.close() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
83 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
84 if os.path.exists(config.DATABASE): |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
85 shutil.rmtree(config.DATABASE) |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
86 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
87 def db_create(config): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
88 """Create the database.""" |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
89 kwargs = connection_dict(config) |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
90 conn = MySQLdb.connect(**kwargs) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
91 cursor = conn.cursor() |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
92 command = "CREATE DATABASE %s"%config.RDBMS_NAME |
|
4085
04843a029ea1
Fix some broken logging.
Richard Jones <richard@users.sourceforge.net>
parents:
4060
diff
changeset
|
93 logging.info(command) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
94 cursor.execute(command) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
95 conn.commit() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
96 conn.close() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
97 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
98 def db_exists(config): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
99 """Check if database already exists.""" |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
100 kwargs = connection_dict(config) |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
101 conn = MySQLdb.connect(**kwargs) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
102 try: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
103 try: |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
104 conn.select_db(config.RDBMS_NAME) |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
105 except MySQLdb.OperationalError: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
106 return 0 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1839
diff
changeset
|
107 finally: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
108 conn.close() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
109 return 1 |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
110 |
|
1434
b953750bdc04
trackers on mysql can be initialised
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1433
diff
changeset
|
111 |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
112 class Database(rdbms_common.Database): |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
113 arg = '%s' |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
114 |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3025
diff
changeset
|
115 # used by some code to switch styles of query |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3025
diff
changeset
|
116 implements_intersect = 0 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3025
diff
changeset
|
117 |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
118 # Backend for MySQL to use. |
|
1913
d929c31a3620
clarified minimum version required for MySQL backend (4.0.16)
Richard Jones <richard@users.sourceforge.net>
parents:
1912
diff
changeset
|
119 # InnoDB is faster, but if you're running <4.0.16 then you'll need to |
|
d929c31a3620
clarified minimum version required for MySQL backend (4.0.16)
Richard Jones <richard@users.sourceforge.net>
parents:
1912
diff
changeset
|
120 # use BDB to pass all unit tests. |
|
d929c31a3620
clarified minimum version required for MySQL backend (4.0.16)
Richard Jones <richard@users.sourceforge.net>
parents:
1912
diff
changeset
|
121 mysql_backend = 'InnoDB' |
|
d929c31a3620
clarified minimum version required for MySQL backend (4.0.16)
Richard Jones <richard@users.sourceforge.net>
parents:
1912
diff
changeset
|
122 #mysql_backend = 'BDB' |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
123 |
|
2099
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
124 hyperdb_to_sql_datatypes = { |
|
3421
0d9406d14186
MySQL now creates String columns using the TEXT column type
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
125 hyperdb.String : 'TEXT', |
|
2099
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
126 hyperdb.Date : 'DATETIME', |
|
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
127 hyperdb.Link : 'INTEGER', |
|
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
128 hyperdb.Interval : 'VARCHAR(255)', |
|
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
129 hyperdb.Password : 'VARCHAR(255)', |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2100
diff
changeset
|
130 hyperdb.Boolean : 'BOOL', |
|
2099
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
131 hyperdb.Number : 'REAL', |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
4887
diff
changeset
|
132 hyperdb.Integer : 'INTEGER', |
|
2099
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
133 } |
|
3837257ca9a5
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
134 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
135 hyperdb_to_sql_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
136 hyperdb.String : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
137 # no fractional seconds for MySQL |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
138 hyperdb.Date : lambda x: x.formal(sep=' '), |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
139 hyperdb.Link : int, |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
140 hyperdb.Interval : str, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
141 hyperdb.Password : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
142 hyperdb.Boolean : int, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
143 hyperdb.Number : lambda x: x, |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
4887
diff
changeset
|
144 hyperdb.Integer : int, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
145 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
146 } |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
147 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
148 def sql_open_connection(self): |
|
2693
59ea85d47d34
add dbnamestr argument to connection_dict...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2679
diff
changeset
|
149 kwargs = connection_dict(self.config, 'db') |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
3969
diff
changeset
|
150 self.log_info('open database %r'%(kwargs['db'],)) |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
151 try: |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
152 conn = MySQLdb.connect(**kwargs) |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
153 except MySQLdb.OperationalError, message: |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
154 raise hyperdb.DatabaseError, message |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
155 cursor = conn.cursor() |
|
3216
27aba1d8fccc
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
156 cursor.execute("SET AUTOCOMMIT=0") |
|
4887
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
157 lvl = isolation_levels [self.config.RDBMS_ISOLATION_LEVEL] |
|
05c857e5dbed
New rdbms configuration option 'isolation_level'
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4466
diff
changeset
|
158 cursor.execute("SET SESSION TRANSACTION ISOLATION LEVEL %s" % lvl) |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
159 cursor.execute("START TRANSACTION") |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
160 return (conn, cursor) |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
161 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
162 def open_connection(self): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
163 # make sure the database actually exists |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
164 if not db_exists(self.config): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
165 db_create(self.config) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
166 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
167 self.conn, self.cursor = self.sql_open_connection() |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
168 |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
169 try: |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
170 self.load_dbschema() |
|
1433
8429095241d7
mysql tests will not be run if there is no chance of passing.
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1415
diff
changeset
|
171 except MySQLdb.OperationalError, message: |
|
8429095241d7
mysql tests will not be run if there is no chance of passing.
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1415
diff
changeset
|
172 if message[0] != ER.NO_DB_ERROR: |
|
8429095241d7
mysql tests will not be run if there is no chance of passing.
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1415
diff
changeset
|
173 raise |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
174 except MySQLdb.ProgrammingError, message: |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
175 if message[0] != ER.NO_SUCH_TABLE: |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
176 raise hyperdb.DatabaseError, message |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
177 self.init_dbschema() |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
178 self.sql("CREATE TABLE `schema` (`schema` TEXT) ENGINE=%s"% |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
179 self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
180 self.sql('''CREATE TABLE ids (name VARCHAR(255), |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
181 num INTEGER) ENGINE=%s'''%self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
182 self.sql('create index ids_name_idx on ids(name)') |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
183 self.create_version_2_tables() |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
184 |
|
3458
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
185 def load_dbschema(self): |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
186 ''' Load the schema definition that the database currently implements |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
187 ''' |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
188 self.cursor.execute('select `schema` from `schema`') |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
189 schema = self.cursor.fetchone() |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
190 if schema: |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
191 self.database_schema = eval(schema[0]) |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
192 else: |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
193 self.database_schema = {} |
|
931dc7ed25c7
handle "schema" being reserved word in MySQL 5+ [SF#1397569]
Richard Jones <richard@users.sourceforge.net>
parents:
3457
diff
changeset
|
194 |
|
3603
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
195 def save_dbschema(self): |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
196 ''' Save the schema definition that the database currently implements |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
197 ''' |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
198 s = repr(self.database_schema) |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
199 self.sql('delete from `schema`') |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
200 self.sql('insert into `schema` values (%s)', (s,)) |
|
f7399e2aa610
escape *all* uses of "schema" in mysql backend [SF#1472120]
Richard Jones <richard@users.sourceforge.net>
parents:
3585
diff
changeset
|
201 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
202 def create_version_2_tables(self): |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
203 # OTK store |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
204 self.sql('''CREATE TABLE otks (otk_key VARCHAR(255), |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
205 otk_value TEXT, otk_time FLOAT(20)) |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
206 ENGINE=%s'''%self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
207 self.sql('CREATE INDEX otks_key_idx ON otks(otk_key)') |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
208 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
209 # Sessions store |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
210 self.sql('''CREATE TABLE sessions (session_key VARCHAR(255), |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
211 session_time FLOAT(20), session_value TEXT) |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
212 ENGINE=%s'''%self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
213 self.sql('''CREATE INDEX sessions_key_idx ON |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
214 sessions(session_key)''') |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
215 |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
216 # full-text indexing store |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
217 self.sql('''CREATE TABLE __textids (_class VARCHAR(255), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
218 _itemid VARCHAR(255), _prop VARCHAR(255), _textid INT) |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
219 ENGINE=%s'''%self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
220 self.sql('''CREATE TABLE __words (_word VARCHAR(30), |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
221 _textid INT) ENGINE=%s'''%self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
222 self.sql('CREATE INDEX words_word_ids ON __words(_word)') |
|
3858
bb30bbfc7cdd
Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3687
diff
changeset
|
223 self.sql('CREATE INDEX words_by_id ON __words (_textid)') |
|
bb30bbfc7cdd
Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3687
diff
changeset
|
224 self.sql('CREATE UNIQUE INDEX __textids_by_props ON ' |
|
bb30bbfc7cdd
Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3687
diff
changeset
|
225 '__textids (_class, _itemid, _prop)') |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
226 sql = 'insert into ids (name, num) values (%s,%s)'%(self.arg, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
227 self.sql(sql, ('__textids', 1)) |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
228 |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
229 def add_new_columns_v2(self): |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
230 '''While we're adding the actor column, we need to update the |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
231 tables to have the correct datatypes.''' |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
232 for klass in self.classes.values(): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
233 cn = klass.classname |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
234 properties = klass.getprops() |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
235 old_spec = self.database_schema['tables'][cn] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
236 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
237 # figure the non-Multilink properties to copy over |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
238 propnames = ['activity', 'creation', 'creator'] |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
239 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
240 # figure actions based on data type |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
241 for name, s_prop in old_spec[1]: |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
242 # s_prop is a repr() string of a hyperdb type object |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
243 if s_prop.find('Multilink') == -1: |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
244 if properties.has_key(name): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
245 propnames.append(name) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
246 continue |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
247 tn = '%s_%s'%(cn, name) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
248 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
249 if properties.has_key(name): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
250 # grabe the current values |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
251 sql = 'select linkid, nodeid from %s'%tn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
252 self.sql(sql) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
253 rows = self.cursor.fetchall() |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
254 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
255 # drop the old table |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
256 self.drop_multilink_table_indexes(cn, name) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
257 sql = 'drop table %s'%tn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
258 self.sql(sql) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
259 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
260 if properties.has_key(name): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
261 # re-create and populate the new table |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
262 self.create_multilink_table(klass, name) |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
263 sql = '''insert into %s (linkid, nodeid) values |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
264 (%s, %s)'''%(tn, self.arg, self.arg) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
265 for linkid, nodeid in rows: |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
266 self.sql(sql, (int(linkid), int(nodeid))) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
267 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
268 # figure the column names to fetch |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
269 fetch = ['_%s'%name for name in propnames] |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
270 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
271 # select the data out of the old table |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
272 fetch.append('id') |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
273 fetch.append('__retired__') |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
274 fetchcols = ','.join(fetch) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
275 sql = 'select %s from _%s'%(fetchcols, cn) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
276 self.sql(sql) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
277 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
278 # unserialise the old data |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
279 olddata = [] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
280 propnames = propnames + ['id', '__retired__'] |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
281 cols = [] |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
282 first = 1 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
283 for entry in self.cursor.fetchall(): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
284 l = [] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
285 olddata.append(l) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
286 for i in range(len(propnames)): |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
287 name = propnames[i] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
288 v = entry[i] |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
289 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
290 if name in ('id', '__retired__'): |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
291 if first: |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
292 cols.append(name) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
293 l.append(int(v)) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
294 continue |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
295 if first: |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
296 cols.append('_' + name) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
297 prop = properties[name] |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
298 if isinstance(prop, hyperdb.Date) and v is not None: |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
299 v = date.Date(v) |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
300 elif isinstance(prop, hyperdb.Interval) and v is not None: |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
301 v = date.Interval(v) |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
302 elif isinstance(prop, hyperdb.Password) and v is not None: |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
303 v = password.Password(encrypted=v) |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
304 elif isinstance(prop, hyperdb.Integer) and v is not None: |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
4887
diff
changeset
|
305 v = int(v) |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
306 elif (isinstance(prop, hyperdb.Boolean) or |
|
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
307 isinstance(prop, hyperdb.Number)) and v is not None: |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
308 v = float(v) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
309 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
310 # convert to new MySQL data type |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
311 prop = properties[name] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
312 if v is not None: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
313 e = self.to_sql_value(prop.__class__)(v) |
|
2431
12fb2aa3c6d2
fixes to mysql upgrade code
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
314 else: |
|
12fb2aa3c6d2
fixes to mysql upgrade code
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
315 e = None |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
316 l.append(e) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
317 |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
318 # Intervals store the seconds value too |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
319 if isinstance(prop, hyperdb.Interval): |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
320 if first: |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
321 cols.append('__' + name + '_int__') |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
322 if v is not None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
323 l.append(v.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
324 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
325 l.append(e) |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
326 first = 0 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
327 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
328 self.drop_class_table_indexes(cn, old_spec[0]) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
329 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
330 # drop the old table |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
331 self.sql('drop table _%s'%cn) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
332 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
333 # create the new table |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
334 self.create_class_table(klass) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
335 |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
336 # do the insert of the old data |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
337 args = ','.join([self.arg for x in cols]) |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
338 cols = ','.join(cols) |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
339 sql = 'insert into _%s (%s) values (%s)'%(cn, cols, args) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
340 for entry in olddata: |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
341 self.sql(sql, tuple(entry)) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
342 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
343 # now load up the old journal data to migrate it |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
344 cols = ','.join('nodeid date tag action params'.split()) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
345 sql = 'select %s from %s__journal'%(cols, cn) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
346 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
347 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
348 # data conversions |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
349 olddata = [] |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
350 for nodeid, journaldate, journaltag, action, params in \ |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
351 self.cursor.fetchall(): |
|
2431
12fb2aa3c6d2
fixes to mysql upgrade code
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
352 #nodeid = int(nodeid) |
|
3140
2487b070043b
slightly better fix
Richard Jones <richard@users.sourceforge.net>
parents:
3088
diff
changeset
|
353 journaldate = date.Date(journaldate) |
|
2431
12fb2aa3c6d2
fixes to mysql upgrade code
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
354 #params = eval(params) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
355 olddata.append((nodeid, journaldate, journaltag, action, |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
356 params)) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
357 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
358 # drop journal table and indexes |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
359 self.drop_journal_table_indexes(cn) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
360 sql = 'drop table %s__journal'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
361 self.sql(sql) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
362 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
363 # re-create journal table |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
364 self.create_journal_table(klass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
365 dc = self.to_sql_value(hyperdb.Date) |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
366 for nodeid, journaldate, journaltag, action, params in olddata: |
|
3140
2487b070043b
slightly better fix
Richard Jones <richard@users.sourceforge.net>
parents:
3088
diff
changeset
|
367 self.save_journal(cn, cols, nodeid, dc(journaldate), |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
368 journaltag, action, params) |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
369 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
370 # make sure the normal schema update code doesn't try to |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
371 # change things |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2099
diff
changeset
|
372 self.database_schema['tables'][cn] = klass.schema() |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
373 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
374 def fix_version_2_tables(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
375 # Convert journal date column to TIMESTAMP, params column to TEXT |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
376 self._convert_journal_tables() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
377 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
378 # Convert all String properties to TEXT |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
379 self._convert_string_properties() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
380 |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
381 def __repr__(self): |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
382 return '<myroundsql 0x%x>'%id(self) |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
383 |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
384 def sql_fetchone(self): |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
385 return self.cursor.fetchone() |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
386 |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
387 def sql_fetchall(self): |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
388 return self.cursor.fetchall() |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
389 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
390 def sql_index_exists(self, table_name, index_name): |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
391 self.sql('show index from %s'%table_name) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
392 for index in self.cursor.fetchall(): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
393 if index[2] == index_name: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
394 return 1 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
395 return 0 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
396 |
|
2424
74474ec41050
argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents:
2413
diff
changeset
|
397 def create_class_table(self, spec, create_sequence=1): |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
398 cols, mls = self.determine_columns(spec.properties.items()) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
399 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
400 # add on our special columns |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
401 cols.append(('id', 'INTEGER PRIMARY KEY')) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
402 cols.append(('__retired__', 'INTEGER DEFAULT 0')) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
403 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
404 # create the base table |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
405 scols = ','.join(['%s %s'%x for x in cols]) |
|
3934
8c75e8fce5dc
Fix a couple more old instances of "type" instead of "ENGINE" for mysql backend.
Richard Jones <richard@users.sourceforge.net>
parents:
3929
diff
changeset
|
406 sql = 'create table _%s (%s) ENGINE=%s'%(spec.classname, scols, |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
407 self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
408 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
409 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
410 self.create_class_table_indexes(spec) |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
411 return cols, mls |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
412 |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
413 def create_class_table_indexes(self, spec): |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
414 ''' create the class table for the given spec |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
415 ''' |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
416 # create __retired__ index |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
417 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%( |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
418 spec.classname, spec.classname) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
419 self.sql(index_sql2) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
420 |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
421 # create index for key property |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
422 if spec.key: |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
423 if isinstance(spec.properties[spec.key], hyperdb.String): |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
424 idx = spec.key + '(255)' |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
425 else: |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
426 idx = spec.key |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
427 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%( |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
428 spec.classname, spec.key, |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
429 spec.classname, idx) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
430 self.sql(index_sql3) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
431 |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
432 # TODO: create indexes on (selected?) Link property columns, as |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
433 # they're more likely to be used for lookup |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
434 |
|
3969
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
435 def add_class_key_required_unique_constraint(self, cn, key): |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
436 # mysql requires sizes on TEXT indexes |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
437 prop = self.classes[cn].getprops()[key] |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
438 if isinstance(prop, hyperdb.String): |
|
3969
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
439 sql = '''create unique index _%s_key_retired_idx |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
440 on _%s(__retired__, _%s(255))'''%(cn, cn, key) |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
441 else: |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
442 sql = '''create unique index _%s_key_retired_idx |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
443 on _%s(__retired__, _%s)'''%(cn, cn, key) |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
444 self.sql(sql) |
|
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
445 |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
446 def create_class_table_key_index(self, cn, key): |
|
3969
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3934
diff
changeset
|
447 # mysql requires sizes on TEXT indexes |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
448 prop = self.classes[cn].getprops()[key] |
|
5075
04a8022ae0de
Remove 'import *' statement from dist/backends/back_mysql.py
John Kristensen <john@jerrykan.com>
parents:
5067
diff
changeset
|
449 if isinstance(prop, hyperdb.String): |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
450 sql = 'create index _%s_%s_idx on _%s(_%s(255))'%(cn, key, cn, key) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
451 else: |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
452 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
453 self.sql(sql) |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
454 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
455 def drop_class_table_indexes(self, cn, key): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
456 # drop the old table indexes first |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
457 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
458 if key: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
459 l.append('_%s_%s_idx'%(cn, key)) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
460 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
461 table_name = '_%s'%cn |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
462 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
463 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
464 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
465 index_sql = 'drop index %s on %s'%(index_name, table_name) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
466 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
467 |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
468 def create_journal_table(self, spec): |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
469 ''' create the journal table for a class given the spec and |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
470 already-determined cols |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
471 ''' |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
472 # journal table |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
473 cols = ','.join(['%s varchar'%x |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
474 for x in 'nodeid date tag action params'.split()]) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
475 sql = '''create table %s__journal ( |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
476 nodeid integer, date datetime, tag varchar(255), |
|
3934
8c75e8fce5dc
Fix a couple more old instances of "type" instead of "ENGINE" for mysql backend.
Richard Jones <richard@users.sourceforge.net>
parents:
3929
diff
changeset
|
477 action varchar(255), params text) ENGINE=%s'''%( |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
478 spec.classname, self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
479 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
480 self.create_journal_table_indexes(spec) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
481 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
482 def drop_journal_table_indexes(self, classname): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
483 index_name = '%s_journ_idx'%classname |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
484 if not self.sql_index_exists('%s__journal'%classname, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
485 return |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
486 index_sql = 'drop index %s on %s__journal'%(index_name, classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
487 self.sql(index_sql) |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
488 |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
489 def create_multilink_table(self, spec, ml): |
|
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
490 sql = '''CREATE TABLE `%s_%s` (linkid VARCHAR(255), |
|
3929
a472391156ae
mysql table creation syntax change
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
491 nodeid VARCHAR(255)) ENGINE=%s'''%(spec.classname, ml, |
|
1839
06f5b36b201b
Fix a couple of failures in mysql backend unit tests.
Richard Jones <richard@users.sourceforge.net>
parents:
1836
diff
changeset
|
492 self.mysql_backend) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
493 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
494 self.create_multilink_table_indexes(spec, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
495 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
496 def drop_multilink_table_indexes(self, classname, ml): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
497 l = [ |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
498 '%s_%s_l_idx'%(classname, ml), |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
499 '%s_%s_n_idx'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
500 ] |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
501 table_name = '%s_%s'%(classname, ml) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
502 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
503 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
504 continue |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
505 sql = 'drop index %s on %s'%(index_name, table_name) |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
506 self.sql(sql) |
|
1375
faf93d3fbf2f
added mysql backend
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
507 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
508 def drop_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
509 table_name = '_%s'%cn |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
510 index_name = '_%s_%s_idx'%(cn, key) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
511 if not self.sql_index_exists(table_name, index_name): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
512 return |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
513 sql = 'drop index %s on %s'%(index_name, table_name) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
514 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
515 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
516 # old-skool id generation |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
517 def newid(self, classname): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
518 ''' Generate a new id for the given class |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
519 ''' |
|
2717
ca2ee58c8310
use row locking in MySQL newid() [SF#1034211]
Richard Jones <richard@users.sourceforge.net>
parents:
2693
diff
changeset
|
520 # get the next ID - "FOR UPDATE" will lock the row for us |
|
ca2ee58c8310
use row locking in MySQL newid() [SF#1034211]
Richard Jones <richard@users.sourceforge.net>
parents:
2693
diff
changeset
|
521 sql = 'select num from ids where name=%s FOR UPDATE'%self.arg |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
522 self.sql(sql, (classname, )) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
523 newid = int(self.cursor.fetchone()[0]) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
524 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
525 # update the counter |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
526 sql = 'update ids set num=%s where name=%s'%(self.arg, self.arg) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
527 vals = (int(newid)+1, classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
528 self.sql(sql, vals) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
529 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
530 # return as string |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
531 return str(newid) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
532 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
533 def setid(self, classname, setid): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
534 ''' Set the id counter: used during import of database |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
535 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
536 We add one to make it behave like the seqeunces in postgres. |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
537 ''' |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
538 sql = 'update ids set num=%s where name=%s'%(self.arg, self.arg) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
539 vals = (int(setid)+1, classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
540 self.sql(sql, vals) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
541 |
|
3310
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
542 def clear(self): |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
543 rdbms_common.Database.clear(self) |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
544 |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
545 # set the id counters to 0 (setid adds one) so we start at 1 |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
546 for cn in self.classes.keys(): |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
547 self.setid(cn, 0) |
|
3518d1ffd940
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
3216
diff
changeset
|
548 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
549 def create_class(self, spec): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
550 rdbms_common.Database.create_class(self, spec) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
551 sql = 'insert into ids (name, num) values (%s, %s)' |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
552 vals = (spec.classname, 1) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
553 self.sql(sql, vals) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
554 |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3635
diff
changeset
|
555 def sql_commit(self, fail_ok=False): |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
556 ''' Actually commit to the database. |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
557 ''' |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
3969
diff
changeset
|
558 self.log_info('commit') |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3635
diff
changeset
|
559 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3635
diff
changeset
|
560 # MySQL commits don't seem to ever fail, the latest update winning. |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3635
diff
changeset
|
561 # makes you wonder why they have transactions... |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
562 self.conn.commit() |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
563 |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
564 # open a new cursor for subsequent work |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
565 self.cursor = self.conn.cursor() |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
566 |
|
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
567 # make sure we're in a new transaction and not autocommitting |
|
3216
27aba1d8fccc
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
568 self.sql("SET AUTOCOMMIT=0") |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
569 self.sql("START TRANSACTION") |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2431
diff
changeset
|
570 |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
571 def sql_close(self): |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
3969
diff
changeset
|
572 self.log_info('close') |
|
3457
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
573 try: |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
574 self.conn.close() |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
575 except MySQLdb.ProgrammingError, message: |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
576 if str(message) != 'closing a closed connection': |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
577 raise |
|
88f0a5030404
Fix invalid date spec in db tests
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
578 |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
579 class MysqlClass: |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4285
diff
changeset
|
580 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4285
diff
changeset
|
581 def supports_subselects(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4285
diff
changeset
|
582 # TODO: AFAIK its version dependent for MySQL |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4285
diff
changeset
|
583 return False |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4285
diff
changeset
|
584 |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
585 def _subselect(self, classname, multilink_table): |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3603
diff
changeset
|
586 ''' "I can't believe it's not a toy RDBMS" |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3603
diff
changeset
|
587 see, even toy RDBMSes like gadfly and sqlite can do sub-selects... |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
588 ''' |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
589 self.db.sql('select nodeid from %s'%multilink_table) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3603
diff
changeset
|
590 s = ','.join([x[0] for x in self.db.sql_fetchall()]) |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
591 return '_%s.id not in (%s)'%(classname, s) |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
592 |
| 4285 | 593 def create_inner(self, **propvalues): |
| 594 try: | |
| 595 return rdbms_common.Class.create_inner(self, **propvalues) | |
| 596 except MySQLdb.IntegrityError, e: | |
| 597 self._handle_integrity_error(e, propvalues) | |
| 598 | |
| 599 def set_inner(self, nodeid, **propvalues): | |
| 600 try: | |
| 601 return rdbms_common.Class.set_inner(self, nodeid, | |
| 602 **propvalues) | |
| 603 except MySQLdb.IntegrityError, e: | |
| 604 self._handle_integrity_error(e, propvalues) | |
| 605 | |
| 606 def _handle_integrity_error(self, e, propvalues): | |
| 607 ''' Handle a MySQL IntegrityError. | |
| 608 | |
| 609 If the error is recognized, then it may be converted into an | |
| 610 alternative exception. Otherwise, it is raised unchanged from | |
| 611 this function.''' | |
| 612 | |
| 613 # There are checks in create_inner/set_inner to see if a node | |
| 614 # is being created with the same key as an existing node. | |
| 615 # But, there is a race condition -- we may pass those checks, | |
| 616 # only to find out that a parallel session has created the | |
| 617 # node by by the time we actually issue the SQL command to | |
| 618 # create the node. Fortunately, MySQL gives us a unique error | |
| 619 # code for this situation, so we can detect it here and handle | |
| 620 # it appropriately. | |
| 621 # | |
| 622 # The details of the race condition are as follows, where | |
| 623 # "X" is a classname, and the term "thread" is meant to | |
| 624 # refer generically to both threads and processes: | |
| 625 # | |
| 626 # Thread A Thread B | |
| 627 # -------- -------- | |
| 628 # read table for X | |
| 629 # create new X object | |
| 630 # commit | |
| 631 # create new X object | |
| 632 # | |
| 633 # In Thread B, the check in create_inner does not notice that | |
| 634 # the new X object is a duplicate of that committed in Thread | |
| 635 # A because MySQL's default "consistent nonlocking read" | |
| 636 # behavior means that Thread B sees a snapshot of the database | |
| 637 # at the point at which its transaction began -- which was | |
| 638 # before Thread A created the object. However, the attempt | |
| 639 # to *write* to the table for X, creating a duplicate entry, | |
| 640 # triggers an error at the point of the write. | |
| 641 # | |
| 642 # If both A and B's transaction begins with creating a new X | |
| 643 # object, then this bug cannot occur because creating the | |
| 644 # object requires getting a new ID, and newid() locks the id | |
| 645 # table until the transaction is committed or rolledback. So, | |
| 646 # B will block until A's commit is complete, and will not | |
| 647 # actually get its snapshot until A's transaction completes. | |
| 648 # But, if the transaction has begun prior to calling newid, | |
| 649 # then the snapshot has already been established. | |
| 650 if e[0] == ER.DUP_ENTRY: | |
| 651 key = propvalues[self.key] | |
| 652 raise ValueError, 'node with key "%s" exists' % key | |
| 653 # We don't know what this exception is; reraise it. | |
| 654 raise | |
| 655 | |
| 656 | |
|
1415
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
657 class Class(MysqlClass, rdbms_common.Class): |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
658 pass |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
659 class IssueClass(MysqlClass, rdbms_common.IssueClass): |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
660 pass |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
661 class FileClass(MysqlClass, rdbms_common.FileClass): |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
662 pass |
|
6883852e9b15
mysql backend passes all tests (at last!)
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1375
diff
changeset
|
663 |
|
2679
b3f0b7b9d20d
for MySQL connection, password argument is called 'passwd'.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2634
diff
changeset
|
664 # vim: set et sts=4 sw=4 : |
