Mercurial > p > roundup > code
annotate roundup/backends/rdbms_common.py @ 3224:1fd11a9803bb
use backend datatype for journal timestamps.
this fixes DateTime parsing errors raised by sqlite if mx.DateTime is installed
| author | Alexander Smishlajev <a1s@users.sourceforge.net> |
|---|---|
| date | Wed, 02 Mar 2005 14:03:44 +0000 |
| parents | 57b60bda9473 |
| children | 440f0a6a2e3c |
| rev | line source |
|---|---|
|
3224
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
1 # $Id: rdbms_common.py,v 1.150 2005-03-02 14:03:44 a1s Exp $ |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
2 ''' Relational database (SQL) backend common code. |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
3 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
4 Basics: |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
5 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
6 - map roundup classes to relational tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
7 - automatically detect schema changes and modify the table schemas |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
8 appropriately (we store the "database version" of the schema in the |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
9 database itself as the only row of the "schema" table) |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
10 - multilinks (which represent a many-to-many relationship) are handled through |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
11 intermediate tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
12 - journals are stored adjunct to the per-class tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
13 - table names and columns have "_" prepended so the names can't clash with |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
14 restricted names (like "order") |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
15 - retirement is determined by the __retired__ column being true |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
16 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
17 Database-specific changes may generally be pushed out to the overridable |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
18 sql_* methods, since everything else should be fairly generic. There's |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
19 probably a bit of work to be done if a database is used that actually |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
20 honors column typing, since the initial databases don't (sqlite stores |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
21 everything as a string.) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
22 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
23 The schema of the hyperdb being mapped to the database is stored in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
24 database itself as a repr()'ed dictionary of information about each Class |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
25 that maps to a table. If that information differs from the hyperdb schema, |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
26 then we update it. We also store in the schema dict a version which |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
27 allows us to upgrade the database schema when necessary. See upgrade_db(). |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
28 ''' |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
29 __docformat__ = 'restructuredtext' |
| 1165 | 30 |
| 31 # standard python modules | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
32 import sys, os, time, re, errno, weakref, copy, logging |
| 1165 | 33 |
| 34 # roundup modules | |
| 35 from roundup import hyperdb, date, password, roundupdb, security | |
| 36 from roundup.hyperdb import String, Password, Date, Interval, Link, \ | |
|
1417
472c21af7f69
fixed error in indexargs_url (thanks Patrick Ohly)
Richard Jones <richard@users.sourceforge.net>
parents:
1415
diff
changeset
|
37 Multilink, DatabaseError, Boolean, Number, Node |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
38 from roundup.backends import locking |
| 1165 | 39 |
| 40 # support | |
| 41 from blobfiles import FileStorage | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
42 from indexer_rdbms import Indexer |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
43 from sessions_rdbms import Sessions, OneTimeKeys |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
44 from roundup.date import Range |
| 1165 | 45 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
46 # number of rows to keep in memory |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
47 ROW_CACHE_SIZE = 100 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
48 |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
49 # dummy value meaning "argument not passed" |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
50 _marker = [] |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
51 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
52 def _num_cvt(num): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
53 num = str(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
54 try: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
55 return int(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
56 except: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
57 return float(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
58 |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
59 def _bool_cvt(value): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
60 if value in ('TRUE', 'FALSE'): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
61 return {'TRUE': 1, 'FALSE': 0}[value] |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
62 # assume it's a number returned from the db API |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
63 return int(value) |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
64 |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
65 def connection_dict(config, dbnamestr=None): |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
66 ''' Used by Postgresql and MySQL to detemine the keyword args for |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
67 opening the database connection.''' |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
68 d = { } |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
69 if dbnamestr: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
70 d[dbnamestr] = config.RDBMS_NAME |
|
3099
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
71 for name in ('host', 'port', 'password', 'user', 'read_default_group', |
|
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
72 'read_default_file'): |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
73 cvar = 'RDBMS_'+name.upper() |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
74 if config[cvar] is not None: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
75 d[name] = config[cvar] |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
76 return d |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
77 |
| 1165 | 78 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
79 ''' Wrapper around an SQL database that presents a hyperdb interface. |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
80 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
81 - some functionality is specific to the actual SQL database, hence |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
82 the sql_* methods that are NotImplemented |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
83 - we keep a cache of the latest ROW_CACHE_SIZE row fetches. |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
84 ''' |
| 1165 | 85 def __init__(self, config, journaltag=None): |
| 86 ''' Open the database and load the schema from it. | |
| 87 ''' | |
| 88 self.config, self.journaltag = config, journaltag | |
| 89 self.dir = config.DATABASE | |
| 90 self.classes = {} | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
91 self.indexer = Indexer(self) |
| 1165 | 92 self.security = security.Security(self) |
| 93 | |
| 94 # additional transaction support for external files and the like | |
| 95 self.transactions = [] | |
| 96 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
97 # keep a cache of the N most recently retrieved rows of any kind |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
98 # (classname, nodeid) = row |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
99 self.cache = {} |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
100 self.cache_lru = [] |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
101 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
102 'filtering': 0} |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
103 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
104 # database lock |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
105 self.lockfile = None |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
106 |
| 1165 | 107 # open a connection to the database, creating the "conn" attribute |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
108 self.open_connection() |
| 1165 | 109 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
110 def clearCache(self): |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
111 self.cache = {} |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
112 self.cache_lru = [] |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
113 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
114 def getSessionManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
115 return Sessions(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
116 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
117 def getOTKManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
118 return OneTimeKeys(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
119 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
120 def open_connection(self): |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
121 ''' Open a connection to the database, creating it if necessary. |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
122 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
123 Must call self.load_dbschema() |
| 1165 | 124 ''' |
| 125 raise NotImplemented | |
| 126 | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
127 def sql(self, sql, args=None): |
| 1165 | 128 ''' Execute the sql with the optional args. |
| 129 ''' | |
| 130 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
131 logging.getLogger('hyperdb').debug('SQL %r %r'%(sql, args)) |
| 1165 | 132 if args: |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
133 self.cursor.execute(sql, args) |
| 1165 | 134 else: |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
135 self.cursor.execute(sql) |
| 1165 | 136 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
137 def sql_fetchone(self): |
| 1165 | 138 ''' Fetch a single row. If there's nothing to fetch, return None. |
| 139 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
140 return self.cursor.fetchone() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
141 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
142 def sql_fetchall(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
143 ''' Fetch all rows. If there's nothing to fetch, return []. |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
144 ''' |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
145 return self.cursor.fetchall() |
| 1165 | 146 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
147 def sql_stringquote(self, value): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
148 ''' Quote the string so it's safe to put in the 'sql quotes' |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
149 ''' |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
150 return re.sub("'", "''", str(value)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
151 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
152 def init_dbschema(self): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
153 self.database_schema = { |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
154 'version': self.current_db_version, |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
155 'tables': {} |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
156 } |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
157 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
158 def load_dbschema(self): |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
159 ''' Load the schema definition that the database currently implements |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
160 ''' |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
161 self.cursor.execute('select schema from schema') |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
162 schema = self.cursor.fetchone() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
163 if schema: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
164 self.database_schema = eval(schema[0]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
165 else: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
166 self.database_schema = {} |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
167 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
168 def save_dbschema(self): |
| 1165 | 169 ''' Save the schema definition that the database currently implements |
| 170 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
171 s = repr(self.database_schema) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
172 self.sql('delete from schema') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
173 self.sql('insert into schema values (%s)', (s,)) |
| 1165 | 174 |
| 175 def post_init(self): | |
| 176 ''' Called once the schema initialisation has finished. | |
| 177 | |
| 178 We should now confirm that the schema defined by our "classes" | |
| 179 attribute actually matches the schema in the database. | |
| 180 ''' | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
181 save = self.upgrade_db() |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
182 |
| 1165 | 183 # now detect changes in the schema |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
184 tables = self.database_schema['tables'] |
| 1165 | 185 for classname, spec in self.classes.items(): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
186 if tables.has_key(classname): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
187 dbspec = tables[classname] |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
188 if self.update_class(spec, dbspec): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
189 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
190 save = 1 |
| 1165 | 191 else: |
| 192 self.create_class(spec) | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
193 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
194 save = 1 |
| 1165 | 195 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
196 for classname, spec in tables.items(): |
| 1165 | 197 if not self.classes.has_key(classname): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
198 self.drop_class(classname, tables[classname]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
199 del tables[classname] |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
200 save = 1 |
| 1165 | 201 |
| 202 # update the database version of the schema | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
203 if save: |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
204 self.save_dbschema() |
| 1165 | 205 |
| 206 # reindex the db if necessary | |
| 207 if self.indexer.should_reindex(): | |
| 208 self.reindex() | |
| 209 | |
| 210 # commit | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
211 self.sql_commit() |
| 1165 | 212 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
213 # update this number when we need to make changes to the SQL structure |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
214 # of the backen database |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
215 current_db_version = 4 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
216 def upgrade_db(self): |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
217 ''' Update the SQL database to reflect changes in the backend code. |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
218 |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
219 Return boolean whether we need to save the schema. |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
220 ''' |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
221 version = self.database_schema.get('version', 1) |
|
2081
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
222 if version == self.current_db_version: |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
223 # nothing to do |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
224 return 0 |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
225 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
226 if version < 2: |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2429
diff
changeset
|
227 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
228 logging.getLogger('hyperdb').info('upgrade to version 2') |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
229 # change the schema structure |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
230 self.database_schema = {'tables': self.database_schema} |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
231 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
232 # version 1 didn't have the actor column (note that in |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
233 # MySQL this will also transition the tables to typed columns) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
234 self.add_new_columns_v2() |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
235 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
236 # version 1 doesn't have the OTK, session and indexing in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
237 # database |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
238 self.create_version_2_tables() |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
239 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
240 if version < 3: |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2429
diff
changeset
|
241 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
242 logging.getLogger('hyperdb').info('upgrade to version 3') |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
243 self.fix_version_2_tables() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
244 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
245 if version < 4: |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
246 self.fix_version_3_tables() |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
247 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
248 self.database_schema['version'] = self.current_db_version |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
249 return 1 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
250 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
251 def fix_version_3_tables(self): |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
252 # drop the shorter VARCHAR OTK column and add a new TEXT one |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
253 for name in ('otk', 'session'): |
|
2755
e3cd28cec23d
Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents:
2733
diff
changeset
|
254 self.sql('DELETE FROM %ss'%name) |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
255 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
256 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
257 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
258 def fix_version_2_tables(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
259 '''Default (used by sqlite): NOOP''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
260 pass |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
261 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
262 def _convert_journal_tables(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
263 '''Get current journal table contents, drop the table and re-create''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
264 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
265 cols = ','.join('nodeid date tag action params'.split()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
266 for klass in self.classes.values(): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
267 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
268 sql = 'select %s from %s__journal order by date'%(cols, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
269 klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
270 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
271 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
272 self.drop_journal_table_indexes(klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
273 c.execute('drop table %s__journal'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
274 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
275 # re-create and re-populate |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
276 self.create_journal_table(klass) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
277 a = self.arg |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
278 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
279 klass.classname, cols, a, a, a, a, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
280 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
281 # no data conversion needed |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
282 self.cursor.execute(sql, row) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
283 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
284 def _convert_string_properties(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
285 '''Get current Class tables that contain String properties, and |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
286 convert the VARCHAR columns to TEXT''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
287 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
288 for klass in self.classes.values(): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
289 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
290 cols, mls = self.determine_columns(klass.properties.items()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
291 scols = ','.join([i[0] for i in cols]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
292 sql = 'select id,%s from _%s'%(scols, klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
293 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
294 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
295 self.drop_class_table_indexes(klass.classname, klass.getkey()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
296 c.execute('drop table _%s'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
297 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
298 # re-create and re-populate |
|
2424
74474ec41050
argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents:
2416
diff
changeset
|
299 self.create_class_table(klass, create_sequence=0) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
300 a = ','.join([self.arg for i in range(len(cols)+1)]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
301 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
302 scols, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
303 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
304 l = [] |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
305 for entry in row: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
306 # mysql will already be a string - psql needs "help" |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
307 if entry is not None and not isinstance(entry, type('')): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
308 entry = str(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
309 l.append(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
310 self.cursor.execute(sql, l) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
311 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
312 def refresh_database(self): |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
313 self.post_init() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
314 |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
315 |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
316 def reindex(self, classname=None): |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
317 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
318 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
319 else: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
320 classes = self.classes.values() |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
321 for klass in classes: |
| 1165 | 322 for nodeid in klass.list(): |
| 323 klass.index(nodeid) | |
| 324 | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
325 hyperdb_to_sql_datatypes = { |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
326 hyperdb.String : 'TEXT', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
327 hyperdb.Date : 'TIMESTAMP', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
328 hyperdb.Link : 'INTEGER', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
329 hyperdb.Interval : 'VARCHAR(255)', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
330 hyperdb.Password : 'VARCHAR(255)', |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
331 hyperdb.Boolean : 'BOOLEAN', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
332 hyperdb.Number : 'REAL', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
333 } |
| 1165 | 334 def determine_columns(self, properties): |
| 335 ''' Figure the column names and multilink properties from the spec | |
| 336 | |
| 337 "properties" is a list of (name, prop) where prop may be an | |
| 338 instance of a hyperdb "type" _or_ a string repr of that type. | |
| 339 ''' | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
340 cols = [ |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
341 ('_actor', self.hyperdb_to_sql_datatypes[hyperdb.Link]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
342 ('_activity', self.hyperdb_to_sql_datatypes[hyperdb.Date]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
343 ('_creator', self.hyperdb_to_sql_datatypes[hyperdb.Link]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
344 ('_creation', self.hyperdb_to_sql_datatypes[hyperdb.Date]), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
345 ] |
| 1165 | 346 mls = [] |
| 347 # add the multilinks separately | |
| 348 for col, prop in properties: | |
| 349 if isinstance(prop, Multilink): | |
| 350 mls.append(col) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
351 continue |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
352 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
353 if isinstance(prop, type('')): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
354 raise ValueError, "string property spec!" |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
355 #and prop.find('Multilink') != -1: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
356 #mls.append(col) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
357 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
358 datatype = self.hyperdb_to_sql_datatypes[prop.__class__] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
359 cols.append(('_'+col, datatype)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
360 |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
361 # Intervals stored as two columns |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
362 if isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
363 cols.append(('__'+col+'_int__', 'BIGINT')) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
364 |
| 1165 | 365 cols.sort() |
| 366 return cols, mls | |
| 367 | |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
368 def update_class(self, spec, old_spec, force=0): |
| 1165 | 369 ''' Determine the differences between the current spec and the |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
370 database version of the spec, and update where necessary. |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
371 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
372 If 'force' is true, update the database anyway. |
| 1165 | 373 ''' |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
374 new_has = spec.properties.has_key |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
375 new_spec = spec.schema() |
|
1515
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
376 new_spec[1].sort() |
|
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
377 old_spec[1].sort() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
378 if not force and new_spec == old_spec: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
379 # no changes |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
380 return 0 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
381 |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
382 logger = logging.getLogger('hyperdb') |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
383 logger.info('update_class %s'%spec.classname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
384 |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
385 logger.debug('old_spec %r'%(old_spec,)) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
386 logger.debug('new_spec %r'%(new_spec,)) |
| 1165 | 387 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
388 # detect key prop change for potential index change |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
389 keyprop_changes = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
390 if new_spec[0] != old_spec[0]: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
391 keyprop_changes = {'remove': old_spec[0], 'add': new_spec[0]} |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
392 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
393 # detect multilinks that have been removed, and drop their table |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
394 old_has = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
395 for name, prop in old_spec[1]: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
396 old_has[name] = 1 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
397 if new_has(name): |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
398 continue |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
399 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
400 if prop.find('Multilink to') != -1: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
401 # first drop indexes. |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
402 self.drop_multilink_table_indexes(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
403 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
404 # now the multilink table itself |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
405 sql = 'drop table %s_%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
406 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
407 # if this is the key prop, drop the index first |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
408 if old_spec[0] == prop: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
409 self.drop_class_table_key_index(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
410 del keyprop_changes['remove'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
411 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
412 # drop the column |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
413 sql = 'alter table _%s drop column _%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
414 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
415 self.sql(sql) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
416 old_has = old_has.has_key |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
417 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
418 # if we didn't remove the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
419 # changed, we still need to remove the old index |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
420 if keyprop_changes.has_key('remove'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
421 self.drop_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
422 keyprop_changes['remove']) |
| 1165 | 423 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
424 # add new columns |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
425 for propname, prop in new_spec[1]: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
426 if old_has(propname): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
427 continue |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
428 prop = spec.properties[propname] |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
429 if isinstance(prop, Multilink): |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
430 self.create_multilink_table(spec, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
431 else: |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
432 # add the column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
433 coltype = self.hyperdb_to_sql_datatypes[prop.__class__] |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
434 sql = 'alter table _%s add column _%s %s'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
435 spec.classname, propname, coltype) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
436 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
437 |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
438 # extra Interval column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
439 if isinstance(prop, Interval): |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
440 sql = 'alter table _%s add column __%s_int__ BIGINT'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
441 spec.classname, propname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
442 self.sql(sql) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
443 |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
444 # if the new column is a key prop, we need an index! |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
445 if new_spec[0] == propname: |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
446 self.create_class_table_key_index(spec.classname, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
447 del keyprop_changes['add'] |
| 1165 | 448 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
449 # if we didn't add the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
450 # changed, we still need to add the new index |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
451 if keyprop_changes.has_key('add'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
452 self.create_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
453 keyprop_changes['add']) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
454 |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
455 return 1 |
| 1165 | 456 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
457 def determine_all_columns(self, spec): |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
458 """Figure out the columns from the spec and also add internal columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
459 |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
460 """ |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
461 cols, mls = self.determine_columns(spec.properties.items()) |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
462 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
463 # add on our special columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
464 cols.append(('id', 'INTEGER PRIMARY KEY')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
465 cols.append(('__retired__', 'INTEGER DEFAULT 0')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
466 return cols, mls |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
467 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
468 def create_class_table(self, spec): |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
469 '''Create the class table for the given Class "spec". Creates the |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
470 indexes too.''' |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
471 cols, mls = self.determine_all_columns(spec) |
| 1165 | 472 |
| 473 # create the base table | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
474 scols = ','.join(['%s %s'%x for x in cols]) |
| 1165 | 475 sql = 'create table _%s (%s)'%(spec.classname, scols) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
476 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
477 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
478 self.create_class_table_indexes(spec) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
479 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
480 return cols, mls |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
481 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
482 def create_class_table_indexes(self, spec): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
483 ''' create the class table for the given spec |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
484 ''' |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
485 # create __retired__ index |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
486 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%( |
|
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
487 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
488 self.sql(index_sql2) |
| 1165 | 489 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
490 # create index for key property |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
491 if spec.key: |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
492 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%( |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
493 spec.classname, spec.key, |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
494 spec.classname, spec.key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
495 self.sql(index_sql3) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
496 |
|
2228
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
497 # TODO: create indexes on (selected?) Link property columns, as |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
498 # they're more likely to be used for lookup |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
499 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
500 def drop_class_table_indexes(self, cn, key): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
501 # drop the old table indexes first |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
502 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
503 if key: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
504 l.append('_%s_%s_idx'%(cn, key)) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
505 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
506 table_name = '_%s'%cn |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
507 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
508 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
509 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
510 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
511 self.sql(index_sql) |
| 1165 | 512 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
513 def create_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
514 ''' create the class table for the given spec |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
515 ''' |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
516 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
517 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
518 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
519 def drop_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
520 table_name = '_%s'%cn |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
521 index_name = '_%s_%s_idx'%(cn, key) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
522 if not self.sql_index_exists(table_name, index_name): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
523 return |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
524 sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
525 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
526 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
527 def create_journal_table(self, spec): |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
528 ''' create the journal table for a class given the spec and |
| 1165 | 529 already-determined cols |
| 530 ''' | |
| 531 # journal table | |
| 532 cols = ','.join(['%s varchar'%x | |
| 533 for x in 'nodeid date tag action params'.split()]) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
534 sql = '''create table %s__journal ( |
|
3224
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
535 nodeid integer, date %s, tag varchar(255), |
|
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
536 action varchar(255), params text)''' % (spec.classname, |
|
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
537 self.hyperdb_to_sql_datatypes[hyperdb.Date]) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
538 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
539 self.create_journal_table_indexes(spec) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
540 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
541 def create_journal_table_indexes(self, spec): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
542 # index on nodeid |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
543 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%( |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
544 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
545 self.sql(sql) |
| 1165 | 546 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
547 def drop_journal_table_indexes(self, classname): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
548 index_name = '%s_journ_idx'%classname |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
549 if not self.sql_index_exists('%s__journal'%classname, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
550 return |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
551 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
552 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
553 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
554 def create_multilink_table(self, spec, ml): |
| 1165 | 555 ''' Create a multilink table for the "ml" property of the class |
| 556 given by the spec | |
| 557 ''' | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
558 # create the table |
|
2828
3a2f138fcb28
use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2755
diff
changeset
|
559 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%( |
| 1165 | 560 spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
561 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
562 self.create_multilink_table_indexes(spec, ml) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
563 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
564 def create_multilink_table_indexes(self, spec, ml): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
565 # create index on linkid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
566 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
567 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
568 self.sql(index_sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
569 |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
570 # create index on nodeid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
571 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
572 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
573 self.sql(index_sql) |
| 1165 | 574 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
575 def drop_multilink_table_indexes(self, classname, ml): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
576 l = [ |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
577 '%s_%s_l_idx'%(classname, ml), |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
578 '%s_%s_n_idx'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
579 ] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
580 table_name = '%s_%s'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
581 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
582 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
583 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
584 index_sql = 'drop index %s'%index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
585 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
586 |
| 1165 | 587 def create_class(self, spec): |
| 588 ''' Create a database table according to the given spec. | |
| 589 ''' | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
590 cols, mls = self.create_class_table(spec) |
|
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
591 self.create_journal_table(spec) |
| 1165 | 592 |
| 593 # now create the multilink tables | |
| 594 for ml in mls: | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
595 self.create_multilink_table(spec, ml) |
| 1165 | 596 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
597 def drop_class(self, cn, spec): |
| 1165 | 598 ''' Drop the given table from the database. |
| 599 | |
| 600 Drop the journal and multilink tables too. | |
| 601 ''' | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
602 properties = spec[1] |
| 1165 | 603 # figure the multilinks |
| 604 mls = [] | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
605 for propanme, prop in properties: |
| 1165 | 606 if isinstance(prop, Multilink): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
607 mls.append(propname) |
| 1165 | 608 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
609 # drop class table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
610 self.drop_class_table_indexes(cn, spec[0]) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
611 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
612 self.drop_class_table(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
613 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
614 # drop journal table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
615 self.drop_journal_table_indexes(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
616 sql = 'drop table %s__journal'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
617 self.sql(sql) |
| 1165 | 618 |
| 619 for ml in mls: | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
620 # drop multilink table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
621 self.drop_multilink_table_indexes(cn, ml) |
| 1165 | 622 sql = 'drop table %s_%s'%(spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
623 self.sql(sql) |
| 1165 | 624 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
625 def drop_class_table(self, cn): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
626 sql = 'drop table _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
627 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
628 |
| 1165 | 629 # |
| 630 # Classes | |
| 631 # | |
| 632 def __getattr__(self, classname): | |
| 633 ''' A convenient way of calling self.getclass(classname). | |
| 634 ''' | |
| 635 if self.classes.has_key(classname): | |
| 636 return self.classes[classname] | |
| 637 raise AttributeError, classname | |
| 638 | |
| 639 def addclass(self, cl): | |
| 640 ''' Add a Class to the hyperdatabase. | |
| 641 ''' | |
| 642 cn = cl.classname | |
| 643 if self.classes.has_key(cn): | |
| 644 raise ValueError, cn | |
| 645 self.classes[cn] = cl | |
| 646 | |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
647 # add default Edit and View permissions |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
648 self.security.addPermission(name="Create", klass=cn, |
|
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
649 description="User is allowed to create "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
650 self.security.addPermission(name="Edit", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
651 description="User is allowed to edit "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
652 self.security.addPermission(name="View", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
653 description="User is allowed to access "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
654 |
| 1165 | 655 def getclasses(self): |
| 656 ''' Return a list of the names of all existing classes. | |
| 657 ''' | |
| 658 l = self.classes.keys() | |
| 659 l.sort() | |
| 660 return l | |
| 661 | |
| 662 def getclass(self, classname): | |
| 663 '''Get the Class object representing a particular class. | |
| 664 | |
| 665 If 'classname' is not a valid class name, a KeyError is raised. | |
| 666 ''' | |
| 667 try: | |
| 668 return self.classes[classname] | |
| 669 except KeyError: | |
| 670 raise KeyError, 'There is no class called "%s"'%classname | |
| 671 | |
| 672 def clear(self): | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
673 '''Delete all database contents. |
| 1165 | 674 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
675 Note: I don't commit here, which is different behaviour to the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
676 "nuke from orbit" behaviour in the dbs. |
| 1165 | 677 ''' |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
678 logging.getLogger('hyperdb').info('clear') |
| 1165 | 679 for cn in self.classes.keys(): |
| 680 sql = 'delete from _%s'%cn | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
681 self.sql(sql) |
|
3057
b77961cdb9a0
revert ID back to 1, not 0
Richard Jones <richard@users.sourceforge.net>
parents:
3016
diff
changeset
|
682 self.setid(cn, 1) |
| 1165 | 683 |
| 684 # | |
| 685 # Nodes | |
| 686 # | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
687 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
688 hyperdb_to_sql_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
689 hyperdb.String : str, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
690 # fractional seconds by default |
|
2102
666402433998
Fix some tests.
Richard Jones <richard@users.sourceforge.net>
parents:
2100
diff
changeset
|
691 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%.3f'), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
692 hyperdb.Link : int, |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
693 hyperdb.Interval : str, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
694 hyperdb.Password : str, |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
695 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
696 hyperdb.Number : lambda x: x, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
697 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
698 } |
| 1165 | 699 def addnode(self, classname, nodeid, node): |
| 700 ''' Add the specified node to its class's db. | |
| 701 ''' | |
| 702 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
703 logging.getLogger('hyperdb').debug('addnode %s%s %r'%(classname, |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
704 nodeid, node)) |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
705 |
|
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
706 # determine the column definitions and multilink tables |
| 1165 | 707 cl = self.classes[classname] |
| 708 cols, mls = self.determine_columns(cl.properties.items()) | |
| 709 | |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
710 # we'll be supplied these props if we're doing an import |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
711 values = node.copy() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
712 if not values.has_key('creator'): |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
713 # add in the "calculated" properties (dupe so we don't affect |
|
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
714 # calling code's node assumptions) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
715 values['creation'] = values['activity'] = date.Date() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
716 values['actor'] = values['creator'] = self.getuid() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
717 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
718 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
719 props = cl.getprops(protected=1) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
720 del props['id'] |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
721 |
| 1165 | 722 # default the non-multilink columns |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
723 for col, prop in props.items(): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
724 if not values.has_key(col): |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
725 if isinstance(prop, Multilink): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
726 values[col] = [] |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
727 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
728 values[col] = None |
| 1165 | 729 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
730 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
731 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
732 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
733 del self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
734 self.cache_lru.remove(key) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
735 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
736 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
737 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
738 for col,dt in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
739 # this is somewhat dodgy.... |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
740 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
741 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
742 value = values[col[2:-6]] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
743 # this is an Interval special "int" column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
744 if value is not None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
745 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
746 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
747 vals.append(value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
748 continue |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
749 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
750 prop = props[col[1:]] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
751 value = values[col[1:]] |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
752 if value is not None: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
753 value = self.hyperdb_to_sql_value[prop.__class__](value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
754 vals.append(value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
755 vals.append(nodeid) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
756 vals = tuple(vals) |
| 1165 | 757 |
| 758 # make sure the ordering is correct for column name -> column value | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
759 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
760 cols = ','.join([col for col,dt in cols]) + ',id' |
| 1165 | 761 |
| 762 # perform the inserts | |
| 763 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
764 self.sql(sql, vals) |
| 1165 | 765 |
| 766 # insert the multilink rows | |
| 767 for col in mls: | |
| 768 t = '%s_%s'%(classname, col) | |
| 769 for entry in node[col]: | |
| 770 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, | |
| 771 self.arg, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
772 self.sql(sql, (entry, nodeid)) |
| 1165 | 773 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
774 def setnode(self, classname, nodeid, values, multilink_changes={}): |
| 1165 | 775 ''' Change the specified node. |
| 776 ''' | |
| 777 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
778 logging.getLogger('hyperdb').debug('setnode %s%s %r' |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
779 % (classname, nodeid, values)) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
780 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
781 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
782 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
783 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
784 del self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
785 self.cache_lru.remove(key) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
786 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
787 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
788 props = cl.getprops() |
| 1165 | 789 |
| 790 cols = [] | |
| 791 mls = [] | |
| 792 # add the multilinks separately | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
793 for col in values.keys(): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
794 prop = props[col] |
| 1165 | 795 if isinstance(prop, Multilink): |
| 796 mls.append(col) | |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
797 elif isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
798 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
799 cols.append(col) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
800 # extra leading '_' added by code below |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
801 cols.append('_' +col + '_int__') |
| 1165 | 802 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
803 cols.append(col) |
| 1165 | 804 cols.sort() |
| 805 | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
806 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
807 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
808 for col in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
809 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
810 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
811 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
812 col = col[1:-6] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
813 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
814 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
815 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
816 vals.append(None) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
817 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
818 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
819 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
820 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
821 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
822 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
823 e = None |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
824 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
825 e = self.hyperdb_to_sql_value[prop.__class__](value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
826 vals.append(e) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
827 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
828 vals.append(int(nodeid)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
829 vals = tuple(vals) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
830 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
831 # if there's any updates to regular columns, do them |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
832 if cols: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
833 # make sure the ordering is correct for column name -> column value |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
834 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols]) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
835 cols = ','.join(cols) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
836 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
837 # perform the update |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
838 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
839 self.sql(sql, vals) |
| 1165 | 840 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
841 # we're probably coming from an import, not a change |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
842 if not multilink_changes: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
843 for name in mls: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
844 prop = props[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
845 value = values[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
846 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
847 t = '%s_%s'%(classname, name) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
848 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
849 # clear out previous values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
850 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
851 self.sql('delete from %s where nodeid=%s'%(t, self.arg), |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
852 (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
853 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
854 # insert the values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
855 for entry in values[name]: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
856 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
857 self.arg, self.arg) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
858 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
859 self.sql(sql, (entry, nodeid)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
860 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
861 # we have multilink changes to apply |
| 1165 | 862 for col, (add, remove) in multilink_changes.items(): |
| 863 tn = '%s_%s'%(classname, col) | |
| 864 if add: | |
| 865 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn, | |
| 866 self.arg, self.arg) | |
| 867 for addid in add: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
868 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
869 self.sql(sql, (int(nodeid), int(addid))) |
| 1165 | 870 if remove: |
| 871 sql = 'delete from %s where nodeid=%s and linkid=%s'%(tn, | |
| 872 self.arg, self.arg) | |
| 873 for removeid in remove: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
874 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
875 self.sql(sql, (int(nodeid), int(removeid))) |
| 1165 | 876 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
877 sql_to_hyperdb_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
878 hyperdb.String : str, |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
879 hyperdb.Date : lambda x:date.Date(str(x).replace(' ', '.')), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
880 # hyperdb.Link : int, # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
881 hyperdb.Link : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
882 hyperdb.Interval : date.Interval, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
883 hyperdb.Password : lambda x: password.Password(encrypted=x), |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
884 hyperdb.Boolean : _bool_cvt, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
885 hyperdb.Number : _num_cvt, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
886 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
887 } |
| 1165 | 888 def getnode(self, classname, nodeid): |
| 889 ''' Get a node from the database. | |
| 890 ''' | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
891 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
892 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
893 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
894 # push us back to the top of the LRU |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
895 self.cache_lru.remove(key) |
|
1185
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
896 self.cache_lru.insert(0, key) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
897 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
898 self.stats['cache_hits'] += 1 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
899 # return the cached information |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
900 return self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
901 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
902 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
903 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
904 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
905 |
| 1165 | 906 # figure the columns we're fetching |
| 907 cl = self.classes[classname] | |
| 908 cols, mls = self.determine_columns(cl.properties.items()) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
909 scols = ','.join([col for col,dt in cols]) |
| 1165 | 910 |
| 911 # perform the basic property fetch | |
| 912 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
913 self.sql(sql, (nodeid,)) |
| 1165 | 914 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
915 values = self.sql_fetchone() |
| 1165 | 916 if values is None: |
| 917 raise IndexError, 'no such %s node %s'%(classname, nodeid) | |
| 918 | |
| 919 # make up the node | |
| 920 node = {} | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
921 props = cl.getprops(protected=1) |
| 1165 | 922 for col in range(len(cols)): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
923 name = cols[col][0][1:] |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
924 if name.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
925 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
926 # ignore the special Interval-as-seconds column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
927 continue |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
928 value = values[col] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
929 if value is not None: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
930 value = self.sql_to_hyperdb_value[props[name].__class__](value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
931 node[name] = value |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
932 |
| 1165 | 933 |
| 934 # now the multilinks | |
| 935 for col in mls: | |
| 936 # get the link ids | |
| 937 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col, | |
| 938 self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
939 self.cursor.execute(sql, (nodeid,)) |
| 1165 | 940 # extract the first column from the result |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
941 # XXX numeric ids |
|
2887
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
942 items = [int(x[0]) for x in self.cursor.fetchall()] |
|
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
943 items.sort () |
|
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
944 node[col] = [str(x) for x in items] |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
945 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
946 # save off in the cache |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
947 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
948 self.cache[key] = node |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
949 # update the LRU |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
950 self.cache_lru.insert(0, key) |
|
1185
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
951 if len(self.cache_lru) > ROW_CACHE_SIZE: |
|
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
952 del self.cache[self.cache_lru.pop()] |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
953 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
954 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
955 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
956 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
957 return node |
| 1165 | 958 |
| 959 def destroynode(self, classname, nodeid): | |
| 960 '''Remove a node from the database. Called exclusively by the | |
| 961 destroy() method on Class. | |
| 962 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
963 logging.getLogger('hyperdb').info('destroynode %s%s'%(classname, nodeid)) |
| 1165 | 964 |
| 965 # make sure the node exists | |
| 966 if not self.hasnode(classname, nodeid): | |
| 967 raise IndexError, '%s has no node %s'%(classname, nodeid) | |
| 968 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
969 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
970 if self.cache.has_key((classname, nodeid)): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
971 del self.cache[(classname, nodeid)] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
972 |
| 1165 | 973 # see if there's any obvious commit actions that we should get rid of |
| 974 for entry in self.transactions[:]: | |
| 975 if entry[1][:2] == (classname, nodeid): | |
| 976 self.transactions.remove(entry) | |
| 977 | |
| 978 # now do the SQL | |
| 979 sql = 'delete from _%s where id=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
980 self.sql(sql, (nodeid,)) |
| 1165 | 981 |
| 982 # remove from multilnks | |
| 983 cl = self.getclass(classname) | |
| 984 x, mls = self.determine_columns(cl.properties.items()) | |
| 985 for col in mls: | |
| 986 # get the link ids | |
| 987 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg) | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
988 self.sql(sql, (nodeid,)) |
| 1165 | 989 |
| 990 # remove journal entries | |
| 991 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
992 self.sql(sql, (nodeid,)) |
| 1165 | 993 |
| 994 def hasnode(self, classname, nodeid): | |
| 995 ''' Determine if the database has a given node. | |
| 996 ''' | |
| 997 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
998 self.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
999 return int(self.cursor.fetchone()[0]) |
| 1165 | 1000 |
| 1001 def countnodes(self, classname): | |
| 1002 ''' Count the number of nodes that exist for a particular Class. | |
| 1003 ''' | |
| 1004 sql = 'select count(*) from _%s'%classname | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1005 self.sql(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1006 return self.cursor.fetchone()[0] |
| 1165 | 1007 |
| 1008 def addjournal(self, classname, nodeid, action, params, creator=None, | |
| 1009 creation=None): | |
| 1010 ''' Journal the Action | |
| 1011 'action' may be: | |
| 1012 | |
| 1013 'create' or 'set' -- 'params' is a dictionary of property values | |
| 1014 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) | |
| 1015 'retire' -- 'params' is None | |
| 1016 ''' | |
| 1017 # handle supply of the special journalling parameters (usually | |
| 1018 # supplied on importing an existing database) | |
| 1019 if creator: | |
| 1020 journaltag = creator | |
| 1021 else: | |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1022 journaltag = self.getuid() |
| 1165 | 1023 if creation: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1024 journaldate = creation |
| 1165 | 1025 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1026 journaldate = date.Date() |
| 1165 | 1027 |
| 1028 # create the journal entry | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1029 cols = 'nodeid,date,tag,action,params' |
| 1165 | 1030 |
| 1031 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1032 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(classname, |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1033 nodeid, journaldate, journaltag, action, params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1034 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1035 # make the journalled data marshallable |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1036 if isinstance(params, type({})): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1037 self._journal_marshal(params, classname) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1038 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1039 params = repr(params) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1040 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1041 dc = self.hyperdb_to_sql_value[hyperdb.Date] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1042 journaldate = dc(journaldate) |
| 1165 | 1043 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1044 self.save_journal(classname, cols, nodeid, journaldate, |
| 1165 | 1045 journaltag, action, params) |
| 1046 | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1047 def setjournal(self, classname, nodeid, journal): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1048 '''Set the journal to the "journal" list.''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1049 # clear out any existing entries |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1050 self.sql('delete from %s__journal where nodeid=%s'%(classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1051 self.arg), (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1052 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1053 # create the journal entry |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1054 cols = 'nodeid,date,tag,action,params' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1055 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1056 dc = self.hyperdb_to_sql_value[hyperdb.Date] |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1057 for nodeid, journaldate, journaltag, action, params in journal: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1058 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1059 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%( |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1060 classname, nodeid, journaldate, journaltag, action, |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1061 params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1062 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1063 # make the journalled data marshallable |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1064 if isinstance(params, type({})): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1065 self._journal_marshal(params, classname) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1066 params = repr(params) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1067 |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1068 self.save_journal(classname, cols, nodeid, dc(journaldate), |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1069 journaltag, action, params) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1070 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1071 def _journal_marshal(self, params, classname): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1072 '''Convert the journal params values into safely repr'able and |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1073 eval'able values.''' |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1074 properties = self.getclass(classname).getprops() |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1075 for param, value in params.items(): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1076 if not value: |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1077 continue |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1078 property = properties[param] |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1079 cvt = self.hyperdb_to_sql_value[property.__class__] |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1080 if isinstance(property, Password): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1081 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1082 elif isinstance(property, Date): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1083 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1084 elif isinstance(property, Interval): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1085 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1086 elif isinstance(property, Boolean): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1087 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1088 |
| 1165 | 1089 def getjournal(self, classname, nodeid): |
| 1090 ''' get the journal for id | |
| 1091 ''' | |
| 1092 # make sure the node exists | |
| 1093 if not self.hasnode(classname, nodeid): | |
| 1094 raise IndexError, '%s has no node %s'%(classname, nodeid) | |
| 1095 | |
| 1096 cols = ','.join('nodeid date tag action params'.split()) | |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1097 journal = self.load_journal(classname, cols, nodeid) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1098 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1099 # now unmarshal the data |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1100 dc = self.sql_to_hyperdb_value[hyperdb.Date] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1101 res = [] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1102 properties = self.getclass(classname).getprops() |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1103 for nodeid, date_stamp, user, action, params in journal: |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1104 params = eval(params) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1105 if isinstance(params, type({})): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1106 for param, value in params.items(): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1107 if not value: |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1108 continue |
|
2731
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1109 property = properties.get(param, None) |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1110 if property is None: |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1111 # deleted property |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1112 continue |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1113 cvt = self.sql_to_hyperdb_value[property.__class__] |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1114 if isinstance(property, Password): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1115 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1116 elif isinstance(property, Date): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1117 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1118 elif isinstance(property, Interval): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1119 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1120 elif isinstance(property, Boolean): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1121 params[param] = cvt(value) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1122 # XXX numeric ids |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1123 res.append((str(nodeid), dc(date_stamp), user, action, params)) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1124 return res |
| 1165 | 1125 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1126 def save_journal(self, classname, cols, nodeid, journaldate, |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1127 journaltag, action, params): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1128 ''' Save the journal entry to the database |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1129 ''' |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1130 entry = (nodeid, journaldate, journaltag, action, params) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1131 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1132 # do the insert |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1133 a = self.arg |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1134 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1135 classname, cols, a, a, a, a, a) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1136 self.sql(sql, entry) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1137 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1138 def load_journal(self, classname, cols, nodeid): |
| 1165 | 1139 ''' Load the journal from the database |
| 1140 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1141 # now get the journal entries |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1142 sql = 'select %s from %s__journal where nodeid=%s order by date'%( |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1143 cols, classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1144 self.sql(sql, (nodeid,)) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1145 return self.cursor.fetchall() |
| 1165 | 1146 |
| 1147 def pack(self, pack_before): | |
| 1148 ''' Delete all journal entries except "create" before 'pack_before'. | |
| 1149 ''' | |
| 2416 | 1150 date_stamp = self.hyperdb_to_sql_value[Date](pack_before) |
| 1165 | 1151 |
| 1152 # do the delete | |
| 1153 for classname in self.classes.keys(): | |
| 1154 sql = "delete from %s__journal where date<%s and "\ | |
| 1155 "action<>'create'"%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1156 self.sql(sql, (date_stamp,)) |
| 1165 | 1157 |
| 1158 def sql_commit(self): | |
| 1159 ''' Actually commit to the database. | |
| 1160 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1161 logging.getLogger('hyperdb').info('commit') |
| 1165 | 1162 self.conn.commit() |
| 1163 | |
|
2374
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1164 # open a new cursor for subsequent work |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1165 self.cursor = self.conn.cursor() |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1166 |
| 1165 | 1167 def commit(self): |
| 1168 ''' Commit the current transactions. | |
| 1169 | |
| 1170 Save all data changed since the database was opened or since the | |
| 1171 last commit() or rollback(). | |
| 1172 ''' | |
| 1173 # commit the database | |
| 1174 self.sql_commit() | |
| 1175 | |
| 1176 # now, do all the other transaction stuff | |
| 1177 for method, args in self.transactions: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1178 method(*args) |
| 1165 | 1179 |
| 1180 # clear out the transactions | |
| 1181 self.transactions = [] | |
| 1182 | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1183 def sql_rollback(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1184 self.conn.rollback() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1185 |
| 1165 | 1186 def rollback(self): |
| 1187 ''' Reverse all actions from the current transaction. | |
| 1188 | |
| 1189 Undo all the changes made since the database was opened or the last | |
| 1190 commit() or rollback() was performed. | |
| 1191 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1192 logging.getLogger('hyperdb').info('rollback') |
| 1165 | 1193 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1194 self.sql_rollback() |
| 1165 | 1195 |
| 1196 # roll back "other" transaction stuff | |
| 1197 for method, args in self.transactions: | |
| 1198 # delete temporary files | |
| 1199 if method == self.doStoreFile: | |
| 1200 self.rollbackStoreFile(*args) | |
| 1201 self.transactions = [] | |
| 1202 | |
|
1492
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1203 # clear the cache |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1204 self.clearCache() |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1205 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1206 def sql_close(self): |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1207 logging.getLogger('hyperdb').info('close') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1208 self.conn.close() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1209 |
| 1165 | 1210 def close(self): |
| 1211 ''' Close off the connection. | |
| 1212 ''' | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
1213 self.indexer.close() |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1214 self.sql_close() |
| 1165 | 1215 |
| 1216 # | |
| 1217 # The base Class class | |
| 1218 # | |
| 1219 class Class(hyperdb.Class): | |
| 1220 ''' The handle to a particular class of nodes in a hyperdatabase. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1221 |
| 1165 | 1222 All methods except __repr__ and getnode must be implemented by a |
| 1223 concrete backend Class. | |
| 1224 ''' | |
| 1225 | |
| 1226 def __init__(self, db, classname, **properties): | |
| 1227 '''Create a new class with a given name and property specification. | |
| 1228 | |
| 1229 'classname' must not collide with the name of an existing class, | |
| 1230 or a ValueError is raised. The keyword arguments in 'properties' | |
| 1231 must map names to property objects, or a TypeError is raised. | |
| 1232 ''' | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1233 for name in 'creation activity creator actor'.split(): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1234 if properties.has_key(name): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1235 raise ValueError, '"creation", "activity", "creator" and '\ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1236 '"actor" are reserved' |
| 1165 | 1237 |
| 1238 self.classname = classname | |
| 1239 self.properties = properties | |
| 1240 self.db = weakref.proxy(db) # use a weak ref to avoid circularity | |
| 1241 self.key = '' | |
| 1242 | |
| 1243 # should we journal changes (default yes) | |
| 1244 self.do_journal = 1 | |
| 1245 | |
| 1246 # do the db-related init stuff | |
| 1247 db.addclass(self) | |
| 1248 | |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1249 self.auditors = {'create': [], 'set': [], 'retire': [], 'restore': []} |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1250 self.reactors = {'create': [], 'set': [], 'retire': [], 'restore': []} |
| 1165 | 1251 |
| 1252 def schema(self): | |
| 1253 ''' A dumpable version of the schema that we can store in the | |
| 1254 database | |
| 1255 ''' | |
| 1256 return (self.key, [(x, repr(y)) for x,y in self.properties.items()]) | |
| 1257 | |
| 1258 def enableJournalling(self): | |
| 1259 '''Turn journalling on for this class | |
| 1260 ''' | |
| 1261 self.do_journal = 1 | |
| 1262 | |
| 1263 def disableJournalling(self): | |
| 1264 '''Turn journalling off for this class | |
| 1265 ''' | |
| 1266 self.do_journal = 0 | |
| 1267 | |
| 1268 # Editing nodes: | |
| 1269 def create(self, **propvalues): | |
| 1270 ''' Create a new node of this class and return its id. | |
| 1271 | |
| 1272 The keyword arguments in 'propvalues' map property names to values. | |
| 1273 | |
| 1274 The values of arguments must be acceptable for the types of their | |
| 1275 corresponding properties or a TypeError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1276 |
| 1165 | 1277 If this class has a key property, it must be present and its value |
| 1278 must not collide with other key strings or a ValueError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1279 |
| 1165 | 1280 Any other properties on this class that are missing from the |
| 1281 'propvalues' dictionary are set to None. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1282 |
| 1165 | 1283 If an id in a link or multilink property does not refer to a valid |
| 1284 node, an IndexError is raised. | |
| 1285 ''' | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1286 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1287 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1288 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1289 return newid |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1290 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1291 def create_inner(self, **propvalues): |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1292 ''' Called by create, in-between the audit and react calls. |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1293 ''' |
| 1165 | 1294 if propvalues.has_key('id'): |
| 1295 raise KeyError, '"id" is reserved' | |
| 1296 | |
| 1297 if self.db.journaltag is None: | |
| 1298 raise DatabaseError, 'Database open read-only' | |
| 1299 | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1300 if propvalues.has_key('creator') or propvalues.has_key('actor') or \ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1301 propvalues.has_key('creation') or propvalues.has_key('activity'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1302 raise KeyError, '"creator", "actor", "creation" and '\ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1303 '"activity" are reserved' |
| 1165 | 1304 |
| 1305 # new node's id | |
| 1306 newid = self.db.newid(self.classname) | |
| 1307 | |
| 1308 # validate propvalues | |
| 1309 num_re = re.compile('^\d+$') | |
| 1310 for key, value in propvalues.items(): | |
| 1311 if key == self.key: | |
| 1312 try: | |
| 1313 self.lookup(value) | |
| 1314 except KeyError: | |
| 1315 pass | |
| 1316 else: | |
| 1317 raise ValueError, 'node with key "%s" exists'%value | |
| 1318 | |
| 1319 # try to handle this property | |
| 1320 try: | |
| 1321 prop = self.properties[key] | |
| 1322 except KeyError: | |
| 1323 raise KeyError, '"%s" has no property "%s"'%(self.classname, | |
| 1324 key) | |
| 1325 | |
| 1326 if value is not None and isinstance(prop, Link): | |
| 1327 if type(value) != type(''): | |
| 1328 raise ValueError, 'link value must be String' | |
| 1329 link_class = self.properties[key].classname | |
| 1330 # if it isn't a number, it's a key | |
| 1331 if not num_re.match(value): | |
| 1332 try: | |
| 1333 value = self.db.classes[link_class].lookup(value) | |
| 1334 except (TypeError, KeyError): | |
| 1335 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1336 key, value, link_class) | |
| 1337 elif not self.db.getclass(link_class).hasnode(value): | |
| 1338 raise IndexError, '%s has no node %s'%(link_class, value) | |
| 1339 | |
| 1340 # save off the value | |
| 1341 propvalues[key] = value | |
| 1342 | |
| 1343 # register the link with the newly linked node | |
| 1344 if self.do_journal and self.properties[key].do_journal: | |
| 1345 self.db.addjournal(link_class, value, 'link', | |
| 1346 (self.classname, newid, key)) | |
| 1347 | |
| 1348 elif isinstance(prop, Multilink): | |
| 1349 if type(value) != type([]): | |
| 1350 raise TypeError, 'new property "%s" not a list of ids'%key | |
| 1351 | |
| 1352 # clean up and validate the list of links | |
| 1353 link_class = self.properties[key].classname | |
| 1354 l = [] | |
| 1355 for entry in value: | |
| 1356 if type(entry) != type(''): | |
| 1357 raise ValueError, '"%s" multilink value (%r) '\ | |
| 1358 'must contain Strings'%(key, value) | |
| 1359 # if it isn't a number, it's a key | |
| 1360 if not num_re.match(entry): | |
| 1361 try: | |
| 1362 entry = self.db.classes[link_class].lookup(entry) | |
| 1363 except (TypeError, KeyError): | |
| 1364 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1365 key, entry, self.properties[key].classname) | |
| 1366 l.append(entry) | |
| 1367 value = l | |
| 1368 propvalues[key] = value | |
| 1369 | |
| 1370 # handle additions | |
| 1371 for nodeid in value: | |
| 1372 if not self.db.getclass(link_class).hasnode(nodeid): | |
| 1373 raise IndexError, '%s has no node %s'%(link_class, | |
| 1374 nodeid) | |
| 1375 # register the link with the newly linked node | |
| 1376 if self.do_journal and self.properties[key].do_journal: | |
| 1377 self.db.addjournal(link_class, nodeid, 'link', | |
| 1378 (self.classname, newid, key)) | |
| 1379 | |
| 1380 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1381 if type(value) != type('') and type(value) != type(u''): |
| 1165 | 1382 raise TypeError, 'new property "%s" not a string'%key |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1383 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1384 self.db.indexer.add_text((self.classname, newid, key), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1385 value) |
| 1165 | 1386 |
| 1387 elif isinstance(prop, Password): | |
| 1388 if not isinstance(value, password.Password): | |
| 1389 raise TypeError, 'new property "%s" not a Password'%key | |
| 1390 | |
| 1391 elif isinstance(prop, Date): | |
| 1392 if value is not None and not isinstance(value, date.Date): | |
| 1393 raise TypeError, 'new property "%s" not a Date'%key | |
| 1394 | |
| 1395 elif isinstance(prop, Interval): | |
| 1396 if value is not None and not isinstance(value, date.Interval): | |
| 1397 raise TypeError, 'new property "%s" not an Interval'%key | |
| 1398 | |
| 1399 elif value is not None and isinstance(prop, Number): | |
| 1400 try: | |
| 1401 float(value) | |
| 1402 except ValueError: | |
| 1403 raise TypeError, 'new property "%s" not numeric'%key | |
| 1404 | |
| 1405 elif value is not None and isinstance(prop, Boolean): | |
| 1406 try: | |
| 1407 int(value) | |
| 1408 except ValueError: | |
| 1409 raise TypeError, 'new property "%s" not boolean'%key | |
| 1410 | |
| 1411 # make sure there's data where there needs to be | |
| 1412 for key, prop in self.properties.items(): | |
| 1413 if propvalues.has_key(key): | |
| 1414 continue | |
| 1415 if key == self.key: | |
| 1416 raise ValueError, 'key property "%s" is required'%key | |
| 1417 if isinstance(prop, Multilink): | |
| 1418 propvalues[key] = [] | |
| 1419 else: | |
| 1420 propvalues[key] = None | |
| 1421 | |
| 1422 # done | |
| 1423 self.db.addnode(self.classname, newid, propvalues) | |
| 1424 if self.do_journal: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1425 self.db.addjournal(self.classname, newid, ''"create", {}) |
| 1165 | 1426 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1427 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1428 return str(newid) |
| 1165 | 1429 |
| 1430 def get(self, nodeid, propname, default=_marker, cache=1): | |
| 1431 '''Get the value of a property on an existing node of this class. | |
| 1432 | |
| 1433 'nodeid' must be the id of an existing node of this class or an | |
| 1434 IndexError is raised. 'propname' must be the name of a property | |
| 1435 of this class or a KeyError is raised. | |
| 1436 | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1437 'cache' exists for backwards compatibility, and is not used. |
| 1165 | 1438 ''' |
| 1439 if propname == 'id': | |
| 1440 return nodeid | |
| 1441 | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1442 # get the node's dict |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1443 d = self.db.getnode(self.classname, nodeid) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1444 |
| 1165 | 1445 if propname == 'creation': |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1446 if d.has_key('creation'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1447 return d['creation'] |
| 1165 | 1448 else: |
| 1449 return date.Date() | |
| 1450 if propname == 'activity': | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1451 if d.has_key('activity'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1452 return d['activity'] |
| 1165 | 1453 else: |
| 1454 return date.Date() | |
| 1455 if propname == 'creator': | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1456 if d.has_key('creator'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1457 return d['creator'] |
| 1165 | 1458 else: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1459 return self.db.getuid() |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1460 if propname == 'actor': |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1461 if d.has_key('actor'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1462 return d['actor'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1463 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1464 return self.db.getuid() |
| 1165 | 1465 |
| 1466 # get the property (raises KeyErorr if invalid) | |
| 1467 prop = self.properties[propname] | |
| 1468 | |
|
2690
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1469 # XXX may it be that propname is valid property name |
|
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1470 # (above error is not raised) and not d.has_key(propname)??? |
|
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1471 if (not d.has_key(propname)) or (d[propname] is None): |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
1472 if default is _marker: |
| 1165 | 1473 if isinstance(prop, Multilink): |
| 1474 return [] | |
| 1475 else: | |
| 1476 return None | |
| 1477 else: | |
| 1478 return default | |
| 1479 | |
| 1480 # don't pass our list to other code | |
| 1481 if isinstance(prop, Multilink): | |
| 1482 return d[propname][:] | |
| 1483 | |
| 1484 return d[propname] | |
| 1485 | |
| 1486 def set(self, nodeid, **propvalues): | |
| 1487 '''Modify a property on an existing node of this class. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1488 |
| 1165 | 1489 'nodeid' must be the id of an existing node of this class or an |
| 1490 IndexError is raised. | |
| 1491 | |
| 1492 Each key in 'propvalues' must be the name of a property of this | |
| 1493 class or a KeyError is raised. | |
| 1494 | |
| 1495 All values in 'propvalues' must be acceptable types for their | |
| 1496 corresponding properties or a TypeError is raised. | |
| 1497 | |
| 1498 If the value of the key property is set, it must not collide with | |
| 1499 other key strings or a ValueError is raised. | |
| 1500 | |
| 1501 If the value of a Link or Multilink property contains an invalid | |
| 1502 node id, a ValueError is raised. | |
| 1503 ''' | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1504 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1505 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1506 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1507 self.fireReactors('set', nodeid, oldvalues) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1508 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1509 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1510 def set_inner(self, nodeid, **propvalues): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1511 ''' Called by set, in-between the audit and react calls. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1512 ''' |
| 1165 | 1513 if not propvalues: |
| 1514 return propvalues | |
| 1515 | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1516 if propvalues.has_key('creation') or propvalues.has_key('creator') or \ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1517 propvalues.has_key('actor') or propvalues.has_key('activity'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1518 raise KeyError, '"creation", "creator", "actor" and '\ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1519 '"activity" are reserved' |
| 1165 | 1520 |
| 1521 if propvalues.has_key('id'): | |
| 1522 raise KeyError, '"id" is reserved' | |
| 1523 | |
| 1524 if self.db.journaltag is None: | |
| 1525 raise DatabaseError, 'Database open read-only' | |
| 1526 | |
| 1527 node = self.db.getnode(self.classname, nodeid) | |
| 1528 if self.is_retired(nodeid): | |
| 1529 raise IndexError, 'Requested item is retired' | |
| 1530 num_re = re.compile('^\d+$') | |
| 1531 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1532 # make a copy of the values dictionary - we'll modify the contents |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1533 propvalues = propvalues.copy() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1534 |
| 1165 | 1535 # if the journal value is to be different, store it in here |
| 1536 journalvalues = {} | |
| 1537 | |
| 1538 # remember the add/remove stuff for multilinks, making it easier | |
| 1539 # for the Database layer to do its stuff | |
| 1540 multilink_changes = {} | |
| 1541 | |
| 1542 for propname, value in propvalues.items(): | |
| 1543 # check to make sure we're not duplicating an existing key | |
| 1544 if propname == self.key and node[propname] != value: | |
| 1545 try: | |
| 1546 self.lookup(value) | |
| 1547 except KeyError: | |
| 1548 pass | |
| 1549 else: | |
| 1550 raise ValueError, 'node with key "%s" exists'%value | |
| 1551 | |
| 1552 # this will raise the KeyError if the property isn't valid | |
| 1553 # ... we don't use getprops() here because we only care about | |
| 1554 # the writeable properties. | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1555 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1556 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1557 except KeyError: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1558 raise KeyError, '"%s" has no property named "%s"'%( |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1559 self.classname, propname) |
| 1165 | 1560 |
| 1561 # if the value's the same as the existing value, no sense in | |
| 1562 # doing anything | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1563 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1564 if value == current: |
| 1165 | 1565 del propvalues[propname] |
| 1566 continue | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1567 journalvalues[propname] = current |
| 1165 | 1568 |
| 1569 # do stuff based on the prop type | |
| 1570 if isinstance(prop, Link): | |
| 1571 link_class = prop.classname | |
| 1572 # if it isn't a number, it's a key | |
| 1573 if value is not None and not isinstance(value, type('')): | |
| 1574 raise ValueError, 'property "%s" link value be a string'%( | |
| 1575 propname) | |
| 1576 if isinstance(value, type('')) and not num_re.match(value): | |
| 1577 try: | |
| 1578 value = self.db.classes[link_class].lookup(value) | |
| 1579 except (TypeError, KeyError): | |
| 1580 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1581 propname, value, prop.classname) | |
| 1582 | |
| 1583 if (value is not None and | |
| 1584 not self.db.getclass(link_class).hasnode(value)): | |
| 1585 raise IndexError, '%s has no node %s'%(link_class, value) | |
| 1586 | |
| 1587 if self.do_journal and prop.do_journal: | |
| 1588 # register the unlink with the old linked node | |
| 1589 if node[propname] is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1590 self.db.addjournal(link_class, node[propname], |
|
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1591 ''"unlink", (self.classname, nodeid, propname)) |
| 1165 | 1592 |
| 1593 # register the link with the newly linked node | |
| 1594 if value is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1595 self.db.addjournal(link_class, value, ''"link", |
| 1165 | 1596 (self.classname, nodeid, propname)) |
| 1597 | |
| 1598 elif isinstance(prop, Multilink): | |
| 1599 if type(value) != type([]): | |
| 1600 raise TypeError, 'new property "%s" not a list of'\ | |
| 1601 ' ids'%propname | |
| 1602 link_class = self.properties[propname].classname | |
| 1603 l = [] | |
| 1604 for entry in value: | |
| 1605 # if it isn't a number, it's a key | |
| 1606 if type(entry) != type(''): | |
| 1607 raise ValueError, 'new property "%s" link value ' \ | |
| 1608 'must be a string'%propname | |
| 1609 if not num_re.match(entry): | |
| 1610 try: | |
| 1611 entry = self.db.classes[link_class].lookup(entry) | |
| 1612 except (TypeError, KeyError): | |
| 1613 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1614 propname, entry, | |
| 1615 self.properties[propname].classname) | |
| 1616 l.append(entry) | |
| 1617 value = l | |
| 1618 propvalues[propname] = value | |
| 1619 | |
| 1620 # figure the journal entry for this property | |
| 1621 add = [] | |
| 1622 remove = [] | |
| 1623 | |
| 1624 # handle removals | |
| 1625 if node.has_key(propname): | |
| 1626 l = node[propname] | |
| 1627 else: | |
| 1628 l = [] | |
| 1629 for id in l[:]: | |
| 1630 if id in value: | |
| 1631 continue | |
| 1632 # register the unlink with the old linked node | |
| 1633 if self.do_journal and self.properties[propname].do_journal: | |
| 1634 self.db.addjournal(link_class, id, 'unlink', | |
| 1635 (self.classname, nodeid, propname)) | |
| 1636 l.remove(id) | |
| 1637 remove.append(id) | |
| 1638 | |
| 1639 # handle additions | |
| 1640 for id in value: | |
| 1641 if not self.db.getclass(link_class).hasnode(id): | |
| 1642 raise IndexError, '%s has no node %s'%(link_class, id) | |
| 1643 if id in l: | |
| 1644 continue | |
| 1645 # register the link with the newly linked node | |
| 1646 if self.do_journal and self.properties[propname].do_journal: | |
| 1647 self.db.addjournal(link_class, id, 'link', | |
| 1648 (self.classname, nodeid, propname)) | |
| 1649 l.append(id) | |
| 1650 add.append(id) | |
| 1651 | |
| 1652 # figure the journal entry | |
| 1653 l = [] | |
| 1654 if add: | |
| 1655 l.append(('+', add)) | |
| 1656 if remove: | |
| 1657 l.append(('-', remove)) | |
| 1658 multilink_changes[propname] = (add, remove) | |
| 1659 if l: | |
| 1660 journalvalues[propname] = tuple(l) | |
| 1661 | |
| 1662 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1663 if value is not None and type(value) != type('') and type(value) != type(u''): |
| 1165 | 1664 raise TypeError, 'new property "%s" not a string'%propname |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1665 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1666 self.db.indexer.add_text((self.classname, nodeid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1667 value) |
| 1165 | 1668 |
| 1669 elif isinstance(prop, Password): | |
| 1670 if not isinstance(value, password.Password): | |
| 1671 raise TypeError, 'new property "%s" not a Password'%propname | |
| 1672 propvalues[propname] = value | |
| 1673 | |
| 1674 elif value is not None and isinstance(prop, Date): | |
| 1675 if not isinstance(value, date.Date): | |
| 1676 raise TypeError, 'new property "%s" not a Date'% propname | |
| 1677 propvalues[propname] = value | |
| 1678 | |
| 1679 elif value is not None and isinstance(prop, Interval): | |
| 1680 if not isinstance(value, date.Interval): | |
| 1681 raise TypeError, 'new property "%s" not an '\ | |
| 1682 'Interval'%propname | |
| 1683 propvalues[propname] = value | |
| 1684 | |
| 1685 elif value is not None and isinstance(prop, Number): | |
| 1686 try: | |
| 1687 float(value) | |
| 1688 except ValueError: | |
| 1689 raise TypeError, 'new property "%s" not numeric'%propname | |
| 1690 | |
| 1691 elif value is not None and isinstance(prop, Boolean): | |
| 1692 try: | |
| 1693 int(value) | |
| 1694 except ValueError: | |
| 1695 raise TypeError, 'new property "%s" not boolean'%propname | |
| 1696 | |
| 1697 # nothing to do? | |
| 1698 if not propvalues: | |
| 1699 return propvalues | |
| 1700 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1701 # update the activity time |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1702 propvalues['activity'] = date.Date() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1703 propvalues['actor'] = self.db.getuid() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1704 |
| 1165 | 1705 # do the set, and journal it |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1706 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes) |
| 1165 | 1707 |
| 1708 if self.do_journal: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1709 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues) |
| 1165 | 1710 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1711 return propvalues |
| 1165 | 1712 |
| 1713 def retire(self, nodeid): | |
| 1714 '''Retire a node. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1715 |
| 1165 | 1716 The properties on the node remain available from the get() method, |
| 1717 and the node's id is never reused. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1718 |
| 1165 | 1719 Retired nodes are not returned by the find(), list(), or lookup() |
| 1720 methods, and other nodes may reuse the values of their key properties. | |
| 1721 ''' | |
| 1722 if self.db.journaltag is None: | |
| 1723 raise DatabaseError, 'Database open read-only' | |
| 1724 | |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1725 self.fireAuditors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1726 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1727 # use the arg for __retired__ to cope with any odd database type |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1728 # conversion (hello, sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1729 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1730 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1731 self.db.sql(sql, (1, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1732 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1733 self.db.addjournal(self.classname, nodeid, ''"retired", None) |
| 1165 | 1734 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1735 self.fireReactors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1736 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1737 def restore(self, nodeid): |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
1738 '''Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1739 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1740 Make node available for all operations like it was before retirement. |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1741 ''' |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1742 if self.db.journaltag is None: |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1743 raise DatabaseError, 'Database open read-only' |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1744 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1745 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1746 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1747 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1748 try: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1749 id = self.lookup(node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1750 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1751 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1752 else: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1753 raise KeyError, "Key property (%s) of retired node clashes with \ |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1754 existing one (%s)" % (key, node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1755 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1756 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1757 # use the arg for __retired__ to cope with any odd database type |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1758 # conversion (hello, sqlite) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1759 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1760 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1761 self.db.sql(sql, (0, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1762 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1763 self.db.addjournal(self.classname, nodeid, ''"restored", None) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1764 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1765 self.fireReactors('restore', nodeid, None) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1766 |
| 1165 | 1767 def is_retired(self, nodeid): |
| 1768 '''Return true if the node is rerired | |
| 1769 ''' | |
| 1770 sql = 'select __retired__ from _%s where id=%s'%(self.classname, | |
| 1771 self.db.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1772 self.db.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1773 return int(self.db.sql_fetchone()[0]) |
| 1165 | 1774 |
| 1775 def destroy(self, nodeid): | |
| 1776 '''Destroy a node. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1777 |
| 1165 | 1778 WARNING: this method should never be used except in extremely rare |
| 1779 situations where there could never be links to the node being | |
| 1780 deleted | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1781 |
| 1165 | 1782 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1783 |
| 1165 | 1784 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1785 |
| 1165 | 1786 WARNING: really, use retire() instead |
| 1787 | |
| 1788 Well, I think that's enough warnings. This method exists mostly to | |
| 1789 support the session storage of the cgi interface. | |
| 1790 | |
| 1791 The node is completely removed from the hyperdb, including all journal | |
| 1792 entries. It will no longer be available, and will generally break code | |
| 1793 if there are any references to the node. | |
| 1794 ''' | |
| 1795 if self.db.journaltag is None: | |
| 1796 raise DatabaseError, 'Database open read-only' | |
| 1797 self.db.destroynode(self.classname, nodeid) | |
| 1798 | |
| 1799 def history(self, nodeid): | |
| 1800 '''Retrieve the journal of edits on a particular node. | |
| 1801 | |
| 1802 'nodeid' must be the id of an existing node of this class or an | |
| 1803 IndexError is raised. | |
| 1804 | |
| 1805 The returned list contains tuples of the form | |
| 1806 | |
|
1409
8dc60d87ab42
Fixed a backlog of bug reports, and worked on python 2.3 compatibility:
Richard Jones <richard@users.sourceforge.net>
parents:
1383
diff
changeset
|
1807 (nodeid, date, tag, action, params) |
| 1165 | 1808 |
| 1809 'date' is a Timestamp object specifying the time of the change and | |
| 1810 'tag' is the journaltag specified when the database was opened. | |
| 1811 ''' | |
| 1812 if not self.do_journal: | |
| 1813 raise ValueError, 'Journalling is disabled for this class' | |
| 1814 return self.db.getjournal(self.classname, nodeid) | |
| 1815 | |
| 1816 # Locating nodes: | |
| 1817 def hasnode(self, nodeid): | |
| 1818 '''Determine if the given nodeid actually exists | |
| 1819 ''' | |
| 1820 return self.db.hasnode(self.classname, nodeid) | |
| 1821 | |
| 1822 def setkey(self, propname): | |
| 1823 '''Select a String property of this class to be the key property. | |
| 1824 | |
| 1825 'propname' must be the name of a String property of this class or | |
| 1826 None, or a TypeError is raised. The values of the key property on | |
| 1827 all existing nodes must be unique or a ValueError is raised. | |
| 1828 ''' | |
| 1829 prop = self.getprops()[propname] | |
| 1830 if not isinstance(prop, String): | |
| 1831 raise TypeError, 'key properties must be String' | |
| 1832 self.key = propname | |
| 1833 | |
| 1834 def getkey(self): | |
| 1835 '''Return the name of the key property for this class or None.''' | |
| 1836 return self.key | |
| 1837 | |
| 1838 def labelprop(self, default_to_id=0): | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1839 '''Return the property name for a label for the given node. |
| 1165 | 1840 |
| 1841 This method attempts to generate a consistent label for the node. | |
| 1842 It tries the following in order: | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1843 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1844 1. key property |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1845 2. "name" property |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1846 3. "title" property |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1847 4. first property from the sorted property name list |
| 1165 | 1848 ''' |
| 1849 k = self.getkey() | |
| 1850 if k: | |
| 1851 return k | |
| 1852 props = self.getprops() | |
| 1853 if props.has_key('name'): | |
| 1854 return 'name' | |
| 1855 elif props.has_key('title'): | |
| 1856 return 'title' | |
| 1857 if default_to_id: | |
| 1858 return 'id' | |
| 1859 props = props.keys() | |
| 1860 props.sort() | |
| 1861 return props[0] | |
| 1862 | |
| 1863 def lookup(self, keyvalue): | |
| 1864 '''Locate a particular node by its key property and return its id. | |
| 1865 | |
| 1866 If this class has no key property, a TypeError is raised. If the | |
| 1867 'keyvalue' matches one of the values for the key property among | |
| 1868 the nodes in this class, the matching node's id is returned; | |
| 1869 otherwise a KeyError is raised. | |
| 1870 ''' | |
| 1871 if not self.key: | |
| 1872 raise TypeError, 'No key property set for class %s'%self.classname | |
| 1873 | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1874 # use the arg to handle any odd database type conversion (hello, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1875 # sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1876 sql = "select id from _%s where _%s=%s and __retired__ <> %s"%( |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1877 self.classname, self.key, self.db.arg, self.db.arg) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1878 self.db.sql(sql, (keyvalue, 1)) |
| 1165 | 1879 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1880 # see if there was a result that's not retired |
|
1203
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1881 row = self.db.sql_fetchone() |
|
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1882 if not row: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1883 raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key, |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1884 keyvalue, self.classname) |
| 1165 | 1885 |
| 1886 # return the id | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1887 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1888 return str(row[0]) |
| 1165 | 1889 |
| 1890 def find(self, **propspec): | |
| 1891 '''Get the ids of nodes in this class which link to the given nodes. | |
| 1892 | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1893 'propspec' consists of keyword args propname=nodeid or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1894 propname={nodeid:1, } |
| 1165 | 1895 'propname' must be the name of a property in this class, or a |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1896 KeyError is raised. That property must be a Link or |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1897 Multilink property, or a TypeError is raised. |
| 1165 | 1898 |
| 1899 Any node in this class whose 'propname' property links to any of the | |
| 1900 nodeids will be returned. Used by the full text indexing, which knows | |
| 1901 that "foo" occurs in msg1, msg3 and file7, so we have hits on these | |
| 1902 issues: | |
| 1903 | |
| 1904 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) | |
| 1905 ''' | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1906 # shortcut |
| 1165 | 1907 if not propspec: |
| 1908 return [] | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1909 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1910 # validate the args |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1911 props = self.getprops() |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1912 propspec = propspec.items() |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1913 for propname, nodeids in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1914 # check the prop is OK |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1915 prop = props[propname] |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1916 if not isinstance(prop, Link) and not isinstance(prop, Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1917 raise TypeError, "'%s' not a Link/Multilink property"%propname |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1918 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1919 # first, links |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1920 a = self.db.arg |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1921 allvalues = () |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1922 sql = [] |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1923 where = [] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1924 for prop, values in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1925 if not isinstance(props[prop], hyperdb.Link): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1926 continue |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1927 if type(values) is type({}) and len(values) == 1: |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1928 values = values.keys()[0] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1929 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1930 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1931 where.append('_%s = %s'%(prop, a)) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
1932 elif values is None: |
|
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
1933 where.append('_%s is NULL'%prop) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1934 else: |
|
2494
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1935 values = values.keys() |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1936 s = '' |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1937 if None in values: |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1938 values.remove(None) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1939 s = '_%s is NULL or '%prop |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1940 allvalues += tuple(values) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1941 s += '_%s in (%s)'%(prop, ','.join([a]*len(values))) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
1942 where.append('(' + s +')') |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1943 if where: |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1944 allvalues = (1, ) + allvalues |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1945 sql.append('''select id from _%s where __retired__ <> %s |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1946 and %s'''%(self.classname, a, ' and '.join(where))) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1947 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1948 # now multilinks |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1949 for prop, values in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1950 if not isinstance(props[prop], hyperdb.Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1951 continue |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1952 if not values: |
|
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1953 continue |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1954 allvalues += (1, ) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1955 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1956 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1957 s = a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1958 else: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1959 allvalues += tuple(values.keys()) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1960 s = ','.join([a]*len(values)) |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1961 tn = '%s_%s'%(self.classname, prop) |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1962 sql.append('''select id from _%s, %s where __retired__ <> %s |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1963 and id = %s.nodeid and %s.linkid in (%s)'''%(self.classname, |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1964 tn, a, tn, tn, s)) |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1965 |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1966 if not sql: |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1967 return [] |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1968 sql = ' union '.join(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1969 self.db.sql(sql, allvalues) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1970 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1971 l = [str(x[0]) for x in self.db.sql_fetchall()] |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1972 return l |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1973 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1974 def stringFind(self, **requirements): |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1975 '''Locate a particular node by matching a set of its String |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1976 properties in a caseless search. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1977 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1978 If the property is not a String property, a TypeError is raised. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1979 |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1980 The return is a list of the id of all nodes that match. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1981 ''' |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1982 where = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1983 args = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1984 for propname in requirements.keys(): |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1985 prop = self.properties[propname] |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
1986 if not isinstance(prop, String): |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1987 raise TypeError, "'%s' not a String property"%propname |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1988 where.append(propname) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1989 args.append(requirements[propname].lower()) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1990 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1991 # generate the where clause |
|
1549
a53a7e197360
fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents:
1539
diff
changeset
|
1992 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where]) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1993 sql = 'select id from _%s where %s and __retired__<>%s'%( |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1994 self.classname, s, self.db.arg) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1995 args.append(1) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1996 self.db.sql(sql, tuple(args)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1997 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1998 l = [str(x[0]) for x in self.db.sql_fetchall()] |
| 1165 | 1999 return l |
| 2000 | |
| 2001 def list(self): | |
| 2002 ''' Return a list of the ids of the active nodes in this class. | |
| 2003 ''' | |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2004 return self.getnodeids(retired=0) |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2005 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2006 def getnodeids(self, retired=None): |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2007 ''' Retrieve all the ids of the nodes for a particular Class. |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2008 |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2009 Set retired=None to get all nodes. Otherwise it'll get all the |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2010 retired or non-retired nodes, depending on the flag. |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2011 ''' |
|
1707
3d627e34f18e
sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1599
diff
changeset
|
2012 # flip the sense of the 'retired' flag if we don't want all of them |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2013 if retired is not None: |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2014 if retired: |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2015 args = (0, ) |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2016 else: |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2017 args = (1, ) |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2018 sql = 'select id from _%s where __retired__ <> %s'%(self.classname, |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2019 self.db.arg) |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2020 else: |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2021 args = () |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2022 sql = 'select id from _%s'%self.classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2023 self.db.sql(sql, args) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2024 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2025 ids = [str(x[0]) for x in self.db.cursor.fetchall()] |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2026 return ids |
| 1165 | 2027 |
|
1249
6c24a86a12ae
Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents:
1244
diff
changeset
|
2028 def filter(self, search_matches, filterspec, sort=(None,None), |
|
6c24a86a12ae
Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents:
1244
diff
changeset
|
2029 group=(None,None)): |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2030 '''Return a list of the ids of the active nodes in this class that |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2031 match the 'filter' spec, sorted by the group spec and then the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2032 sort spec |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2033 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2034 "filterspec" is {propname: value(s)} |
| 1165 | 2035 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2036 "sort" and "group" are (dir, prop) where dir is '+', '-' or None |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2037 and prop is a prop name or None |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2038 |
|
2362
10fc45eea226
fix SearchAction use of Class.filter(), and clarify API docs for same
Richard Jones <richard@users.sourceforge.net>
parents:
2319
diff
changeset
|
2039 "search_matches" is {nodeid: marker} or None |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2040 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2041 The filter must match all properties specificed - but if the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2042 property value to match is a list, any one of the values in the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2043 list may match for that property to match. |
| 1165 | 2044 ''' |
|
2578
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2045 # we can't match anything if search_matches is empty |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2046 if search_matches == {}: |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2047 return [] |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2048 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2049 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2050 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2051 |
| 1165 | 2052 cn = self.classname |
| 2053 | |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2054 timezone = self.db.getUserTimezone() |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2055 |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2056 # vars to hold the components of the SQL statement |
|
2586
3c67f4bfa225
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
2057 frum = [] # FROM clauses |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2058 loj = [] # LEFT OUTER JOIN clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2059 where = [] # WHERE clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2060 args = [] # *any* positional arguments |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2061 a = self.db.arg |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2062 |
| 1165 | 2063 # figure the WHERE clause from the filterspec |
| 2064 props = self.getprops() | |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2065 mlfilt = 0 # are we joining with Multilink tables? |
| 1165 | 2066 for k, v in filterspec.items(): |
| 2067 propclass = props[k] | |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2068 # now do other where clause stuff |
| 1165 | 2069 if isinstance(propclass, Multilink): |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2070 mlfilt = 1 |
| 1165 | 2071 tn = '%s_%s'%(cn, k) |
|
1556
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2072 if v in ('-1', ['-1']): |
|
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2073 # only match rows that have count(linkid)=0 in the |
|
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2074 # corresponding multilink table) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2075 where.append('_%s.id not in (select nodeid from %s)'%(cn, |
|
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2076 tn)) |
|
1556
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2077 elif isinstance(v, type([])): |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1549
diff
changeset
|
2078 frum.append(tn) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
2079 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2080 where.append('_%s.id=%s.nodeid and %s.linkid in (%s)'%(cn, |
|
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2081 tn, tn, s)) |
| 1165 | 2082 args = args + v |
| 2083 else: | |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1549
diff
changeset
|
2084 frum.append(tn) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2085 where.append('_%s.id=%s.nodeid and %s.linkid=%s'%(cn, tn, |
|
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2086 tn, a)) |
| 1165 | 2087 args.append(v) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2088 elif k == 'id': |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2089 if isinstance(v, type([])): |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2090 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2091 where.append('_%s.%s in (%s)'%(cn, k, s)) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2092 args = args + v |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2093 else: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2094 where.append('_%s.%s=%s'%(cn, k, a)) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2095 args.append(v) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2096 elif isinstance(propclass, String): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2097 if not isinstance(v, type([])): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2098 v = [v] |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2099 |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2100 # Quote the bits in the string that need it and then embed |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2101 # in a "substring" search. Note - need to quote the '%' so |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2102 # they make it through the python layer happily |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2103 v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v] |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2104 |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2105 # now add to the where clause |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2106 where.append('(' |
|
3147
dbf80d7db63f
consistent text searching behaviour (AND everywhere) [SF#1101036]
Richard Jones <richard@users.sourceforge.net>
parents:
3099
diff
changeset
|
2107 +' and '.join(["_%s._%s LIKE '%s'"%(cn, k, s) for s in v]) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2108 +')') |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2109 # note: args are embedded in the query string now |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2110 elif isinstance(propclass, Link): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2111 if isinstance(v, type([])): |
|
2508
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2112 d = {} |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2113 for entry in v: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2114 if entry == '-1': |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2115 entry = None |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2116 d[entry] = entry |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2117 l = [] |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2118 if d.has_key(None) or not d: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2119 del d[None] |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2120 l.append('_%s._%s is NULL'%(cn, k)) |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2121 if d: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2122 v = d.keys() |
|
1295
9c3459cb8ab6
work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
2123 s = ','.join([a for x in v]) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2124 l.append('(_%s._%s in (%s))'%(cn, k, s)) |
|
1295
9c3459cb8ab6
work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
2125 args = args + v |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2126 if l: |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2127 where.append('(' + ' or '.join(l) +')') |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2128 else: |
|
2508
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2129 if v in ('-1', None): |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2130 v = None |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2131 where.append('_%s._%s is NULL'%(cn, k)) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2132 else: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2133 where.append('_%s._%s=%s'%(cn, k, a)) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2134 args.append(v) |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2135 elif isinstance(propclass, Date): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2136 dc = self.db.hyperdb_to_sql_value[hyperdb.Date] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2137 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2138 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2139 where.append('_%s._%s in (%s)'%(cn, k, s)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2140 args = args + [dc(date.Date(v)) for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2141 else: |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2142 try: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2143 # Try to filter on range of dates |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2144 date_rng = Range(v, date.Date, offset=timezone) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2145 if date_rng.from_value: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2146 where.append('_%s._%s >= %s'%(cn, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2147 args.append(dc(date_rng.from_value)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2148 if date_rng.to_value: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2149 where.append('_%s._%s <= %s'%(cn, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2150 args.append(dc(date_rng.to_value)) |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2151 except ValueError: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2152 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2153 pass |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2154 elif isinstance(propclass, Interval): |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2155 # filter using the __<prop>_int__ column |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2156 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2157 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2158 where.append('_%s.__%s_int__ in (%s)'%(cn, k, s)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2159 args = args + [date.Interval(x).as_seconds() for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2160 else: |
|
1596
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2161 try: |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2162 # Try to filter on range of intervals |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2163 date_rng = Range(v, date.Interval) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2164 if date_rng.from_value: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2165 where.append('_%s.__%s_int__ >= %s'%(cn, k, a)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2166 args.append(date_rng.from_value.as_seconds()) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2167 if date_rng.to_value: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2168 where.append('_%s.__%s_int__ <= %s'%(cn, k, a)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2169 args.append(date_rng.to_value.as_seconds()) |
|
1596
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2170 except ValueError: |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2171 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2172 pass |
| 1165 | 2173 else: |
| 2174 if isinstance(v, type([])): | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2175 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2176 where.append('_%s._%s in (%s)'%(cn, k, s)) |
| 1165 | 2177 args = args + v |
| 2178 else: | |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2179 where.append('_%s._%s=%s'%(cn, k, a)) |
| 1165 | 2180 args.append(v) |
| 2181 | |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2182 # don't match retired nodes |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2183 where.append('_%s.__retired__ <> 1'%cn) |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2184 |
| 1165 | 2185 # add results of full text search |
| 2186 if search_matches is not None: | |
| 2187 v = search_matches.keys() | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2188 s = ','.join([a for x in v]) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2189 where.append('_%s.id in (%s)'%(cn, s)) |
| 1165 | 2190 args = args + v |
| 2191 | |
|
2720
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2192 # sanity check: sorting *and* grouping on the same property? |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2193 if group[1] == sort[1]: |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2194 sort = (None, None) |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2195 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2196 # "grouping" is just the first-order sorting in the SQL fetch |
| 1165 | 2197 orderby = [] |
| 2198 ordercols = [] | |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2199 mlsort = [] |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2200 for sortby in group, sort: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2201 sdir, prop = sortby |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2202 if sdir and prop: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2203 if isinstance(props[prop], Multilink): |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2204 mlsort.append(sortby) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2205 continue |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2206 elif isinstance(props[prop], Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2207 # use the int column for sorting |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2208 o = '__'+prop+'_int__' |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2209 ordercols.append(o) |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2210 elif isinstance(props[prop], Link): |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2211 # determine whether the linked Class has an order property |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2212 lcn = props[prop].classname |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2213 link = self.db.classes[lcn] |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2214 o = '_%s._%s'%(cn, prop) |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2215 if link.getprops().has_key('order'): |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2216 tn = '_' + lcn |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2217 loj.append('LEFT OUTER JOIN %s on %s=%s.id'%(tn, |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2218 o, tn)) |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2219 ordercols.append(tn + '._order') |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2220 o = tn + '._order' |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2221 else: |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2222 ordercols.append(o) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2223 elif prop == 'id': |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2224 o = '_%s.id'%cn |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2225 else: |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2226 o = '_%s._%s'%(cn, prop) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2227 ordercols.append(o) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2228 if sdir == '-': |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2229 o += ' desc' |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2230 orderby.append(o) |
| 1165 | 2231 |
| 2232 # construct the SQL | |
|
2536
70ede89e8056
handle postgresql bug in SQL generation [SF#984591]
Richard Jones <richard@users.sourceforge.net>
parents:
2530
diff
changeset
|
2233 frum.append('_'+cn) |
| 1165 | 2234 frum = ','.join(frum) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2235 if where: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2236 where = ' where ' + (' and '.join(where)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2237 else: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2238 where = '' |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2239 if mlfilt: |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2240 # we're joining tables on the id, so we will get dupes if we |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2241 # don't distinct() |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2242 cols = ['distinct(_%s.id)'%cn] |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2243 else: |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2244 cols = ['_%s.id'%cn] |
| 1165 | 2245 if orderby: |
| 2246 cols = cols + ordercols | |
| 2247 order = ' order by %s'%(','.join(orderby)) | |
| 2248 else: | |
| 2249 order = '' | |
| 2250 cols = ','.join(cols) | |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2251 loj = ' '.join(loj) |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2252 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order) |
| 1165 | 2253 args = tuple(args) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2254 __traceback_info__ = (sql, args) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2255 self.db.sql(sql, args) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
2256 l = self.db.sql_fetchall() |
| 1165 | 2257 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2258 # return the IDs (the first column) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2259 # XXX numeric ids |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2260 l = [str(row[0]) for row in l] |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2261 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2262 if not mlsort: |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2263 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2264 self.db.stats['filtering'] += (time.time() - start_t) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2265 return l |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2266 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2267 # ergh. someone wants to sort by a multilink. |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2268 r = [] |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2269 for id in l: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2270 m = [] |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2271 for ml in mlsort: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2272 m.append(self.get(id, ml[1])) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2273 r.append((id, m)) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2274 i = 0 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2275 for sortby in mlsort: |
|
2234
70d21059aa18
fix nested scope bug in multilink sorting in rdbms filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2228
diff
changeset
|
2276 def sortfun(a, b, dir=sortby[i], i=i): |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2277 if dir == '-': |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2278 return cmp(b[1][i], a[1][i]) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2279 else: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2280 return cmp(a[1][i], b[1][i]) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2281 r.sort(sortfun) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2282 i += 1 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2283 r = [i[0] for i in r] |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2284 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2285 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2286 self.db.stats['filtering'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2287 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2288 return r |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2289 |
| 1165 | 2290 def count(self): |
| 2291 '''Get the number of nodes in this class. | |
| 2292 | |
| 2293 If the returned integer is 'numnodes', the ids of all the nodes | |
| 2294 in this class run from 1 to numnodes, and numnodes+1 will be the | |
| 2295 id of the next node to be created in this class. | |
| 2296 ''' | |
| 2297 return self.db.countnodes(self.classname) | |
| 2298 | |
| 2299 # Manipulating properties: | |
| 2300 def getprops(self, protected=1): | |
| 2301 '''Return a dictionary mapping property names to property objects. | |
| 2302 If the "protected" flag is true, we include protected properties - | |
| 2303 those which may not be modified. | |
| 2304 ''' | |
| 2305 d = self.properties.copy() | |
| 2306 if protected: | |
| 2307 d['id'] = String() | |
| 2308 d['creation'] = hyperdb.Date() | |
| 2309 d['activity'] = hyperdb.Date() | |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1175
diff
changeset
|
2310 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2311 d['actor'] = hyperdb.Link('user') |
| 1165 | 2312 return d |
| 2313 | |
| 2314 def addprop(self, **properties): | |
| 2315 '''Add properties to this class. | |
| 2316 | |
| 2317 The keyword arguments in 'properties' must map names to property | |
| 2318 objects, or a TypeError is raised. None of the keys in 'properties' | |
| 2319 may collide with the names of existing properties, or a ValueError | |
| 2320 is raised before any properties have been added. | |
| 2321 ''' | |
| 2322 for key in properties.keys(): | |
| 2323 if self.properties.has_key(key): | |
| 2324 raise ValueError, key | |
| 2325 self.properties.update(properties) | |
| 2326 | |
| 2327 def index(self, nodeid): | |
| 2328 '''Add (or refresh) the node to search indexes | |
| 2329 ''' | |
| 2330 # find all the String properties that have indexme | |
| 2331 for prop, propclass in self.getprops().items(): | |
| 2332 if isinstance(propclass, String) and propclass.indexme: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2333 self.db.indexer.add_text((self.classname, nodeid, prop), |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2334 str(self.get(nodeid, prop))) |
| 1165 | 2335 |
| 2336 | |
| 2337 # | |
| 2338 # Detector interface | |
| 2339 # | |
| 2340 def audit(self, event, detector): | |
| 2341 '''Register a detector | |
| 2342 ''' | |
| 2343 l = self.auditors[event] | |
| 2344 if detector not in l: | |
| 2345 self.auditors[event].append(detector) | |
| 2346 | |
| 2347 def fireAuditors(self, action, nodeid, newvalues): | |
| 2348 '''Fire all registered auditors. | |
| 2349 ''' | |
| 2350 for audit in self.auditors[action]: | |
| 2351 audit(self.db, self, nodeid, newvalues) | |
| 2352 | |
| 2353 def react(self, event, detector): | |
| 2354 '''Register a detector | |
| 2355 ''' | |
| 2356 l = self.reactors[event] | |
| 2357 if detector not in l: | |
| 2358 self.reactors[event].append(detector) | |
| 2359 | |
| 2360 def fireReactors(self, action, nodeid, oldvalues): | |
| 2361 '''Fire all registered reactors. | |
| 2362 ''' | |
| 2363 for react in self.reactors[action]: | |
| 2364 react(self.db, self, nodeid, oldvalues) | |
| 2365 | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2366 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2367 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2368 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2369 def export_list(self, propnames, nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2370 ''' Export a node - generate a list of CSV-able data in the order |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2371 specified by propnames for the given node. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2372 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2373 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2374 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2375 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2376 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2377 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2378 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2379 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2380 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2381 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2382 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2383 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2384 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2385 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2386 value = str(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2387 l.append(repr(value)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2388 l.append(repr(self.is_retired(nodeid))) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2389 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2390 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2391 def import_list(self, propnames, proplist): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2392 ''' Import a node - all information including "id" is present and |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2393 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2394 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2395 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2396 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2397 Return the nodeid of the node imported. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2398 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2399 if self.db.journaltag is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2400 raise DatabaseError, 'Database open read-only' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2401 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2402 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2403 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2404 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2405 retire = 0 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2406 if not "id" in propnames: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2407 newid = self.db.newid(self.classname) |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2408 else: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2409 newid = eval(proplist[propnames.index("id")]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2410 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2411 # Use eval to reverse the repr() used to output the CSV |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2412 value = eval(proplist[i]) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2413 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2414 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2415 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2416 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2417 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2418 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2419 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2420 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2421 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2422 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2423 retire = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2424 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2425 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2426 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2427 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2428 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2429 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2430 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2431 # don't set Nones |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2432 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2433 elif isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2434 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2435 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2436 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2437 elif isinstance(prop, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2438 pwd = password.Password() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2439 pwd.unpack(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2440 value = pwd |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2441 d[propname] = value |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2442 if isinstance(prop, String): |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2443 if type(value) != type('') and type(value) != type(u''): |
|
2691
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2444 raise TypeError, \ |
|
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2445 'new property "%(propname)s" not a string: %(value)r' \ |
|
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2446 % locals() |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2447 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2448 self.db.indexer.add_text((self.classname, newid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2449 value) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2450 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2451 # get a new id if necessary |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2452 if newid is None: |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2453 newid = self.db.newid(self.classname) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2454 |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2455 # insert new node or update existing? |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2456 if not self.hasnode(newid): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2457 self.db.addnode(self.classname, newid, d) # insert |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2458 else: |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2459 self.db.setnode(self.classname, newid, d) # update |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2460 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2461 # retire? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2462 if retire: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2463 # use the arg for __retired__ to cope with any odd database type |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2464 # conversion (hello, sqlite) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2465 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2466 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2467 self.db.sql(sql, (1, newid)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2468 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2469 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2470 def export_journals(self): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2471 '''Export a class's journal - generate a list of lists of |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2472 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2473 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2474 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2475 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2476 No heading here - the columns are fixed. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2477 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2478 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2479 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2480 for nodeid in self.getnodeids(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2481 for nodeid, date, user, action, params in self.history(nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2482 date = date.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2483 if action == 'set': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2484 for propname, value in params.items(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2485 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2486 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2487 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2488 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2489 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2490 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2491 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2492 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2493 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2494 value = str(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2495 params[propname] = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2496 l = [nodeid, date, user, action, params] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2497 r.append(map(repr, l)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2498 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2499 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2500 def import_journals(self, entries): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2501 '''Import a class's journal. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2502 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2503 Uses setjournal() to set the journal for each item.''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2504 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2505 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2506 for l in entries: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2507 l = map(eval, l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2508 nodeid, jdate, user, action, params = l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2509 r = d.setdefault(nodeid, []) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2510 if action == 'set': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2511 for propname, value in params.items(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2512 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2513 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2514 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2515 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2516 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2517 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2518 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2519 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2520 pwd = password.Password() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2521 pwd.unpack(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2522 value = pwd |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2523 params[propname] = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2524 r.append((nodeid, date.Date(jdate), user, action, params)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2525 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2526 for nodeid, l in d.items(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2527 self.db.setjournal(self.classname, nodeid, l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2528 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2586
diff
changeset
|
2529 class FileClass(hyperdb.FileClass, Class): |
| 1165 | 2530 '''This class defines a large chunk of data. To support this, it has a |
| 2531 mandatory String property "content" which is typically saved off | |
| 2532 externally to the hyperdb. | |
| 2533 | |
| 2534 The default MIME type of this data is defined by the | |
| 2535 "default_mime_type" class attribute, which may be overridden by each | |
| 2536 node if the class defines a "type" String property. | |
| 2537 ''' | |
| 2538 default_mime_type = 'text/plain' | |
| 2539 | |
| 2540 def create(self, **propvalues): | |
| 2541 ''' snaffle the file propvalue and store in a file | |
| 2542 ''' | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2543 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2544 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2545 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2546 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2547 # now remove the content property so it's not stored in the db |
| 1165 | 2548 content = propvalues['content'] |
| 2549 del propvalues['content'] | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2550 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2551 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2552 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2553 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2554 # figure the mime type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2555 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2556 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2557 # and index! |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2558 self.db.indexer.add_text((self.classname, newid, 'content'), content, |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2559 mime_type) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2560 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2561 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2562 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2563 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2564 # store off the content as a file |
| 1165 | 2565 self.db.storefile(self.classname, newid, None, content) |
| 2566 return newid | |
| 2567 | |
| 2568 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2569 ''' Trap the content propname and get it from the file |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2570 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2571 'cache' exists for backwards compatibility, and is not used. |
| 1165 | 2572 ''' |
| 2573 poss_msg = 'Possibly a access right configuration problem.' | |
| 2574 if propname == 'content': | |
| 2575 try: | |
| 2576 return self.db.getfile(self.classname, nodeid, None) | |
| 2577 except IOError, (strerror): | |
| 2578 # BUG: by catching this we donot see an error in the log. | |
| 2579 return 'ERROR reading file: %s%s\n%s\n%s'%( | |
| 2580 self.classname, nodeid, poss_msg, strerror) | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
2581 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2582 return Class.get(self, nodeid, propname, default) |
| 1165 | 2583 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2584 return Class.get(self, nodeid, propname) |
| 1165 | 2585 |
| 2586 def getprops(self, protected=1): | |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2587 '''In addition to the actual properties on the node, these methods |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2588 provide the "content" property. If the "protected" flag is true, |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2589 we include protected properties - those which may not be |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2590 modified. |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2591 |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2592 Note that the content prop is indexed separately, hence no indexme. |
| 1165 | 2593 ''' |
| 2594 d = Class.getprops(self, protected=protected).copy() | |
| 2595 d['content'] = hyperdb.String() | |
| 2596 return d | |
| 2597 | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2598 def set(self, itemid, **propvalues): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2599 ''' Snarf the "content" propvalue and update it in a file |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2600 ''' |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2601 self.fireAuditors('set', itemid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2602 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
| 1165 | 2603 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2604 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2605 content = None |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2606 if propvalues.has_key('content'): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2607 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2608 del propvalues['content'] |
| 1165 | 2609 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2610 # do the database create |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2611 propvalues = self.set_inner(itemid, **propvalues) |
| 1165 | 2612 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2613 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2614 if content: |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2615 # store and index |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2616 self.db.storefile(self.classname, itemid, None, content) |
|
2966
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2617 mime_type = None |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
2618 if self.getprops().has_key('type'): |
|
2966
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2619 mime_type = propvalues.get('type', self.get(itemid, 'type')) |
|
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2620 if not mime_type: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2621 mime_type = self.default_mime_type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2622 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2623 content, mime_type) |
| 1165 | 2624 |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
2625 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
2626 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2627 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2628 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2629 return propvalues |
| 1165 | 2630 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2631 def index(self, nodeid): |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2632 '''Add (or refresh) the node to search indexes. |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2633 |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2634 Pass on the content-type property for the content property. |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2635 ''' |
|
2512
f5542d92307a
fix anydbm journal import [SF#983166]
Richard Jones <richard@users.sourceforge.net>
parents:
2508
diff
changeset
|
2636 Class.index(self, nodeid) |
|
2966
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2637 mime_type = None |
|
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2638 if self.getprops().has_key('type'): |
|
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2639 mime_type = self.get(nodeid, 'type') |
|
2ddba486546a
various fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2892
diff
changeset
|
2640 if not mime_type: |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2641 mime_type = self.default_mime_type |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2642 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2643 str(self.get(nodeid, 'content')), mime_type) |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2644 |
| 1165 | 2645 # XXX deviation from spec - was called ItemClass |
| 2646 class IssueClass(Class, roundupdb.IssueClass): | |
| 2647 # Overridden methods: | |
| 2648 def __init__(self, db, classname, **properties): | |
| 2649 '''The newly-created class automatically includes the "messages", | |
| 2650 "files", "nosy", and "superseder" properties. If the 'properties' | |
| 2651 dictionary attempts to specify any of these properties or a | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2652 "creation", "creator", "activity" or "actor" property, a ValueError |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2653 is raised. |
| 1165 | 2654 ''' |
| 2655 if not properties.has_key('title'): | |
| 2656 properties['title'] = hyperdb.String(indexme='yes') | |
| 2657 if not properties.has_key('messages'): | |
| 2658 properties['messages'] = hyperdb.Multilink("msg") | |
| 2659 if not properties.has_key('files'): | |
| 2660 properties['files'] = hyperdb.Multilink("file") | |
| 2661 if not properties.has_key('nosy'): | |
| 2662 # note: journalling is turned off as it really just wastes | |
| 2663 # space. this behaviour may be overridden in an instance | |
| 2664 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
| 2665 if not properties.has_key('superseder'): | |
| 2666 properties['superseder'] = hyperdb.Multilink(classname) | |
| 2667 Class.__init__(self, db, classname, **properties) | |
| 2668 | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
2669 # vim: set et sts=4 sw=4 : |