Mercurial > p > roundup > code
annotate roundup/backends/rdbms_common.py @ 3634:57c66056ffe4
Implemented what I'll call for now "transitive searching"...
...using the filter method. The first idea was mentioned on the
roundup-users mailing list:
http://article.gmane.org/gmane.comp.bug-tracking.roundup.user/6909
We can now search for items which link transitively to other classes
using filter. An example is searching for all items where a certain user
has added a message in the last week:
db.issue.filter (None, {'messages.author' : '42', 'messages.date' : '.-1w;'})
or more readable (but not exactly semantically equivalent, if you're
searching for multiple users in this way it will fail, because string
searches are ANDed): {'messages.author.username':'ralf', ...
We can even extend this further, look for all items that were changed by
users belonging to a certain department (having the same supervisor -- a
property that is not in the user class in standard roundup) in the last
week, the filterspec would be:
{'messages.author.supervisor' : '42', 'messages.date' : '.-1w;'}
If anybody wants to suggest another name instead of transitive
searching, you're welcome.
I've implemented a generic method for this in hyperdb.py -- the backend
now implements _filter in this case. With the generic method, anydbm and
metakit should work (anydbm is tested, metakit breaks for other
reasons). A backend may chose to implement the real transitive filter
itself. This was done for rdbms_common.py. It now has an implementation
of filter that supports transitive searching by creating one big join in
the generated SQL query.
I've added several new regression tests to test for the new features.
All the tests (not just the new ones) run through on python2.3 and
python2.4 with postgres, mysql, sqlite, anydbm -- but metakit was
already broken when I started.
I've generated a tag before commit called 'rsc_before_transitive_search'
and will create the 'after' tag after this commit, so you can merge out
my changes if you don't like them -- if you like them I can remove the
tags.
.-- Ralf
| author | Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net> |
|---|---|
| date | Sat, 08 Jul 2006 18:28:18 +0000 |
| parents | a292054b4393 |
| children | 53987aa153d2 |
| rev | line source |
|---|---|
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
1 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
6 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
11 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
17 # |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
18 #$Id: rdbms_common.py,v 1.172 2006-07-08 18:28:18 schlatterbeck Exp $ |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
19 ''' Relational database (SQL) backend common code. |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
20 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
21 Basics: |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
22 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
23 - map roundup classes to relational tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
24 - automatically detect schema changes and modify the table schemas |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
25 appropriately (we store the "database version" of the schema in the |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
26 database itself as the only row of the "schema" table) |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
27 - multilinks (which represent a many-to-many relationship) are handled through |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
28 intermediate tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
29 - journals are stored adjunct to the per-class tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
30 - table names and columns have "_" prepended so the names can't clash with |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
31 restricted names (like "order") |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
32 - retirement is determined by the __retired__ column being true |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
33 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
34 Database-specific changes may generally be pushed out to the overridable |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
35 sql_* methods, since everything else should be fairly generic. There's |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
36 probably a bit of work to be done if a database is used that actually |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
37 honors column typing, since the initial databases don't (sqlite stores |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
38 everything as a string.) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
39 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
40 The schema of the hyperdb being mapped to the database is stored in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
41 database itself as a repr()'ed dictionary of information about each Class |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
42 that maps to a table. If that information differs from the hyperdb schema, |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
43 then we update it. We also store in the schema dict a version which |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
44 allows us to upgrade the database schema when necessary. See upgrade_db(). |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
45 ''' |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
46 __docformat__ = 'restructuredtext' |
| 1165 | 47 |
| 48 # standard python modules | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
49 import sys, os, time, re, errno, weakref, copy, logging |
| 1165 | 50 |
| 51 # roundup modules | |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
52 from roundup import hyperdb, date, password, roundupdb, security, support |
| 1165 | 53 from roundup.hyperdb import String, Password, Date, Interval, Link, \ |
|
1417
472c21af7f69
fixed error in indexargs_url (thanks Patrick Ohly)
Richard Jones <richard@users.sourceforge.net>
parents:
1415
diff
changeset
|
54 Multilink, DatabaseError, Boolean, Number, Node |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
55 from roundup.backends import locking |
| 1165 | 56 |
| 57 # support | |
| 58 from blobfiles import FileStorage | |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
59 try: |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
60 from indexer_xapian import Indexer |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
61 except ImportError: |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
62 from indexer_rdbms import Indexer |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
63 from sessions_rdbms import Sessions, OneTimeKeys |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
64 from roundup.date import Range |
| 1165 | 65 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
66 # number of rows to keep in memory |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
67 ROW_CACHE_SIZE = 100 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
68 |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
69 # dummy value meaning "argument not passed" |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
70 _marker = [] |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
71 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
72 def _num_cvt(num): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
73 num = str(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
74 try: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
75 return int(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
76 except: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
77 return float(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
78 |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
79 def _bool_cvt(value): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
80 if value in ('TRUE', 'FALSE'): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
81 return {'TRUE': 1, 'FALSE': 0}[value] |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
82 # assume it's a number returned from the db API |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
83 return int(value) |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
84 |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
85 def connection_dict(config, dbnamestr=None): |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
86 ''' Used by Postgresql and MySQL to detemine the keyword args for |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
87 opening the database connection.''' |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
88 d = { } |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
89 if dbnamestr: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
90 d[dbnamestr] = config.RDBMS_NAME |
|
3099
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
91 for name in ('host', 'port', 'password', 'user', 'read_default_group', |
|
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
92 'read_default_file'): |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
93 cvar = 'RDBMS_'+name.upper() |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
94 if config[cvar] is not None: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
95 d[name] = config[cvar] |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
96 return d |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
97 |
| 1165 | 98 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
99 ''' Wrapper around an SQL database that presents a hyperdb interface. |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
100 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
101 - some functionality is specific to the actual SQL database, hence |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
102 the sql_* methods that are NotImplemented |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
103 - we keep a cache of the latest ROW_CACHE_SIZE row fetches. |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
104 ''' |
| 1165 | 105 def __init__(self, config, journaltag=None): |
| 106 ''' Open the database and load the schema from it. | |
| 107 ''' | |
| 108 self.config, self.journaltag = config, journaltag | |
| 109 self.dir = config.DATABASE | |
| 110 self.classes = {} | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
111 self.indexer = Indexer(self) |
| 1165 | 112 self.security = security.Security(self) |
| 113 | |
| 114 # additional transaction support for external files and the like | |
| 115 self.transactions = [] | |
| 116 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
117 # keep a cache of the N most recently retrieved rows of any kind |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
118 # (classname, nodeid) = row |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
119 self.cache = {} |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
120 self.cache_lru = [] |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
121 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
122 'filtering': 0} |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
123 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
124 # database lock |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
125 self.lockfile = None |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
126 |
| 1165 | 127 # open a connection to the database, creating the "conn" attribute |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
128 self.open_connection() |
| 1165 | 129 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
130 def clearCache(self): |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
131 self.cache = {} |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
132 self.cache_lru = [] |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
133 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
134 def getSessionManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
135 return Sessions(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
136 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
137 def getOTKManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
138 return OneTimeKeys(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
139 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
140 def open_connection(self): |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
141 ''' Open a connection to the database, creating it if necessary. |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
142 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
143 Must call self.load_dbschema() |
| 1165 | 144 ''' |
| 145 raise NotImplemented | |
| 146 | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
147 def sql(self, sql, args=None): |
| 1165 | 148 ''' Execute the sql with the optional args. |
| 149 ''' | |
| 150 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
151 logging.getLogger('hyperdb').debug('SQL %r %r'%(sql, args)) |
| 1165 | 152 if args: |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
153 self.cursor.execute(sql, args) |
| 1165 | 154 else: |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
155 self.cursor.execute(sql) |
| 1165 | 156 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
157 def sql_fetchone(self): |
| 1165 | 158 ''' Fetch a single row. If there's nothing to fetch, return None. |
| 159 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
160 return self.cursor.fetchone() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
161 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
162 def sql_fetchall(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
163 ''' Fetch all rows. If there's nothing to fetch, return []. |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
164 ''' |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
165 return self.cursor.fetchall() |
| 1165 | 166 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
167 def sql_stringquote(self, value): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
168 ''' Quote the string so it's safe to put in the 'sql quotes' |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
169 ''' |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
170 return re.sub("'", "''", str(value)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
171 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
172 def init_dbschema(self): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
173 self.database_schema = { |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
174 'version': self.current_db_version, |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
175 'tables': {} |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
176 } |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
177 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
178 def load_dbschema(self): |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
179 ''' Load the schema definition that the database currently implements |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
180 ''' |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
181 self.cursor.execute('select schema from schema') |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
182 schema = self.cursor.fetchone() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
183 if schema: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
184 self.database_schema = eval(schema[0]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
185 else: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
186 self.database_schema = {} |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
187 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
188 def save_dbschema(self): |
| 1165 | 189 ''' Save the schema definition that the database currently implements |
| 190 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
191 s = repr(self.database_schema) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
192 self.sql('delete from schema') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
193 self.sql('insert into schema values (%s)', (s,)) |
| 1165 | 194 |
| 195 def post_init(self): | |
| 196 ''' Called once the schema initialisation has finished. | |
| 197 | |
| 198 We should now confirm that the schema defined by our "classes" | |
| 199 attribute actually matches the schema in the database. | |
| 200 ''' | |
| 3553 | 201 save = 0 |
| 202 | |
| 203 # handle changes in the schema | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
204 tables = self.database_schema['tables'] |
| 1165 | 205 for classname, spec in self.classes.items(): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
206 if tables.has_key(classname): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
207 dbspec = tables[classname] |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
208 if self.update_class(spec, dbspec): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
209 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
210 save = 1 |
| 1165 | 211 else: |
| 212 self.create_class(spec) | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
213 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
214 save = 1 |
| 1165 | 215 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
216 for classname, spec in tables.items(): |
| 1165 | 217 if not self.classes.has_key(classname): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
218 self.drop_class(classname, tables[classname]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
219 del tables[classname] |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
220 save = 1 |
| 1165 | 221 |
| 3553 | 222 # now upgrade the database for column type changes, new internal |
| 223 # tables, etc. | |
| 224 save = save | self.upgrade_db() | |
| 225 | |
| 1165 | 226 # update the database version of the schema |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
227 if save: |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
228 self.save_dbschema() |
| 1165 | 229 |
| 230 # reindex the db if necessary | |
| 231 if self.indexer.should_reindex(): | |
| 232 self.reindex() | |
| 233 | |
| 234 # commit | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
235 self.sql_commit() |
| 1165 | 236 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
237 # update this number when we need to make changes to the SQL structure |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
238 # of the backen database |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
239 current_db_version = 4 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
240 def upgrade_db(self): |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
241 ''' Update the SQL database to reflect changes in the backend code. |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
242 |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
243 Return boolean whether we need to save the schema. |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
244 ''' |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
245 version = self.database_schema.get('version', 1) |
|
2081
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
246 if version == self.current_db_version: |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
247 # nothing to do |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
248 return 0 |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
249 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
250 if version < 2: |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2429
diff
changeset
|
251 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
252 logging.getLogger('hyperdb').info('upgrade to version 2') |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
253 # change the schema structure |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
254 self.database_schema = {'tables': self.database_schema} |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
255 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
256 # version 1 didn't have the actor column (note that in |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
257 # MySQL this will also transition the tables to typed columns) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
258 self.add_new_columns_v2() |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
259 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
260 # version 1 doesn't have the OTK, session and indexing in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
261 # database |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
262 self.create_version_2_tables() |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
263 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
264 if version < 3: |
|
2456
1cd69db95b23
fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents:
2429
diff
changeset
|
265 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
266 logging.getLogger('hyperdb').info('upgrade to version 3') |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
267 self.fix_version_2_tables() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
268 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
269 if version < 4: |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
270 self.fix_version_3_tables() |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
271 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
272 self.database_schema['version'] = self.current_db_version |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
273 return 1 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
274 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
275 def fix_version_3_tables(self): |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
276 # drop the shorter VARCHAR OTK column and add a new TEXT one |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
277 for name in ('otk', 'session'): |
|
2755
e3cd28cec23d
Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents:
2733
diff
changeset
|
278 self.sql('DELETE FROM %ss'%name) |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
279 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
280 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
281 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
282 def fix_version_2_tables(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
283 '''Default (used by sqlite): NOOP''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
284 pass |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
285 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
286 def _convert_journal_tables(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
287 '''Get current journal table contents, drop the table and re-create''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
288 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
289 cols = ','.join('nodeid date tag action params'.split()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
290 for klass in self.classes.values(): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
291 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
292 sql = 'select %s from %s__journal order by date'%(cols, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
293 klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
294 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
295 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
296 self.drop_journal_table_indexes(klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
297 c.execute('drop table %s__journal'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
298 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
299 # re-create and re-populate |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
300 self.create_journal_table(klass) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
301 a = self.arg |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
302 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
303 klass.classname, cols, a, a, a, a, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
304 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
305 # no data conversion needed |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
306 self.cursor.execute(sql, row) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
307 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
308 def _convert_string_properties(self): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
309 '''Get current Class tables that contain String properties, and |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
310 convert the VARCHAR columns to TEXT''' |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
311 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
312 for klass in self.classes.values(): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
313 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
314 cols, mls = self.determine_columns(klass.properties.items()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
315 scols = ','.join([i[0] for i in cols]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
316 sql = 'select id,%s from _%s'%(scols, klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
317 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
318 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
319 self.drop_class_table_indexes(klass.classname, klass.getkey()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
320 c.execute('drop table _%s'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
321 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
322 # re-create and re-populate |
|
2424
74474ec41050
argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents:
2416
diff
changeset
|
323 self.create_class_table(klass, create_sequence=0) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
324 a = ','.join([self.arg for i in range(len(cols)+1)]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
325 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
326 scols, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
327 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
328 l = [] |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
329 for entry in row: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
330 # mysql will already be a string - psql needs "help" |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
331 if entry is not None and not isinstance(entry, type('')): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
332 entry = str(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
333 l.append(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
334 self.cursor.execute(sql, l) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
335 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
336 def refresh_database(self): |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
337 self.post_init() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
338 |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
339 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
340 def reindex(self, classname=None, show_progress=False): |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
341 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
342 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
343 else: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
344 classes = self.classes.values() |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
345 for klass in classes: |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
346 if show_progress: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
347 for nodeid in support.Progress('Reindex %s'%klass.classname, |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
348 klass.list()): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
349 klass.index(nodeid) |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
350 else: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
351 for nodeid in klass.list(): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
352 klass.index(nodeid) |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
353 self.indexer.save_index() |
| 1165 | 354 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
355 hyperdb_to_sql_datatypes = { |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
356 hyperdb.String : 'TEXT', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
357 hyperdb.Date : 'TIMESTAMP', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
358 hyperdb.Link : 'INTEGER', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
359 hyperdb.Interval : 'VARCHAR(255)', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
360 hyperdb.Password : 'VARCHAR(255)', |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
361 hyperdb.Boolean : 'BOOLEAN', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
362 hyperdb.Number : 'REAL', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
363 } |
| 1165 | 364 def determine_columns(self, properties): |
| 365 ''' Figure the column names and multilink properties from the spec | |
| 366 | |
| 367 "properties" is a list of (name, prop) where prop may be an | |
| 368 instance of a hyperdb "type" _or_ a string repr of that type. | |
| 369 ''' | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
370 cols = [ |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
371 ('_actor', self.hyperdb_to_sql_datatypes[hyperdb.Link]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
372 ('_activity', self.hyperdb_to_sql_datatypes[hyperdb.Date]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
373 ('_creator', self.hyperdb_to_sql_datatypes[hyperdb.Link]), |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
374 ('_creation', self.hyperdb_to_sql_datatypes[hyperdb.Date]), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
375 ] |
| 1165 | 376 mls = [] |
| 377 # add the multilinks separately | |
| 378 for col, prop in properties: | |
| 379 if isinstance(prop, Multilink): | |
| 380 mls.append(col) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
381 continue |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
382 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
383 if isinstance(prop, type('')): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
384 raise ValueError, "string property spec!" |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
385 #and prop.find('Multilink') != -1: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
386 #mls.append(col) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
387 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
388 datatype = self.hyperdb_to_sql_datatypes[prop.__class__] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
389 cols.append(('_'+col, datatype)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
390 |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
391 # Intervals stored as two columns |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
392 if isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
393 cols.append(('__'+col+'_int__', 'BIGINT')) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
394 |
| 1165 | 395 cols.sort() |
| 396 return cols, mls | |
| 397 | |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
398 def update_class(self, spec, old_spec, force=0): |
| 1165 | 399 ''' Determine the differences between the current spec and the |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
400 database version of the spec, and update where necessary. |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
401 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
402 If 'force' is true, update the database anyway. |
| 1165 | 403 ''' |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
404 new_has = spec.properties.has_key |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
405 new_spec = spec.schema() |
|
1515
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
406 new_spec[1].sort() |
|
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
407 old_spec[1].sort() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
408 if not force and new_spec == old_spec: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
409 # no changes |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
410 return 0 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
411 |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
412 logger = logging.getLogger('hyperdb') |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
413 logger.info('update_class %s'%spec.classname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
414 |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
415 logger.debug('old_spec %r'%(old_spec,)) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
416 logger.debug('new_spec %r'%(new_spec,)) |
| 1165 | 417 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
418 # detect key prop change for potential index change |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
419 keyprop_changes = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
420 if new_spec[0] != old_spec[0]: |
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
421 if old_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
422 keyprop_changes['remove'] = old_spec[0] |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
423 if new_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
424 keyprop_changes['add'] = new_spec[0] |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
425 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
426 # detect multilinks that have been removed, and drop their table |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
427 old_has = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
428 for name, prop in old_spec[1]: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
429 old_has[name] = 1 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
430 if new_has(name): |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
431 continue |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
432 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
433 if prop.find('Multilink to') != -1: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
434 # first drop indexes. |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
435 self.drop_multilink_table_indexes(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
436 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
437 # now the multilink table itself |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
438 sql = 'drop table %s_%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
439 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
440 # if this is the key prop, drop the index first |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
441 if old_spec[0] == prop: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
442 self.drop_class_table_key_index(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
443 del keyprop_changes['remove'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
444 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
445 # drop the column |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
446 sql = 'alter table _%s drop column _%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
447 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
448 self.sql(sql) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
449 old_has = old_has.has_key |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
450 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
451 # if we didn't remove the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
452 # changed, we still need to remove the old index |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
453 if keyprop_changes.has_key('remove'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
454 self.drop_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
455 keyprop_changes['remove']) |
| 1165 | 456 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
457 # add new columns |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
458 for propname, prop in new_spec[1]: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
459 if old_has(propname): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
460 continue |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
461 prop = spec.properties[propname] |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
462 if isinstance(prop, Multilink): |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
463 self.create_multilink_table(spec, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
464 else: |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
465 # add the column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
466 coltype = self.hyperdb_to_sql_datatypes[prop.__class__] |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
467 sql = 'alter table _%s add column _%s %s'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
468 spec.classname, propname, coltype) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
469 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
470 |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
471 # extra Interval column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
472 if isinstance(prop, Interval): |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
473 sql = 'alter table _%s add column __%s_int__ BIGINT'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
474 spec.classname, propname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
475 self.sql(sql) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
476 |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
477 # if the new column is a key prop, we need an index! |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
478 if new_spec[0] == propname: |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
479 self.create_class_table_key_index(spec.classname, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
480 del keyprop_changes['add'] |
| 1165 | 481 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
482 # if we didn't add the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
483 # changed, we still need to add the new index |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
484 if keyprop_changes.has_key('add'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
485 self.create_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
486 keyprop_changes['add']) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
487 |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
488 return 1 |
| 1165 | 489 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
490 def determine_all_columns(self, spec): |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
491 """Figure out the columns from the spec and also add internal columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
492 |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
493 """ |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
494 cols, mls = self.determine_columns(spec.properties.items()) |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
495 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
496 # add on our special columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
497 cols.append(('id', 'INTEGER PRIMARY KEY')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
498 cols.append(('__retired__', 'INTEGER DEFAULT 0')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
499 return cols, mls |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
500 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
501 def create_class_table(self, spec): |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
502 '''Create the class table for the given Class "spec". Creates the |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
503 indexes too.''' |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
504 cols, mls = self.determine_all_columns(spec) |
| 1165 | 505 |
| 506 # create the base table | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
507 scols = ','.join(['%s %s'%x for x in cols]) |
| 1165 | 508 sql = 'create table _%s (%s)'%(spec.classname, scols) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
509 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
510 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
511 self.create_class_table_indexes(spec) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
512 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
513 return cols, mls |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
514 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
515 def create_class_table_indexes(self, spec): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
516 ''' create the class table for the given spec |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
517 ''' |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
518 # create __retired__ index |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
519 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%( |
|
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
520 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
521 self.sql(index_sql2) |
| 1165 | 522 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
523 # create index for key property |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
524 if spec.key: |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
525 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%( |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
526 spec.classname, spec.key, |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
527 spec.classname, spec.key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
528 self.sql(index_sql3) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
529 |
|
2228
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
530 # TODO: create indexes on (selected?) Link property columns, as |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
531 # they're more likely to be used for lookup |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
532 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
533 def drop_class_table_indexes(self, cn, key): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
534 # drop the old table indexes first |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
535 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
536 if key: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
537 l.append('_%s_%s_idx'%(cn, key)) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
538 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
539 table_name = '_%s'%cn |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
540 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
541 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
542 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
543 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
544 self.sql(index_sql) |
| 1165 | 545 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
546 def create_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
547 ''' create the class table for the given spec |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
548 ''' |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
549 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
550 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
551 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
552 def drop_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
553 table_name = '_%s'%cn |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
554 index_name = '_%s_%s_idx'%(cn, key) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
555 if not self.sql_index_exists(table_name, index_name): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
556 return |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
557 sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
558 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
559 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
560 def create_journal_table(self, spec): |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
561 ''' create the journal table for a class given the spec and |
| 1165 | 562 already-determined cols |
| 563 ''' | |
| 564 # journal table | |
| 565 cols = ','.join(['%s varchar'%x | |
| 566 for x in 'nodeid date tag action params'.split()]) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
567 sql = '''create table %s__journal ( |
|
3224
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
568 nodeid integer, date %s, tag varchar(255), |
|
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
569 action varchar(255), params text)''' % (spec.classname, |
|
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
570 self.hyperdb_to_sql_datatypes[hyperdb.Date]) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
571 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
572 self.create_journal_table_indexes(spec) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
573 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
574 def create_journal_table_indexes(self, spec): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
575 # index on nodeid |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
576 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%( |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
577 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
578 self.sql(sql) |
| 1165 | 579 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
580 def drop_journal_table_indexes(self, classname): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
581 index_name = '%s_journ_idx'%classname |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
582 if not self.sql_index_exists('%s__journal'%classname, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
583 return |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
584 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
585 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
586 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
587 def create_multilink_table(self, spec, ml): |
| 1165 | 588 ''' Create a multilink table for the "ml" property of the class |
| 589 given by the spec | |
| 590 ''' | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
591 # create the table |
|
2828
3a2f138fcb28
use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2755
diff
changeset
|
592 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%( |
| 1165 | 593 spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
594 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
595 self.create_multilink_table_indexes(spec, ml) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
596 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
597 def create_multilink_table_indexes(self, spec, ml): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
598 # create index on linkid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
599 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
600 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
601 self.sql(index_sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
602 |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
603 # create index on nodeid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
604 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
605 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
606 self.sql(index_sql) |
| 1165 | 607 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
608 def drop_multilink_table_indexes(self, classname, ml): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
609 l = [ |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
610 '%s_%s_l_idx'%(classname, ml), |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
611 '%s_%s_n_idx'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
612 ] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
613 table_name = '%s_%s'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
614 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
615 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
616 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
617 index_sql = 'drop index %s'%index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
618 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
619 |
| 1165 | 620 def create_class(self, spec): |
| 621 ''' Create a database table according to the given spec. | |
| 622 ''' | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
623 cols, mls = self.create_class_table(spec) |
|
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
624 self.create_journal_table(spec) |
| 1165 | 625 |
| 626 # now create the multilink tables | |
| 627 for ml in mls: | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
628 self.create_multilink_table(spec, ml) |
| 1165 | 629 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
630 def drop_class(self, cn, spec): |
| 1165 | 631 ''' Drop the given table from the database. |
| 632 | |
| 633 Drop the journal and multilink tables too. | |
| 634 ''' | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
635 properties = spec[1] |
| 1165 | 636 # figure the multilinks |
| 637 mls = [] | |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
638 for propanme, prop in properties: |
| 1165 | 639 if isinstance(prop, Multilink): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
640 mls.append(propname) |
| 1165 | 641 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
642 # drop class table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
643 self.drop_class_table_indexes(cn, spec[0]) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
644 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
645 self.drop_class_table(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
646 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
647 # drop journal table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
648 self.drop_journal_table_indexes(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
649 sql = 'drop table %s__journal'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
650 self.sql(sql) |
| 1165 | 651 |
| 652 for ml in mls: | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
653 # drop multilink table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
654 self.drop_multilink_table_indexes(cn, ml) |
| 1165 | 655 sql = 'drop table %s_%s'%(spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
656 self.sql(sql) |
| 1165 | 657 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
658 def drop_class_table(self, cn): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
659 sql = 'drop table _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
660 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
661 |
| 1165 | 662 # |
| 663 # Classes | |
| 664 # | |
| 665 def __getattr__(self, classname): | |
| 666 ''' A convenient way of calling self.getclass(classname). | |
| 667 ''' | |
| 668 if self.classes.has_key(classname): | |
| 669 return self.classes[classname] | |
| 670 raise AttributeError, classname | |
| 671 | |
| 672 def addclass(self, cl): | |
| 673 ''' Add a Class to the hyperdatabase. | |
| 674 ''' | |
| 675 cn = cl.classname | |
| 676 if self.classes.has_key(cn): | |
| 677 raise ValueError, cn | |
| 678 self.classes[cn] = cl | |
| 679 | |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
680 # add default Edit and View permissions |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
681 self.security.addPermission(name="Create", klass=cn, |
|
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
682 description="User is allowed to create "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
683 self.security.addPermission(name="Edit", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
684 description="User is allowed to edit "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
685 self.security.addPermission(name="View", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
686 description="User is allowed to access "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
687 |
| 1165 | 688 def getclasses(self): |
| 689 ''' Return a list of the names of all existing classes. | |
| 690 ''' | |
| 691 l = self.classes.keys() | |
| 692 l.sort() | |
| 693 return l | |
| 694 | |
| 695 def getclass(self, classname): | |
| 696 '''Get the Class object representing a particular class. | |
| 697 | |
| 698 If 'classname' is not a valid class name, a KeyError is raised. | |
| 699 ''' | |
| 700 try: | |
| 701 return self.classes[classname] | |
| 702 except KeyError: | |
| 703 raise KeyError, 'There is no class called "%s"'%classname | |
| 704 | |
| 705 def clear(self): | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
706 '''Delete all database contents. |
| 1165 | 707 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
708 Note: I don't commit here, which is different behaviour to the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
709 "nuke from orbit" behaviour in the dbs. |
| 1165 | 710 ''' |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
711 logging.getLogger('hyperdb').info('clear') |
| 1165 | 712 for cn in self.classes.keys(): |
| 713 sql = 'delete from _%s'%cn | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
714 self.sql(sql) |
| 1165 | 715 |
| 716 # | |
| 717 # Nodes | |
| 718 # | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
719 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
720 hyperdb_to_sql_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
721 hyperdb.String : str, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
722 # fractional seconds by default |
|
3612
afda59d5d546
prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents:
3601
diff
changeset
|
723 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
724 hyperdb.Link : int, |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
725 hyperdb.Interval : str, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
726 hyperdb.Password : str, |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
727 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
728 hyperdb.Number : lambda x: x, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
729 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
730 } |
| 1165 | 731 def addnode(self, classname, nodeid, node): |
| 732 ''' Add the specified node to its class's db. | |
| 733 ''' | |
| 734 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
735 logging.getLogger('hyperdb').debug('addnode %s%s %r'%(classname, |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
736 nodeid, node)) |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
737 |
|
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
738 # determine the column definitions and multilink tables |
| 1165 | 739 cl = self.classes[classname] |
| 740 cols, mls = self.determine_columns(cl.properties.items()) | |
| 741 | |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
742 # we'll be supplied these props if we're doing an import |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
743 values = node.copy() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
744 if not values.has_key('creator'): |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
745 # add in the "calculated" properties (dupe so we don't affect |
|
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
746 # calling code's node assumptions) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
747 values['creation'] = values['activity'] = date.Date() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
748 values['actor'] = values['creator'] = self.getuid() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
749 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
750 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
751 props = cl.getprops(protected=1) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
752 del props['id'] |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
753 |
| 1165 | 754 # default the non-multilink columns |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
755 for col, prop in props.items(): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
756 if not values.has_key(col): |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
757 if isinstance(prop, Multilink): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
758 values[col] = [] |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
759 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
760 values[col] = None |
| 1165 | 761 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
762 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
763 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
764 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
765 del self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
766 self.cache_lru.remove(key) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
767 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
768 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
769 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
770 for col,dt in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
771 # this is somewhat dodgy.... |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
772 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
773 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
774 value = values[col[2:-6]] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
775 # this is an Interval special "int" column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
776 if value is not None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
777 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
778 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
779 vals.append(value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
780 continue |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
781 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
782 prop = props[col[1:]] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
783 value = values[col[1:]] |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
784 if value is not None: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
785 value = self.hyperdb_to_sql_value[prop.__class__](value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
786 vals.append(value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
787 vals.append(nodeid) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
788 vals = tuple(vals) |
| 1165 | 789 |
| 790 # make sure the ordering is correct for column name -> column value | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
791 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
792 cols = ','.join([col for col,dt in cols]) + ',id' |
| 1165 | 793 |
| 794 # perform the inserts | |
| 795 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
796 self.sql(sql, vals) |
| 1165 | 797 |
| 798 # insert the multilink rows | |
| 799 for col in mls: | |
| 800 t = '%s_%s'%(classname, col) | |
| 801 for entry in node[col]: | |
| 802 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, | |
| 803 self.arg, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
804 self.sql(sql, (entry, nodeid)) |
| 1165 | 805 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
806 def setnode(self, classname, nodeid, values, multilink_changes={}): |
| 1165 | 807 ''' Change the specified node. |
| 808 ''' | |
| 809 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
810 logging.getLogger('hyperdb').debug('setnode %s%s %r' |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
811 % (classname, nodeid, values)) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
812 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
813 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
814 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
815 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
816 del self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
817 self.cache_lru.remove(key) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
818 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
819 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
820 props = cl.getprops() |
| 1165 | 821 |
| 822 cols = [] | |
| 823 mls = [] | |
| 824 # add the multilinks separately | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
825 for col in values.keys(): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
826 prop = props[col] |
| 1165 | 827 if isinstance(prop, Multilink): |
| 828 mls.append(col) | |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
829 elif isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
830 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
831 cols.append(col) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
832 # extra leading '_' added by code below |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
833 cols.append('_' +col + '_int__') |
| 1165 | 834 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
835 cols.append(col) |
| 1165 | 836 cols.sort() |
| 837 | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
838 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
839 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
840 for col in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
841 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
842 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
843 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
844 col = col[1:-6] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
845 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
846 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
847 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
848 vals.append(None) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
849 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
850 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
851 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
852 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
853 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
854 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
855 e = None |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
856 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
857 e = self.hyperdb_to_sql_value[prop.__class__](value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
858 vals.append(e) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
859 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
860 vals.append(int(nodeid)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
861 vals = tuple(vals) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
862 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
863 # if there's any updates to regular columns, do them |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
864 if cols: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
865 # make sure the ordering is correct for column name -> column value |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
866 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols]) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
867 cols = ','.join(cols) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
868 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
869 # perform the update |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
870 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
871 self.sql(sql, vals) |
| 1165 | 872 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
873 # we're probably coming from an import, not a change |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
874 if not multilink_changes: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
875 for name in mls: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
876 prop = props[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
877 value = values[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
878 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
879 t = '%s_%s'%(classname, name) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
880 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
881 # clear out previous values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
882 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
883 self.sql('delete from %s where nodeid=%s'%(t, self.arg), |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
884 (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
885 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
886 # insert the values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
887 for entry in values[name]: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
888 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
889 self.arg, self.arg) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
890 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
891 self.sql(sql, (entry, nodeid)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
892 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
893 # we have multilink changes to apply |
| 1165 | 894 for col, (add, remove) in multilink_changes.items(): |
| 895 tn = '%s_%s'%(classname, col) | |
| 896 if add: | |
| 897 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn, | |
| 898 self.arg, self.arg) | |
| 899 for addid in add: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
900 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
901 self.sql(sql, (int(nodeid), int(addid))) |
| 1165 | 902 if remove: |
| 903 sql = 'delete from %s where nodeid=%s and linkid=%s'%(tn, | |
| 904 self.arg, self.arg) | |
| 905 for removeid in remove: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
906 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
907 self.sql(sql, (int(nodeid), int(removeid))) |
| 1165 | 908 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
909 sql_to_hyperdb_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
910 hyperdb.String : str, |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
911 hyperdb.Date : lambda x:date.Date(str(x).replace(' ', '.')), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
912 # hyperdb.Link : int, # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
913 hyperdb.Link : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
914 hyperdb.Interval : date.Interval, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
915 hyperdb.Password : lambda x: password.Password(encrypted=x), |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
916 hyperdb.Boolean : _bool_cvt, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
917 hyperdb.Number : _num_cvt, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
918 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
919 } |
| 1165 | 920 def getnode(self, classname, nodeid): |
| 921 ''' Get a node from the database. | |
| 922 ''' | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
923 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
924 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
925 if self.cache.has_key(key): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
926 # push us back to the top of the LRU |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
927 self.cache_lru.remove(key) |
|
1185
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
928 self.cache_lru.insert(0, key) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
929 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
930 self.stats['cache_hits'] += 1 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
931 # return the cached information |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
932 return self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
933 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
934 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
935 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
936 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
937 |
| 1165 | 938 # figure the columns we're fetching |
| 939 cl = self.classes[classname] | |
| 940 cols, mls = self.determine_columns(cl.properties.items()) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
941 scols = ','.join([col for col,dt in cols]) |
| 1165 | 942 |
| 943 # perform the basic property fetch | |
| 944 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
945 self.sql(sql, (nodeid,)) |
| 1165 | 946 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
947 values = self.sql_fetchone() |
| 1165 | 948 if values is None: |
| 949 raise IndexError, 'no such %s node %s'%(classname, nodeid) | |
| 950 | |
| 951 # make up the node | |
| 952 node = {} | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
953 props = cl.getprops(protected=1) |
| 1165 | 954 for col in range(len(cols)): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
955 name = cols[col][0][1:] |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
956 if name.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
957 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
958 # ignore the special Interval-as-seconds column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
959 continue |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
960 value = values[col] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
961 if value is not None: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
962 value = self.sql_to_hyperdb_value[props[name].__class__](value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
963 node[name] = value |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
964 |
| 1165 | 965 |
| 966 # now the multilinks | |
| 967 for col in mls: | |
| 968 # get the link ids | |
| 969 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col, | |
| 970 self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
971 self.cursor.execute(sql, (nodeid,)) |
| 1165 | 972 # extract the first column from the result |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
973 # XXX numeric ids |
|
2887
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
974 items = [int(x[0]) for x in self.cursor.fetchall()] |
|
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
975 items.sort () |
|
23ed4195e442
ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents:
2828
diff
changeset
|
976 node[col] = [str(x) for x in items] |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
977 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
978 # save off in the cache |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
979 key = (classname, nodeid) |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
980 self.cache[key] = node |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
981 # update the LRU |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
982 self.cache_lru.insert(0, key) |
|
1185
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
983 if len(self.cache_lru) > ROW_CACHE_SIZE: |
|
3b0735ef8207
fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents:
1183
diff
changeset
|
984 del self.cache[self.cache_lru.pop()] |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
985 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
986 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
987 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
988 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
989 return node |
| 1165 | 990 |
| 991 def destroynode(self, classname, nodeid): | |
| 992 '''Remove a node from the database. Called exclusively by the | |
| 993 destroy() method on Class. | |
| 994 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
995 logging.getLogger('hyperdb').info('destroynode %s%s'%(classname, nodeid)) |
| 1165 | 996 |
| 997 # make sure the node exists | |
| 998 if not self.hasnode(classname, nodeid): | |
| 999 raise IndexError, '%s has no node %s'%(classname, nodeid) | |
| 1000 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1001 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1002 if self.cache.has_key((classname, nodeid)): |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1003 del self.cache[(classname, nodeid)] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1004 |
| 1165 | 1005 # see if there's any obvious commit actions that we should get rid of |
| 1006 for entry in self.transactions[:]: | |
| 1007 if entry[1][:2] == (classname, nodeid): | |
| 1008 self.transactions.remove(entry) | |
| 1009 | |
| 1010 # now do the SQL | |
| 1011 sql = 'delete from _%s where id=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1012 self.sql(sql, (nodeid,)) |
| 1165 | 1013 |
| 1014 # remove from multilnks | |
| 1015 cl = self.getclass(classname) | |
| 1016 x, mls = self.determine_columns(cl.properties.items()) | |
| 1017 for col in mls: | |
| 1018 # get the link ids | |
| 1019 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg) | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
1020 self.sql(sql, (nodeid,)) |
| 1165 | 1021 |
| 1022 # remove journal entries | |
| 1023 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1024 self.sql(sql, (nodeid,)) |
| 1165 | 1025 |
| 1026 def hasnode(self, classname, nodeid): | |
| 1027 ''' Determine if the database has a given node. | |
| 1028 ''' | |
| 1029 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1030 self.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1031 return int(self.cursor.fetchone()[0]) |
| 1165 | 1032 |
| 1033 def countnodes(self, classname): | |
| 1034 ''' Count the number of nodes that exist for a particular Class. | |
| 1035 ''' | |
| 1036 sql = 'select count(*) from _%s'%classname | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1037 self.sql(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1038 return self.cursor.fetchone()[0] |
| 1165 | 1039 |
| 1040 def addjournal(self, classname, nodeid, action, params, creator=None, | |
| 1041 creation=None): | |
| 1042 ''' Journal the Action | |
| 1043 'action' may be: | |
| 1044 | |
| 1045 'create' or 'set' -- 'params' is a dictionary of property values | |
| 1046 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) | |
| 1047 'retire' -- 'params' is None | |
| 1048 ''' | |
| 1049 # handle supply of the special journalling parameters (usually | |
| 1050 # supplied on importing an existing database) | |
| 1051 if creator: | |
| 1052 journaltag = creator | |
| 1053 else: | |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1054 journaltag = self.getuid() |
| 1165 | 1055 if creation: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1056 journaldate = creation |
| 1165 | 1057 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1058 journaldate = date.Date() |
| 1165 | 1059 |
| 1060 # create the journal entry | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1061 cols = 'nodeid,date,tag,action,params' |
| 1165 | 1062 |
| 1063 if __debug__: | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1064 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(classname, |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1065 nodeid, journaldate, journaltag, action, params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1066 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1067 # make the journalled data marshallable |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1068 if isinstance(params, type({})): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1069 self._journal_marshal(params, classname) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1070 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1071 params = repr(params) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1072 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1073 dc = self.hyperdb_to_sql_value[hyperdb.Date] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1074 journaldate = dc(journaldate) |
| 1165 | 1075 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1076 self.save_journal(classname, cols, nodeid, journaldate, |
| 1165 | 1077 journaltag, action, params) |
| 1078 | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1079 def setjournal(self, classname, nodeid, journal): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1080 '''Set the journal to the "journal" list.''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1081 # clear out any existing entries |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1082 self.sql('delete from %s__journal where nodeid=%s'%(classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1083 self.arg), (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1084 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1085 # create the journal entry |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1086 cols = 'nodeid,date,tag,action,params' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1087 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1088 dc = self.hyperdb_to_sql_value[hyperdb.Date] |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1089 for nodeid, journaldate, journaltag, action, params in journal: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1090 if __debug__: |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1091 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%( |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1092 classname, nodeid, journaldate, journaltag, action, |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1093 params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1094 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1095 # make the journalled data marshallable |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1096 if isinstance(params, type({})): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1097 self._journal_marshal(params, classname) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1098 params = repr(params) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1099 |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1100 self.save_journal(classname, cols, nodeid, dc(journaldate), |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1101 journaltag, action, params) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1102 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1103 def _journal_marshal(self, params, classname): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1104 '''Convert the journal params values into safely repr'able and |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1105 eval'able values.''' |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1106 properties = self.getclass(classname).getprops() |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1107 for param, value in params.items(): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1108 if not value: |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1109 continue |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1110 property = properties[param] |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1111 cvt = self.hyperdb_to_sql_value[property.__class__] |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1112 if isinstance(property, Password): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1113 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1114 elif isinstance(property, Date): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1115 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1116 elif isinstance(property, Interval): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1117 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1118 elif isinstance(property, Boolean): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1119 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1120 |
| 1165 | 1121 def getjournal(self, classname, nodeid): |
| 1122 ''' get the journal for id | |
| 1123 ''' | |
| 1124 # make sure the node exists | |
| 1125 if not self.hasnode(classname, nodeid): | |
| 1126 raise IndexError, '%s has no node %s'%(classname, nodeid) | |
| 1127 | |
| 1128 cols = ','.join('nodeid date tag action params'.split()) | |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1129 journal = self.load_journal(classname, cols, nodeid) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1130 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1131 # now unmarshal the data |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1132 dc = self.sql_to_hyperdb_value[hyperdb.Date] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1133 res = [] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1134 properties = self.getclass(classname).getprops() |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1135 for nodeid, date_stamp, user, action, params in journal: |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1136 params = eval(params) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1137 if isinstance(params, type({})): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1138 for param, value in params.items(): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1139 if not value: |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1140 continue |
|
2731
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1141 property = properties.get(param, None) |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1142 if property is None: |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1143 # deleted property |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1144 continue |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1145 cvt = self.sql_to_hyperdb_value[property.__class__] |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1146 if isinstance(property, Password): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1147 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1148 elif isinstance(property, Date): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1149 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1150 elif isinstance(property, Interval): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1151 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1152 elif isinstance(property, Boolean): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1153 params[param] = cvt(value) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1154 # XXX numeric ids |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1155 res.append((str(nodeid), dc(date_stamp), user, action, params)) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1156 return res |
| 1165 | 1157 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1158 def save_journal(self, classname, cols, nodeid, journaldate, |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1159 journaltag, action, params): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1160 ''' Save the journal entry to the database |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1161 ''' |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1162 entry = (nodeid, journaldate, journaltag, action, params) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1163 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1164 # do the insert |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1165 a = self.arg |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1166 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1167 classname, cols, a, a, a, a, a) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1168 self.sql(sql, entry) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1169 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1170 def load_journal(self, classname, cols, nodeid): |
| 1165 | 1171 ''' Load the journal from the database |
| 1172 ''' | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1173 # now get the journal entries |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1174 sql = 'select %s from %s__journal where nodeid=%s order by date'%( |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1175 cols, classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1176 self.sql(sql, (nodeid,)) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1177 return self.cursor.fetchall() |
| 1165 | 1178 |
| 1179 def pack(self, pack_before): | |
| 1180 ''' Delete all journal entries except "create" before 'pack_before'. | |
| 1181 ''' | |
| 2416 | 1182 date_stamp = self.hyperdb_to_sql_value[Date](pack_before) |
| 1165 | 1183 |
| 1184 # do the delete | |
| 1185 for classname in self.classes.keys(): | |
| 1186 sql = "delete from %s__journal where date<%s and "\ | |
| 1187 "action<>'create'"%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1188 self.sql(sql, (date_stamp,)) |
| 1165 | 1189 |
| 1190 def sql_commit(self): | |
| 1191 ''' Actually commit to the database. | |
| 1192 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1193 logging.getLogger('hyperdb').info('commit') |
| 1165 | 1194 self.conn.commit() |
| 1195 | |
|
2374
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1196 # open a new cursor for subsequent work |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1197 self.cursor = self.conn.cursor() |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1198 |
| 1165 | 1199 def commit(self): |
| 1200 ''' Commit the current transactions. | |
| 1201 | |
| 1202 Save all data changed since the database was opened or since the | |
| 1203 last commit() or rollback(). | |
| 1204 ''' | |
| 1205 # commit the database | |
| 1206 self.sql_commit() | |
| 1207 | |
| 1208 # now, do all the other transaction stuff | |
| 1209 for method, args in self.transactions: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1210 method(*args) |
| 1165 | 1211 |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1212 # save the indexer |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1213 self.indexer.save_index() |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1214 |
| 1165 | 1215 # clear out the transactions |
| 1216 self.transactions = [] | |
| 1217 | |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1218 def sql_rollback(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1219 self.conn.rollback() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1220 |
| 1165 | 1221 def rollback(self): |
| 1222 ''' Reverse all actions from the current transaction. | |
| 1223 | |
| 1224 Undo all the changes made since the database was opened or the last | |
| 1225 commit() or rollback() was performed. | |
| 1226 ''' | |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1227 logging.getLogger('hyperdb').info('rollback') |
| 1165 | 1228 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1229 self.sql_rollback() |
| 1165 | 1230 |
| 1231 # roll back "other" transaction stuff | |
| 1232 for method, args in self.transactions: | |
| 1233 # delete temporary files | |
| 1234 if method == self.doStoreFile: | |
| 1235 self.rollbackStoreFile(*args) | |
| 1236 self.transactions = [] | |
| 1237 | |
|
1492
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1238 # clear the cache |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1239 self.clearCache() |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1240 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1241 def sql_close(self): |
|
3155
57b60bda9473
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents:
3147
diff
changeset
|
1242 logging.getLogger('hyperdb').info('close') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1243 self.conn.close() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1244 |
| 1165 | 1245 def close(self): |
| 1246 ''' Close off the connection. | |
| 1247 ''' | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
1248 self.indexer.close() |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1249 self.sql_close() |
| 1165 | 1250 |
| 1251 # | |
| 1252 # The base Class class | |
| 1253 # | |
| 1254 class Class(hyperdb.Class): | |
| 1255 ''' The handle to a particular class of nodes in a hyperdatabase. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1256 |
| 1165 | 1257 All methods except __repr__ and getnode must be implemented by a |
| 1258 concrete backend Class. | |
| 1259 ''' | |
| 1260 | |
| 1261 def schema(self): | |
| 1262 ''' A dumpable version of the schema that we can store in the | |
| 1263 database | |
| 1264 ''' | |
| 1265 return (self.key, [(x, repr(y)) for x,y in self.properties.items()]) | |
| 1266 | |
| 1267 def enableJournalling(self): | |
| 1268 '''Turn journalling on for this class | |
| 1269 ''' | |
| 1270 self.do_journal = 1 | |
| 1271 | |
| 1272 def disableJournalling(self): | |
| 1273 '''Turn journalling off for this class | |
| 1274 ''' | |
| 1275 self.do_journal = 0 | |
| 1276 | |
| 1277 # Editing nodes: | |
| 1278 def create(self, **propvalues): | |
| 1279 ''' Create a new node of this class and return its id. | |
| 1280 | |
| 1281 The keyword arguments in 'propvalues' map property names to values. | |
| 1282 | |
| 1283 The values of arguments must be acceptable for the types of their | |
| 1284 corresponding properties or a TypeError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1285 |
| 1165 | 1286 If this class has a key property, it must be present and its value |
| 1287 must not collide with other key strings or a ValueError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1288 |
| 1165 | 1289 Any other properties on this class that are missing from the |
| 1290 'propvalues' dictionary are set to None. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1291 |
| 1165 | 1292 If an id in a link or multilink property does not refer to a valid |
| 1293 node, an IndexError is raised. | |
| 1294 ''' | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1295 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1296 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1297 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1298 return newid |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1299 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1300 def create_inner(self, **propvalues): |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1301 ''' Called by create, in-between the audit and react calls. |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1302 ''' |
| 1165 | 1303 if propvalues.has_key('id'): |
| 1304 raise KeyError, '"id" is reserved' | |
| 1305 | |
| 1306 if self.db.journaltag is None: | |
| 1307 raise DatabaseError, 'Database open read-only' | |
| 1308 | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1309 if propvalues.has_key('creator') or propvalues.has_key('actor') or \ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1310 propvalues.has_key('creation') or propvalues.has_key('activity'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1311 raise KeyError, '"creator", "actor", "creation" and '\ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1312 '"activity" are reserved' |
| 1165 | 1313 |
| 1314 # new node's id | |
| 1315 newid = self.db.newid(self.classname) | |
| 1316 | |
| 1317 # validate propvalues | |
| 1318 num_re = re.compile('^\d+$') | |
| 1319 for key, value in propvalues.items(): | |
| 1320 if key == self.key: | |
| 1321 try: | |
| 1322 self.lookup(value) | |
| 1323 except KeyError: | |
| 1324 pass | |
| 1325 else: | |
| 1326 raise ValueError, 'node with key "%s" exists'%value | |
| 1327 | |
| 1328 # try to handle this property | |
| 1329 try: | |
| 1330 prop = self.properties[key] | |
| 1331 except KeyError: | |
| 1332 raise KeyError, '"%s" has no property "%s"'%(self.classname, | |
| 1333 key) | |
| 1334 | |
| 1335 if value is not None and isinstance(prop, Link): | |
| 1336 if type(value) != type(''): | |
| 1337 raise ValueError, 'link value must be String' | |
| 1338 link_class = self.properties[key].classname | |
| 1339 # if it isn't a number, it's a key | |
| 1340 if not num_re.match(value): | |
| 1341 try: | |
| 1342 value = self.db.classes[link_class].lookup(value) | |
| 1343 except (TypeError, KeyError): | |
| 1344 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1345 key, value, link_class) | |
| 1346 elif not self.db.getclass(link_class).hasnode(value): | |
| 1347 raise IndexError, '%s has no node %s'%(link_class, value) | |
| 1348 | |
| 1349 # save off the value | |
| 1350 propvalues[key] = value | |
| 1351 | |
| 1352 # register the link with the newly linked node | |
| 1353 if self.do_journal and self.properties[key].do_journal: | |
| 1354 self.db.addjournal(link_class, value, 'link', | |
| 1355 (self.classname, newid, key)) | |
| 1356 | |
| 1357 elif isinstance(prop, Multilink): | |
| 1358 if type(value) != type([]): | |
| 1359 raise TypeError, 'new property "%s" not a list of ids'%key | |
| 1360 | |
| 1361 # clean up and validate the list of links | |
| 1362 link_class = self.properties[key].classname | |
| 1363 l = [] | |
| 1364 for entry in value: | |
| 1365 if type(entry) != type(''): | |
| 1366 raise ValueError, '"%s" multilink value (%r) '\ | |
| 1367 'must contain Strings'%(key, value) | |
| 1368 # if it isn't a number, it's a key | |
| 1369 if not num_re.match(entry): | |
| 1370 try: | |
| 1371 entry = self.db.classes[link_class].lookup(entry) | |
| 1372 except (TypeError, KeyError): | |
| 1373 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1374 key, entry, self.properties[key].classname) | |
| 1375 l.append(entry) | |
| 1376 value = l | |
| 1377 propvalues[key] = value | |
| 1378 | |
| 1379 # handle additions | |
| 1380 for nodeid in value: | |
| 1381 if not self.db.getclass(link_class).hasnode(nodeid): | |
| 1382 raise IndexError, '%s has no node %s'%(link_class, | |
| 1383 nodeid) | |
| 1384 # register the link with the newly linked node | |
| 1385 if self.do_journal and self.properties[key].do_journal: | |
| 1386 self.db.addjournal(link_class, nodeid, 'link', | |
| 1387 (self.classname, newid, key)) | |
| 1388 | |
| 1389 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1390 if type(value) != type('') and type(value) != type(u''): |
| 1165 | 1391 raise TypeError, 'new property "%s" not a string'%key |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1392 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1393 self.db.indexer.add_text((self.classname, newid, key), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1394 value) |
| 1165 | 1395 |
| 1396 elif isinstance(prop, Password): | |
| 1397 if not isinstance(value, password.Password): | |
| 1398 raise TypeError, 'new property "%s" not a Password'%key | |
| 1399 | |
| 1400 elif isinstance(prop, Date): | |
| 1401 if value is not None and not isinstance(value, date.Date): | |
| 1402 raise TypeError, 'new property "%s" not a Date'%key | |
| 1403 | |
| 1404 elif isinstance(prop, Interval): | |
| 1405 if value is not None and not isinstance(value, date.Interval): | |
| 1406 raise TypeError, 'new property "%s" not an Interval'%key | |
| 1407 | |
| 1408 elif value is not None and isinstance(prop, Number): | |
| 1409 try: | |
| 1410 float(value) | |
| 1411 except ValueError: | |
| 1412 raise TypeError, 'new property "%s" not numeric'%key | |
| 1413 | |
| 1414 elif value is not None and isinstance(prop, Boolean): | |
| 1415 try: | |
| 1416 int(value) | |
| 1417 except ValueError: | |
| 1418 raise TypeError, 'new property "%s" not boolean'%key | |
| 1419 | |
| 1420 # make sure there's data where there needs to be | |
| 1421 for key, prop in self.properties.items(): | |
| 1422 if propvalues.has_key(key): | |
| 1423 continue | |
| 1424 if key == self.key: | |
| 1425 raise ValueError, 'key property "%s" is required'%key | |
| 1426 if isinstance(prop, Multilink): | |
| 1427 propvalues[key] = [] | |
| 1428 else: | |
| 1429 propvalues[key] = None | |
| 1430 | |
| 1431 # done | |
| 1432 self.db.addnode(self.classname, newid, propvalues) | |
| 1433 if self.do_journal: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1434 self.db.addjournal(self.classname, newid, ''"create", {}) |
| 1165 | 1435 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1436 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1437 return str(newid) |
| 1165 | 1438 |
| 1439 def get(self, nodeid, propname, default=_marker, cache=1): | |
| 1440 '''Get the value of a property on an existing node of this class. | |
| 1441 | |
| 1442 'nodeid' must be the id of an existing node of this class or an | |
| 1443 IndexError is raised. 'propname' must be the name of a property | |
| 1444 of this class or a KeyError is raised. | |
| 1445 | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1446 'cache' exists for backwards compatibility, and is not used. |
| 1165 | 1447 ''' |
| 1448 if propname == 'id': | |
| 1449 return nodeid | |
| 1450 | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1451 # get the node's dict |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1452 d = self.db.getnode(self.classname, nodeid) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1453 |
| 1165 | 1454 if propname == 'creation': |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1455 if d.has_key('creation'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1456 return d['creation'] |
| 1165 | 1457 else: |
| 1458 return date.Date() | |
| 1459 if propname == 'activity': | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1460 if d.has_key('activity'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1461 return d['activity'] |
| 1165 | 1462 else: |
| 1463 return date.Date() | |
| 1464 if propname == 'creator': | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1465 if d.has_key('creator'): |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1466 return d['creator'] |
| 1165 | 1467 else: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1468 return self.db.getuid() |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1469 if propname == 'actor': |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1470 if d.has_key('actor'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1471 return d['actor'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1472 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1473 return self.db.getuid() |
| 1165 | 1474 |
| 1475 # get the property (raises KeyErorr if invalid) | |
| 1476 prop = self.properties[propname] | |
| 1477 | |
|
2690
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1478 # XXX may it be that propname is valid property name |
|
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1479 # (above error is not raised) and not d.has_key(propname)??? |
|
9054d546fc9f
Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1480 if (not d.has_key(propname)) or (d[propname] is None): |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
1481 if default is _marker: |
| 1165 | 1482 if isinstance(prop, Multilink): |
| 1483 return [] | |
| 1484 else: | |
| 1485 return None | |
| 1486 else: | |
| 1487 return default | |
| 1488 | |
| 1489 # don't pass our list to other code | |
| 1490 if isinstance(prop, Multilink): | |
| 1491 return d[propname][:] | |
| 1492 | |
| 1493 return d[propname] | |
| 1494 | |
| 1495 def set(self, nodeid, **propvalues): | |
| 1496 '''Modify a property on an existing node of this class. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1497 |
| 1165 | 1498 'nodeid' must be the id of an existing node of this class or an |
| 1499 IndexError is raised. | |
| 1500 | |
| 1501 Each key in 'propvalues' must be the name of a property of this | |
| 1502 class or a KeyError is raised. | |
| 1503 | |
| 1504 All values in 'propvalues' must be acceptable types for their | |
| 1505 corresponding properties or a TypeError is raised. | |
| 1506 | |
| 1507 If the value of the key property is set, it must not collide with | |
| 1508 other key strings or a ValueError is raised. | |
| 1509 | |
| 1510 If the value of a Link or Multilink property contains an invalid | |
| 1511 node id, a ValueError is raised. | |
| 1512 ''' | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1513 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1514 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1515 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1516 self.fireReactors('set', nodeid, oldvalues) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1517 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1518 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1519 def set_inner(self, nodeid, **propvalues): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1520 ''' Called by set, in-between the audit and react calls. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1521 ''' |
| 1165 | 1522 if not propvalues: |
| 1523 return propvalues | |
| 1524 | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1525 if propvalues.has_key('creation') or propvalues.has_key('creator') or \ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1526 propvalues.has_key('actor') or propvalues.has_key('activity'): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1527 raise KeyError, '"creation", "creator", "actor" and '\ |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1528 '"activity" are reserved' |
| 1165 | 1529 |
| 1530 if propvalues.has_key('id'): | |
| 1531 raise KeyError, '"id" is reserved' | |
| 1532 | |
| 1533 if self.db.journaltag is None: | |
| 1534 raise DatabaseError, 'Database open read-only' | |
| 1535 | |
| 1536 node = self.db.getnode(self.classname, nodeid) | |
| 1537 if self.is_retired(nodeid): | |
| 1538 raise IndexError, 'Requested item is retired' | |
| 1539 num_re = re.compile('^\d+$') | |
| 1540 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1541 # make a copy of the values dictionary - we'll modify the contents |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1542 propvalues = propvalues.copy() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1543 |
| 1165 | 1544 # if the journal value is to be different, store it in here |
| 1545 journalvalues = {} | |
| 1546 | |
| 1547 # remember the add/remove stuff for multilinks, making it easier | |
| 1548 # for the Database layer to do its stuff | |
| 1549 multilink_changes = {} | |
| 1550 | |
| 1551 for propname, value in propvalues.items(): | |
| 1552 # check to make sure we're not duplicating an existing key | |
| 1553 if propname == self.key and node[propname] != value: | |
| 1554 try: | |
| 1555 self.lookup(value) | |
| 1556 except KeyError: | |
| 1557 pass | |
| 1558 else: | |
| 1559 raise ValueError, 'node with key "%s" exists'%value | |
| 1560 | |
| 1561 # this will raise the KeyError if the property isn't valid | |
| 1562 # ... we don't use getprops() here because we only care about | |
| 1563 # the writeable properties. | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1564 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1565 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1566 except KeyError: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1567 raise KeyError, '"%s" has no property named "%s"'%( |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1568 self.classname, propname) |
| 1165 | 1569 |
| 1570 # if the value's the same as the existing value, no sense in | |
| 1571 # doing anything | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1572 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1573 if value == current: |
| 1165 | 1574 del propvalues[propname] |
| 1575 continue | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1576 journalvalues[propname] = current |
| 1165 | 1577 |
| 1578 # do stuff based on the prop type | |
| 1579 if isinstance(prop, Link): | |
| 1580 link_class = prop.classname | |
| 1581 # if it isn't a number, it's a key | |
| 1582 if value is not None and not isinstance(value, type('')): | |
| 1583 raise ValueError, 'property "%s" link value be a string'%( | |
| 1584 propname) | |
| 1585 if isinstance(value, type('')) and not num_re.match(value): | |
| 1586 try: | |
| 1587 value = self.db.classes[link_class].lookup(value) | |
| 1588 except (TypeError, KeyError): | |
| 1589 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1590 propname, value, prop.classname) | |
| 1591 | |
| 1592 if (value is not None and | |
| 1593 not self.db.getclass(link_class).hasnode(value)): | |
| 1594 raise IndexError, '%s has no node %s'%(link_class, value) | |
| 1595 | |
| 1596 if self.do_journal and prop.do_journal: | |
| 1597 # register the unlink with the old linked node | |
| 1598 if node[propname] is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1599 self.db.addjournal(link_class, node[propname], |
|
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1600 ''"unlink", (self.classname, nodeid, propname)) |
| 1165 | 1601 |
| 1602 # register the link with the newly linked node | |
| 1603 if value is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1604 self.db.addjournal(link_class, value, ''"link", |
| 1165 | 1605 (self.classname, nodeid, propname)) |
| 1606 | |
| 1607 elif isinstance(prop, Multilink): | |
| 1608 if type(value) != type([]): | |
| 1609 raise TypeError, 'new property "%s" not a list of'\ | |
| 1610 ' ids'%propname | |
| 1611 link_class = self.properties[propname].classname | |
| 1612 l = [] | |
| 1613 for entry in value: | |
| 1614 # if it isn't a number, it's a key | |
| 1615 if type(entry) != type(''): | |
| 1616 raise ValueError, 'new property "%s" link value ' \ | |
| 1617 'must be a string'%propname | |
| 1618 if not num_re.match(entry): | |
| 1619 try: | |
| 1620 entry = self.db.classes[link_class].lookup(entry) | |
| 1621 except (TypeError, KeyError): | |
| 1622 raise IndexError, 'new property "%s": %s not a %s'%( | |
| 1623 propname, entry, | |
| 1624 self.properties[propname].classname) | |
| 1625 l.append(entry) | |
| 1626 value = l | |
| 1627 propvalues[propname] = value | |
| 1628 | |
| 1629 # figure the journal entry for this property | |
| 1630 add = [] | |
| 1631 remove = [] | |
| 1632 | |
| 1633 # handle removals | |
| 1634 if node.has_key(propname): | |
| 1635 l = node[propname] | |
| 1636 else: | |
| 1637 l = [] | |
| 1638 for id in l[:]: | |
| 1639 if id in value: | |
| 1640 continue | |
| 1641 # register the unlink with the old linked node | |
| 1642 if self.do_journal and self.properties[propname].do_journal: | |
| 1643 self.db.addjournal(link_class, id, 'unlink', | |
| 1644 (self.classname, nodeid, propname)) | |
| 1645 l.remove(id) | |
| 1646 remove.append(id) | |
| 1647 | |
| 1648 # handle additions | |
| 1649 for id in value: | |
| 1650 if not self.db.getclass(link_class).hasnode(id): | |
| 1651 raise IndexError, '%s has no node %s'%(link_class, id) | |
| 1652 if id in l: | |
| 1653 continue | |
| 1654 # register the link with the newly linked node | |
| 1655 if self.do_journal and self.properties[propname].do_journal: | |
| 1656 self.db.addjournal(link_class, id, 'link', | |
| 1657 (self.classname, nodeid, propname)) | |
| 1658 l.append(id) | |
| 1659 add.append(id) | |
| 1660 | |
| 1661 # figure the journal entry | |
| 1662 l = [] | |
| 1663 if add: | |
| 1664 l.append(('+', add)) | |
| 1665 if remove: | |
| 1666 l.append(('-', remove)) | |
| 1667 multilink_changes[propname] = (add, remove) | |
| 1668 if l: | |
| 1669 journalvalues[propname] = tuple(l) | |
| 1670 | |
| 1671 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1672 if value is not None and type(value) != type('') and type(value) != type(u''): |
| 1165 | 1673 raise TypeError, 'new property "%s" not a string'%propname |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1674 if prop.indexme: |
|
3399
3a0d4e4a0f34
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
1675 if value is None: value = '' |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1676 self.db.indexer.add_text((self.classname, nodeid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1677 value) |
| 1165 | 1678 |
| 1679 elif isinstance(prop, Password): | |
| 1680 if not isinstance(value, password.Password): | |
| 1681 raise TypeError, 'new property "%s" not a Password'%propname | |
| 1682 propvalues[propname] = value | |
| 1683 | |
| 1684 elif value is not None and isinstance(prop, Date): | |
| 1685 if not isinstance(value, date.Date): | |
| 1686 raise TypeError, 'new property "%s" not a Date'% propname | |
| 1687 propvalues[propname] = value | |
| 1688 | |
| 1689 elif value is not None and isinstance(prop, Interval): | |
| 1690 if not isinstance(value, date.Interval): | |
| 1691 raise TypeError, 'new property "%s" not an '\ | |
| 1692 'Interval'%propname | |
| 1693 propvalues[propname] = value | |
| 1694 | |
| 1695 elif value is not None and isinstance(prop, Number): | |
| 1696 try: | |
| 1697 float(value) | |
| 1698 except ValueError: | |
| 1699 raise TypeError, 'new property "%s" not numeric'%propname | |
| 1700 | |
| 1701 elif value is not None and isinstance(prop, Boolean): | |
| 1702 try: | |
| 1703 int(value) | |
| 1704 except ValueError: | |
| 1705 raise TypeError, 'new property "%s" not boolean'%propname | |
| 1706 | |
| 1707 # nothing to do? | |
| 1708 if not propvalues: | |
| 1709 return propvalues | |
| 1710 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1711 # update the activity time |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1712 propvalues['activity'] = date.Date() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1713 propvalues['actor'] = self.db.getuid() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1714 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1715 # do the set |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1716 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes) |
| 1165 | 1717 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1718 # remove the activity props now they're handled |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1719 del propvalues['activity'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1720 del propvalues['actor'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1721 |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1722 # journal the set |
| 1165 | 1723 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1724 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues) |
| 1165 | 1725 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1726 return propvalues |
| 1165 | 1727 |
| 1728 def retire(self, nodeid): | |
| 1729 '''Retire a node. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1730 |
| 1165 | 1731 The properties on the node remain available from the get() method, |
| 1732 and the node's id is never reused. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1733 |
| 1165 | 1734 Retired nodes are not returned by the find(), list(), or lookup() |
| 1735 methods, and other nodes may reuse the values of their key properties. | |
| 1736 ''' | |
| 1737 if self.db.journaltag is None: | |
| 1738 raise DatabaseError, 'Database open read-only' | |
| 1739 | |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1740 self.fireAuditors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1741 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1742 # use the arg for __retired__ to cope with any odd database type |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1743 # conversion (hello, sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1744 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1745 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1746 self.db.sql(sql, (1, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1747 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1748 self.db.addjournal(self.classname, nodeid, ''"retired", None) |
| 1165 | 1749 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1750 self.fireReactors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1751 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1752 def restore(self, nodeid): |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
1753 '''Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1754 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1755 Make node available for all operations like it was before retirement. |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1756 ''' |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1757 if self.db.journaltag is None: |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1758 raise DatabaseError, 'Database open read-only' |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1759 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1760 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1761 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1762 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1763 try: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1764 id = self.lookup(node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1765 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1766 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1767 else: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1768 raise KeyError, "Key property (%s) of retired node clashes with \ |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1769 existing one (%s)" % (key, node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1770 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1771 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1772 # use the arg for __retired__ to cope with any odd database type |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1773 # conversion (hello, sqlite) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1774 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1775 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1776 self.db.sql(sql, (0, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1777 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1778 self.db.addjournal(self.classname, nodeid, ''"restored", None) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1779 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
1780 self.fireReactors('restore', nodeid, None) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1781 |
| 1165 | 1782 def is_retired(self, nodeid): |
| 1783 '''Return true if the node is rerired | |
| 1784 ''' | |
| 1785 sql = 'select __retired__ from _%s where id=%s'%(self.classname, | |
| 1786 self.db.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1787 self.db.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1788 return int(self.db.sql_fetchone()[0]) |
| 1165 | 1789 |
| 1790 def destroy(self, nodeid): | |
| 1791 '''Destroy a node. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1792 |
| 1165 | 1793 WARNING: this method should never be used except in extremely rare |
| 1794 situations where there could never be links to the node being | |
| 1795 deleted | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1796 |
| 1165 | 1797 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1798 |
| 1165 | 1799 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
1800 |
| 1165 | 1801 WARNING: really, use retire() instead |
| 1802 | |
| 1803 Well, I think that's enough warnings. This method exists mostly to | |
| 1804 support the session storage of the cgi interface. | |
| 1805 | |
| 1806 The node is completely removed from the hyperdb, including all journal | |
| 1807 entries. It will no longer be available, and will generally break code | |
| 1808 if there are any references to the node. | |
| 1809 ''' | |
| 1810 if self.db.journaltag is None: | |
| 1811 raise DatabaseError, 'Database open read-only' | |
| 1812 self.db.destroynode(self.classname, nodeid) | |
| 1813 | |
| 1814 def history(self, nodeid): | |
| 1815 '''Retrieve the journal of edits on a particular node. | |
| 1816 | |
| 1817 'nodeid' must be the id of an existing node of this class or an | |
| 1818 IndexError is raised. | |
| 1819 | |
| 1820 The returned list contains tuples of the form | |
| 1821 | |
|
1409
8dc60d87ab42
Fixed a backlog of bug reports, and worked on python 2.3 compatibility:
Richard Jones <richard@users.sourceforge.net>
parents:
1383
diff
changeset
|
1822 (nodeid, date, tag, action, params) |
| 1165 | 1823 |
| 1824 'date' is a Timestamp object specifying the time of the change and | |
| 1825 'tag' is the journaltag specified when the database was opened. | |
| 1826 ''' | |
| 1827 if not self.do_journal: | |
| 1828 raise ValueError, 'Journalling is disabled for this class' | |
| 1829 return self.db.getjournal(self.classname, nodeid) | |
| 1830 | |
| 1831 # Locating nodes: | |
| 1832 def hasnode(self, nodeid): | |
| 1833 '''Determine if the given nodeid actually exists | |
| 1834 ''' | |
| 1835 return self.db.hasnode(self.classname, nodeid) | |
| 1836 | |
| 1837 def setkey(self, propname): | |
| 1838 '''Select a String property of this class to be the key property. | |
| 1839 | |
| 1840 'propname' must be the name of a String property of this class or | |
| 1841 None, or a TypeError is raised. The values of the key property on | |
| 1842 all existing nodes must be unique or a ValueError is raised. | |
| 1843 ''' | |
| 1844 prop = self.getprops()[propname] | |
| 1845 if not isinstance(prop, String): | |
| 1846 raise TypeError, 'key properties must be String' | |
| 1847 self.key = propname | |
| 1848 | |
| 1849 def getkey(self): | |
| 1850 '''Return the name of the key property for this class or None.''' | |
| 1851 return self.key | |
| 1852 | |
| 1853 def lookup(self, keyvalue): | |
| 1854 '''Locate a particular node by its key property and return its id. | |
| 1855 | |
| 1856 If this class has no key property, a TypeError is raised. If the | |
| 1857 'keyvalue' matches one of the values for the key property among | |
| 1858 the nodes in this class, the matching node's id is returned; | |
| 1859 otherwise a KeyError is raised. | |
| 1860 ''' | |
| 1861 if not self.key: | |
| 1862 raise TypeError, 'No key property set for class %s'%self.classname | |
| 1863 | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1864 # use the arg to handle any odd database type conversion (hello, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1865 # sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1866 sql = "select id from _%s where _%s=%s and __retired__ <> %s"%( |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1867 self.classname, self.key, self.db.arg, self.db.arg) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1868 self.db.sql(sql, (keyvalue, 1)) |
| 1165 | 1869 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1870 # see if there was a result that's not retired |
|
1203
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1871 row = self.db.sql_fetchone() |
|
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1872 if not row: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1873 raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key, |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1874 keyvalue, self.classname) |
| 1165 | 1875 |
| 1876 # return the id | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1877 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1878 return str(row[0]) |
| 1165 | 1879 |
| 1880 def find(self, **propspec): | |
| 1881 '''Get the ids of nodes in this class which link to the given nodes. | |
| 1882 | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1883 'propspec' consists of keyword args propname=nodeid or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1884 propname={nodeid:1, } |
| 1165 | 1885 'propname' must be the name of a property in this class, or a |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1886 KeyError is raised. That property must be a Link or |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1887 Multilink property, or a TypeError is raised. |
| 1165 | 1888 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
1889 Any node in this class whose 'propname' property links to any of |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
1890 the nodeids will be returned. Examples:: |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
1891 |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
1892 db.issue.find(messages='1') |
| 1165 | 1893 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) |
| 1894 ''' | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1895 # shortcut |
| 1165 | 1896 if not propspec: |
| 1897 return [] | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1898 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1899 # validate the args |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1900 props = self.getprops() |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1901 propspec = propspec.items() |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1902 for propname, nodeids in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1903 # check the prop is OK |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1904 prop = props[propname] |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1905 if not isinstance(prop, Link) and not isinstance(prop, Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1906 raise TypeError, "'%s' not a Link/Multilink property"%propname |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1907 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1908 # first, links |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1909 a = self.db.arg |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1910 allvalues = () |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1911 sql = [] |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1912 where = [] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1913 for prop, values in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1914 if not isinstance(props[prop], hyperdb.Link): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1915 continue |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1916 if type(values) is type({}) and len(values) == 1: |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
1917 values = values.keys()[0] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1918 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1919 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1920 where.append('_%s = %s'%(prop, a)) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
1921 elif values is None: |
|
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
1922 where.append('_%s is NULL'%prop) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1923 else: |
|
2494
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1924 values = values.keys() |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1925 s = '' |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1926 if None in values: |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1927 values.remove(None) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1928 s = '_%s is NULL or '%prop |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1929 allvalues += tuple(values) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
1930 s += '_%s in (%s)'%(prop, ','.join([a]*len(values))) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
1931 where.append('(' + s +')') |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1932 if where: |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1933 allvalues = (1, ) + allvalues |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1934 sql.append('''select id from _%s where __retired__ <> %s |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1935 and %s'''%(self.classname, a, ' and '.join(where))) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1936 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1937 # now multilinks |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1938 for prop, values in propspec: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1939 if not isinstance(props[prop], hyperdb.Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1940 continue |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1941 if not values: |
|
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1942 continue |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1943 allvalues += (1, ) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1944 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1945 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1946 s = a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1947 else: |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1948 allvalues += tuple(values.keys()) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1949 s = ','.join([a]*len(values)) |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1950 tn = '%s_%s'%(self.classname, prop) |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1951 sql.append('''select id from _%s, %s where __retired__ <> %s |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1952 and id = %s.nodeid and %s.linkid in (%s)'''%(self.classname, |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1953 tn, a, tn, tn, s)) |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1954 |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1955 if not sql: |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
1956 return [] |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
1957 sql = ' union '.join(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1958 self.db.sql(sql, allvalues) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1959 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1960 l = [str(x[0]) for x in self.db.sql_fetchall()] |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1961 return l |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1962 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1963 def stringFind(self, **requirements): |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1964 '''Locate a particular node by matching a set of its String |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1965 properties in a caseless search. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1966 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1967 If the property is not a String property, a TypeError is raised. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1968 |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1969 The return is a list of the id of all nodes that match. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1970 ''' |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1971 where = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1972 args = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1973 for propname in requirements.keys(): |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1974 prop = self.properties[propname] |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
1975 if not isinstance(prop, String): |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1976 raise TypeError, "'%s' not a String property"%propname |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1977 where.append(propname) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1978 args.append(requirements[propname].lower()) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1979 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1980 # generate the where clause |
|
1549
a53a7e197360
fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents:
1539
diff
changeset
|
1981 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where]) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1982 sql = 'select id from _%s where %s and __retired__<>%s'%( |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1983 self.classname, s, self.db.arg) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1984 args.append(1) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
1985 self.db.sql(sql, tuple(args)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1986 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1987 l = [str(x[0]) for x in self.db.sql_fetchall()] |
| 1165 | 1988 return l |
| 1989 | |
| 1990 def list(self): | |
| 1991 ''' Return a list of the ids of the active nodes in this class. | |
| 1992 ''' | |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1993 return self.getnodeids(retired=0) |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1994 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1995 def getnodeids(self, retired=None): |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1996 ''' Retrieve all the ids of the nodes for a particular Class. |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1997 |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1998 Set retired=None to get all nodes. Otherwise it'll get all the |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
1999 retired or non-retired nodes, depending on the flag. |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2000 ''' |
|
1707
3d627e34f18e
sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1599
diff
changeset
|
2001 # flip the sense of the 'retired' flag if we don't want all of them |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2002 if retired is not None: |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2003 if retired: |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2004 args = (0, ) |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2005 else: |
|
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2006 args = (1, ) |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2007 sql = 'select id from _%s where __retired__ <> %s'%(self.classname, |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2008 self.db.arg) |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2009 else: |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2010 args = () |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2011 sql = 'select id from _%s'%self.classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2012 self.db.sql(sql, args) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2013 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2014 ids = [str(x[0]) for x in self.db.cursor.fetchall()] |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2015 return ids |
| 1165 | 2016 |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2017 def _subselect (self, cn, tn) : |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2018 '''Create a subselect. This is factored out because some |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2019 databases (hmm only one, so far) doesn't support subselects |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2020 look for "I can't believe it's not a toy RDBMS" in the mysql |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2021 backend. |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2022 ''' |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2023 return '_%s.id not in (select nodeid from %s)'%(cn, tn) |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2024 |
|
1249
6c24a86a12ae
Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents:
1244
diff
changeset
|
2025 def filter(self, search_matches, filterspec, sort=(None,None), |
|
6c24a86a12ae
Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents:
1244
diff
changeset
|
2026 group=(None,None)): |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2027 '''Return a list of the ids of the active nodes in this class that |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2028 match the 'filter' spec, sorted by the group spec and then the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2029 sort spec |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2030 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2031 "filterspec" is {propname: value(s)} |
| 1165 | 2032 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2033 "sort" and "group" are (dir, prop) where dir is '+', '-' or None |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2034 and prop is a prop name or None |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2035 |
|
2362
10fc45eea226
fix SearchAction use of Class.filter(), and clarify API docs for same
Richard Jones <richard@users.sourceforge.net>
parents:
2319
diff
changeset
|
2036 "search_matches" is {nodeid: marker} or None |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2037 |
|
3455
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
2038 The filter must match all properties specificed. If the property |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
2039 value to match is a list: |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
2040 |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
2041 1. String properties must match all elements in the list, and |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
2042 2. Other properties must match any of the elements in the list. |
| 1165 | 2043 ''' |
|
2578
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2044 # we can't match anything if search_matches is empty |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2045 if search_matches == {}: |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2046 return [] |
|
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2047 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2048 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2049 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2050 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2051 icn = self.classname |
| 1165 | 2052 |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2053 # vars to hold the components of the SQL statement |
|
2586
3c67f4bfa225
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
2054 frum = [] # FROM clauses |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2055 loj = [] # LEFT OUTER JOIN clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2056 where = [] # WHERE clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2057 args = [] # *any* positional arguments |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2058 a = self.db.arg |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2059 |
| 1165 | 2060 # figure the WHERE clause from the filterspec |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2061 mlfilt = 0 # are we joining with Multilink tables? |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2062 proptree = self._proptree (filterspec) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2063 for p in proptree : |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2064 cn = p.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2065 ln = p.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2066 pln = p.parent.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2067 pcn = p.parent.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2068 k = p.name |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2069 v = p.val |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2070 propclass = p.prcls |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2071 if p.children : |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2072 if isinstance (propclass, Multilink) : |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2073 mlfilt = 1 |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2074 mn = '%s_%s'%(pcn, p.name) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2075 frum.append(mn) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2076 frum.append('_%s as _%s' % (cn, ln)) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2077 where.append('_%s.id=%s.nodeid and %s.linkid=_%s.id'%(pln, |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2078 mn, mn, ln)) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2079 else : |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2080 if not isinstance (propclass, Link) : |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2081 raise ValueError,"%s must be Link/Multilink property"%k |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2082 frum.append('_%s as _%s' % (cn, ln)) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2083 where.append('_%s._%s=_%s.id'%(pln, k, ln)) |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2084 continue |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2085 # now do other where clause stuff |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2086 elif isinstance(propclass, Multilink): |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2087 mlfilt = 1 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2088 tn = '%s_%s'%(pcn, k) |
|
1556
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2089 if v in ('-1', ['-1']): |
|
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2090 # only match rows that have count(linkid)=0 in the |
|
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2091 # corresponding multilink table) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2092 where.append(self._subselect(pcn, tn)) |
|
1556
9e89f5394f6c
handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents:
1555
diff
changeset
|
2093 elif isinstance(v, type([])): |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1549
diff
changeset
|
2094 frum.append(tn) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
2095 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2096 where.append('_%s.id=%s.nodeid and %s.linkid in (%s)'%(pln, |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2097 tn, tn, s)) |
| 1165 | 2098 args = args + v |
| 2099 else: | |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1549
diff
changeset
|
2100 frum.append(tn) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2101 where.append('_%s.id=%s.nodeid and %s.linkid=%s'%(pln, tn, |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2102 tn, a)) |
| 1165 | 2103 args.append(v) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2104 elif k == 'id': |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2105 if isinstance(v, type([])): |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2106 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2107 where.append('_%s.%s in (%s)'%(pln, k, s)) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2108 args = args + v |
|
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2109 else: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2110 where.append('_%s.%s=%s'%(pln, k, a)) |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2111 args.append(v) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2112 elif isinstance(propclass, String): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2113 if not isinstance(v, type([])): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2114 v = [v] |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2115 |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2116 # Quote the bits in the string that need it and then embed |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2117 # in a "substring" search. Note - need to quote the '%' so |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2118 # they make it through the python layer happily |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2119 v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v] |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2120 |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2121 # now add to the where clause |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2122 where.append('(' |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2123 +' and '.join(["_%s._%s LIKE '%s'"%(pln, k, s) for s in v]) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2124 +')') |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2125 # note: args are embedded in the query string now |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2126 elif isinstance(propclass, Link): |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2127 if isinstance(v, type([])): |
|
2508
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2128 d = {} |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2129 for entry in v: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2130 if entry == '-1': |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2131 entry = None |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2132 d[entry] = entry |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2133 l = [] |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2134 if d.has_key(None) or not d: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2135 del d[None] |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2136 l.append('_%s._%s is NULL'%(pln, k)) |
|
2508
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2137 if d: |
|
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2138 v = d.keys() |
|
1295
9c3459cb8ab6
work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
2139 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2140 l.append('(_%s._%s in (%s))'%(pln, k, s)) |
|
1295
9c3459cb8ab6
work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
2141 args = args + v |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2142 if l: |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2143 where.append('(' + ' or '.join(l) +')') |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2144 else: |
|
2508
8e1c1623710b
rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
2145 if v in ('-1', None): |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2146 v = None |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2147 where.append('_%s._%s is NULL'%(pln, k)) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2148 else: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2149 where.append('_%s._%s=%s'%(pln, k, a)) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2150 args.append(v) |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2151 elif isinstance(propclass, Date): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2152 dc = self.db.hyperdb_to_sql_value[hyperdb.Date] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2153 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2154 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2155 where.append('_%s._%s in (%s)'%(pln, k, s)) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2156 args = args + [dc(date.Date(x)) for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2157 else: |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2158 try: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2159 # Try to filter on range of dates |
|
3586
f47bddab5a49
date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents:
3554
diff
changeset
|
2160 date_rng = propclass.range_from_raw(v, self.db) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2161 if date_rng.from_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2162 where.append('_%s._%s >= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2163 args.append(dc(date_rng.from_value)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2164 if date_rng.to_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2165 where.append('_%s._%s <= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2166 args.append(dc(date_rng.to_value)) |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2167 except ValueError: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2168 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2169 pass |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2170 elif isinstance(propclass, Interval): |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2171 # filter using the __<prop>_int__ column |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2172 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2173 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2174 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2175 args = args + [date.Interval(x).as_seconds() for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2176 else: |
|
1596
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2177 try: |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2178 # Try to filter on range of intervals |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2179 date_rng = Range(v, date.Interval) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2180 if date_rng.from_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2181 where.append('_%s.__%s_int__ >= %s'%(pln, k, a)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2182 args.append(date_rng.from_value.as_seconds()) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2183 if date_rng.to_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2184 where.append('_%s.__%s_int__ <= %s'%(pln, k, a)) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2185 args.append(date_rng.to_value.as_seconds()) |
|
1596
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2186 except ValueError: |
|
33a0d94c7658
searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1578
diff
changeset
|
2187 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2188 pass |
| 1165 | 2189 else: |
| 2190 if isinstance(v, type([])): | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2191 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2192 where.append('_%s._%s in (%s)'%(pln, k, s)) |
| 1165 | 2193 args = args + v |
| 2194 else: | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2195 where.append('_%s._%s=%s'%(pln, k, a)) |
| 1165 | 2196 args.append(v) |
| 2197 | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2198 props = self.getprops() |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2199 |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2200 # don't match retired nodes |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2201 where.append('_%s.__retired__ <> 1'%icn) |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2202 |
| 1165 | 2203 # add results of full text search |
| 2204 if search_matches is not None: | |
| 2205 v = search_matches.keys() | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2206 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2207 where.append('_%s.id in (%s)'%(icn, s)) |
| 1165 | 2208 args = args + v |
| 2209 | |
|
2720
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2210 # sanity check: sorting *and* grouping on the same property? |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2211 if group[1] == sort[1]: |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2212 sort = (None, None) |
|
15d945dff53d
add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents:
2692
diff
changeset
|
2213 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2214 # "grouping" is just the first-order sorting in the SQL fetch |
| 1165 | 2215 orderby = [] |
| 2216 ordercols = [] | |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2217 mlsort = [] |
|
3525
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2218 rhsnum = 0 |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2219 for sortby in group, sort: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2220 sdir, prop = sortby |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2221 if sdir and prop: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2222 if isinstance(props[prop], Multilink): |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2223 mlsort.append(sortby) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2224 continue |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2225 elif isinstance(props[prop], Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2226 # use the int column for sorting |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2227 o = '__'+prop+'_int__' |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2228 ordercols.append(o) |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2229 elif isinstance(props[prop], Link): |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2230 # determine whether the linked Class has an order property |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2231 lcn = props[prop].classname |
|
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2232 link = self.db.classes[lcn] |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2233 o = '_%s._%s'%(icn, prop) |
|
3476
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
2234 op = link.orderprop() |
|
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
2235 if op != 'id': |
|
2318
fa2f7ba34399
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2275
diff
changeset
|
2236 tn = '_' + lcn |
|
3525
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2237 rhs = 'rhs%s_'%rhsnum |
|
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2238 rhsnum += 1 |
|
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2239 loj.append('LEFT OUTER JOIN %s as %s on %s=%s.id'%( |
|
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2240 tn, rhs, o, rhs)) |
|
7be25d75c3d5
Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents:
3489
diff
changeset
|
2241 o = '%s._%s'%(rhs, op) |
|
3476
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
2242 ordercols.append(o) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2243 elif prop == 'id': |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2244 o = '_%s.id'%icn |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2245 else: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2246 o = '_%s._%s'%(icn, prop) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2247 ordercols.append(o) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2248 if sdir == '-': |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2249 o += ' desc' |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2250 orderby.append(o) |
| 1165 | 2251 |
| 2252 # construct the SQL | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2253 frum.append('_'+icn) |
| 1165 | 2254 frum = ','.join(frum) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2255 if where: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2256 where = ' where ' + (' and '.join(where)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2257 else: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2258 where = '' |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2259 if mlfilt: |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2260 # we're joining tables on the id, so we will get dupes if we |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2261 # don't distinct() |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2262 cols = ['distinct(_%s.id)'%icn] |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2263 else: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2264 cols = ['_%s.id'%icn] |
| 1165 | 2265 if orderby: |
| 2266 cols = cols + ordercols | |
| 2267 order = ' order by %s'%(','.join(orderby)) | |
| 2268 else: | |
| 2269 order = '' | |
| 2270 cols = ','.join(cols) | |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2271 loj = ' '.join(loj) |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2272 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order) |
| 1165 | 2273 args = tuple(args) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2274 __traceback_info__ = (sql, args) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2275 self.db.sql(sql, args) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
2276 l = self.db.sql_fetchall() |
| 1165 | 2277 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2278 # return the IDs (the first column) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2279 # XXX numeric ids |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2280 l = [str(row[0]) for row in l] |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2281 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2282 if not mlsort: |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2283 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2284 self.db.stats['filtering'] += (time.time() - start_t) |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2285 return l |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2286 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2287 # ergh. someone wants to sort by a multilink. |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2288 r = [] |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2289 for id in l: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2290 m = [] |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2291 for ml in mlsort: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2292 m.append(self.get(id, ml[1])) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2293 r.append((id, m)) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2294 i = 0 |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2295 for sortby in mlsort: |
|
2234
70d21059aa18
fix nested scope bug in multilink sorting in rdbms filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2228
diff
changeset
|
2296 def sortfun(a, b, dir=sortby[i], i=i): |
|
2185
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2297 if dir == '-': |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2298 return cmp(b[1][i], a[1][i]) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2299 else: |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2300 return cmp(a[1][i], b[1][i]) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2301 r.sort(sortfun) |
|
c52a931879c4
sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents:
2175
diff
changeset
|
2302 i += 1 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2303 r = [i[0] for i in r] |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2304 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2305 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2306 self.db.stats['filtering'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2307 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2308 return r |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2309 |
| 1165 | 2310 def count(self): |
| 2311 '''Get the number of nodes in this class. | |
| 2312 | |
| 2313 If the returned integer is 'numnodes', the ids of all the nodes | |
| 2314 in this class run from 1 to numnodes, and numnodes+1 will be the | |
| 2315 id of the next node to be created in this class. | |
| 2316 ''' | |
| 2317 return self.db.countnodes(self.classname) | |
| 2318 | |
| 2319 # Manipulating properties: | |
| 2320 def getprops(self, protected=1): | |
| 2321 '''Return a dictionary mapping property names to property objects. | |
| 2322 If the "protected" flag is true, we include protected properties - | |
| 2323 those which may not be modified. | |
| 2324 ''' | |
| 2325 d = self.properties.copy() | |
| 2326 if protected: | |
| 2327 d['id'] = String() | |
| 2328 d['creation'] = hyperdb.Date() | |
| 2329 d['activity'] = hyperdb.Date() | |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1175
diff
changeset
|
2330 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2331 d['actor'] = hyperdb.Link('user') |
| 1165 | 2332 return d |
| 2333 | |
| 2334 def addprop(self, **properties): | |
| 2335 '''Add properties to this class. | |
| 2336 | |
| 2337 The keyword arguments in 'properties' must map names to property | |
| 2338 objects, or a TypeError is raised. None of the keys in 'properties' | |
| 2339 may collide with the names of existing properties, or a ValueError | |
| 2340 is raised before any properties have been added. | |
| 2341 ''' | |
| 2342 for key in properties.keys(): | |
| 2343 if self.properties.has_key(key): | |
| 2344 raise ValueError, key | |
| 2345 self.properties.update(properties) | |
| 2346 | |
| 2347 def index(self, nodeid): | |
| 2348 '''Add (or refresh) the node to search indexes | |
| 2349 ''' | |
| 2350 # find all the String properties that have indexme | |
| 2351 for prop, propclass in self.getprops().items(): | |
| 2352 if isinstance(propclass, String) and propclass.indexme: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2353 self.db.indexer.add_text((self.classname, nodeid, prop), |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2354 str(self.get(nodeid, prop))) |
| 1165 | 2355 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2356 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2357 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2358 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2359 def export_list(self, propnames, nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2360 ''' Export a node - generate a list of CSV-able data in the order |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2361 specified by propnames for the given node. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2362 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2363 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2364 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2365 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2366 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2367 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2368 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2369 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2370 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2371 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2372 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2373 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2374 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2375 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2376 value = str(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2377 l.append(repr(value)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2378 l.append(repr(self.is_retired(nodeid))) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2379 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2380 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2381 def import_list(self, propnames, proplist): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2382 ''' Import a node - all information including "id" is present and |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2383 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2384 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2385 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2386 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2387 Return the nodeid of the node imported. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2388 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2389 if self.db.journaltag is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2390 raise DatabaseError, 'Database open read-only' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2391 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2392 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2393 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2394 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2395 retire = 0 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2396 if not "id" in propnames: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2397 newid = self.db.newid(self.classname) |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2398 else: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2399 newid = eval(proplist[propnames.index("id")]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2400 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2401 # Use eval to reverse the repr() used to output the CSV |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2402 value = eval(proplist[i]) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2403 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2404 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2405 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2406 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2407 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2408 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2409 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2410 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2411 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2412 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2413 retire = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2414 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2415 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2416 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2417 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2418 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2419 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2420 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2421 # don't set Nones |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2422 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2423 elif isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2424 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2425 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2426 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2427 elif isinstance(prop, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2428 pwd = password.Password() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2429 pwd.unpack(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2430 value = pwd |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2431 elif isinstance(prop, String): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2432 if isinstance(value, unicode): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2433 value = value.encode('utf8') |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2434 if not isinstance(value, str): |
|
2691
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2435 raise TypeError, \ |
|
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2436 'new property "%(propname)s" not a string: %(value)r' \ |
|
4008af78746a
Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2690
diff
changeset
|
2437 % locals() |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2438 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2439 self.db.indexer.add_text((self.classname, newid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2440 value) |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2441 d[propname] = value |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2442 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2443 # get a new id if necessary |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2444 if newid is None: |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2445 newid = self.db.newid(self.classname) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2446 |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2447 # insert new node or update existing? |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2448 if not self.hasnode(newid): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2449 self.db.addnode(self.classname, newid, d) # insert |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2450 else: |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2451 self.db.setnode(self.classname, newid, d) # update |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2452 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2453 # retire? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2454 if retire: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2455 # use the arg for __retired__ to cope with any odd database type |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2456 # conversion (hello, sqlite) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2457 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2458 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2459 self.db.sql(sql, (1, newid)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2460 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2461 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2462 def export_journals(self): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2463 '''Export a class's journal - generate a list of lists of |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2464 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2465 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2466 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2467 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2468 No heading here - the columns are fixed. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2469 ''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2470 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2471 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2472 for nodeid in self.getnodeids(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2473 for nodeid, date, user, action, params in self.history(nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2474 date = date.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2475 if action == 'set': |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2476 export_data = {} |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2477 for propname, value in params.items(): |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2478 if not properties.has_key(propname): |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2479 # property no longer in the schema |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2480 continue |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2481 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2482 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2483 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2484 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2485 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2486 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2487 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2488 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2489 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2490 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2491 value = str(value) |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2492 export_data[propname] = value |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2493 params = export_data |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2494 l = [nodeid, date, user, action, params] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2495 r.append(map(repr, l)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2496 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2497 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2498 def import_journals(self, entries): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2499 '''Import a class's journal. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2500 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2501 Uses setjournal() to set the journal for each item.''' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2502 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2503 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2504 for l in entries: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2505 l = map(eval, l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2506 nodeid, jdate, user, action, params = l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2507 r = d.setdefault(nodeid, []) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2508 if action == 'set': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2509 for propname, value in params.items(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2510 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2511 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2512 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2513 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2514 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2515 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2516 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2517 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2518 pwd = password.Password() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2519 pwd.unpack(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2520 value = pwd |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2521 params[propname] = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2522 r.append((nodeid, date.Date(jdate), user, action, params)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2523 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2524 for nodeid, l in d.items(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2525 self.db.setjournal(self.classname, nodeid, l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2526 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2586
diff
changeset
|
2527 class FileClass(hyperdb.FileClass, Class): |
| 1165 | 2528 '''This class defines a large chunk of data. To support this, it has a |
| 2529 mandatory String property "content" which is typically saved off | |
| 2530 externally to the hyperdb. | |
| 2531 | |
| 2532 The default MIME type of this data is defined by the | |
| 2533 "default_mime_type" class attribute, which may be overridden by each | |
| 2534 node if the class defines a "type" String property. | |
| 2535 ''' | |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2536 def __init__(self, db, classname, **properties): |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2537 '''The newly-created class automatically includes the "content" |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2538 and "type" properties. |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2539 ''' |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2540 if not properties.has_key('content'): |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2541 properties['content'] = hyperdb.String(indexme='yes') |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2542 if not properties.has_key('type'): |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2543 properties['type'] = hyperdb.String() |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2544 Class.__init__(self, db, classname, **properties) |
| 1165 | 2545 |
| 2546 def create(self, **propvalues): | |
| 2547 ''' snaffle the file propvalue and store in a file | |
| 2548 ''' | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2549 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2550 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2551 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2552 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2553 # now remove the content property so it's not stored in the db |
| 1165 | 2554 content = propvalues['content'] |
| 2555 del propvalues['content'] | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2556 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2557 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2558 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2559 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2560 # figure the mime type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2561 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2562 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2563 # and index! |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2564 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2565 self.db.indexer.add_text((self.classname, newid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2566 content, mime_type) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2567 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2568 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2569 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2570 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2571 # store off the content as a file |
| 1165 | 2572 self.db.storefile(self.classname, newid, None, content) |
| 2573 return newid | |
| 2574 | |
| 2575 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2576 ''' Trap the content propname and get it from the file |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2577 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2578 'cache' exists for backwards compatibility, and is not used. |
| 1165 | 2579 ''' |
| 2580 poss_msg = 'Possibly a access right configuration problem.' | |
| 2581 if propname == 'content': | |
| 2582 try: | |
| 2583 return self.db.getfile(self.classname, nodeid, None) | |
| 2584 except IOError, (strerror): | |
| 2585 # BUG: by catching this we donot see an error in the log. | |
| 2586 return 'ERROR reading file: %s%s\n%s\n%s'%( | |
| 2587 self.classname, nodeid, poss_msg, strerror) | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
2588 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2589 return Class.get(self, nodeid, propname, default) |
| 1165 | 2590 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2591 return Class.get(self, nodeid, propname) |
| 1165 | 2592 |
| 2593 def getprops(self, protected=1): | |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2594 '''In addition to the actual properties on the node, these methods |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2595 provide the "content" property. If the "protected" flag is true, |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2596 we include protected properties - those which may not be |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2597 modified. |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2598 |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2599 Note that the content prop is indexed separately, hence no indexme. |
| 1165 | 2600 ''' |
| 2601 d = Class.getprops(self, protected=protected).copy() | |
| 2602 d['content'] = hyperdb.String() | |
| 2603 return d | |
| 2604 | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2605 def set(self, itemid, **propvalues): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2606 ''' Snarf the "content" propvalue and update it in a file |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2607 ''' |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2608 self.fireAuditors('set', itemid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2609 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
| 1165 | 2610 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2611 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2612 content = None |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2613 if propvalues.has_key('content'): |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2614 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2615 del propvalues['content'] |
| 1165 | 2616 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2617 # do the database create |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2618 propvalues = self.set_inner(itemid, **propvalues) |
| 1165 | 2619 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2620 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2621 if content: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2622 # store and possibly index |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2623 self.db.storefile(self.classname, itemid, None, content) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2624 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2625 mime_type = self.get(itemid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2626 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2627 content, mime_type) |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
2628 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
2629 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2630 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2631 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2632 return propvalues |
| 1165 | 2633 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2634 def index(self, nodeid): |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2635 ''' Add (or refresh) the node to search indexes. |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2636 |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2637 Use the content-type property for the content property. |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2638 ''' |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2639 # find all the String properties that have indexme |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2640 for prop, propclass in self.getprops().items(): |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2641 if prop == 'content' and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2642 mime_type = self.get(nodeid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2643 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2644 str(self.get(nodeid, 'content')), mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2645 elif isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2646 # index them under (classname, nodeid, property) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2647 try: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2648 value = str(self.get(nodeid, prop)) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2649 except IndexError: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2650 # node has been destroyed |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2651 continue |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2652 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2653 |
| 1165 | 2654 # XXX deviation from spec - was called ItemClass |
| 2655 class IssueClass(Class, roundupdb.IssueClass): | |
| 2656 # Overridden methods: | |
| 2657 def __init__(self, db, classname, **properties): | |
| 2658 '''The newly-created class automatically includes the "messages", | |
| 2659 "files", "nosy", and "superseder" properties. If the 'properties' | |
| 2660 dictionary attempts to specify any of these properties or a | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2661 "creation", "creator", "activity" or "actor" property, a ValueError |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2662 is raised. |
| 1165 | 2663 ''' |
| 2664 if not properties.has_key('title'): | |
| 2665 properties['title'] = hyperdb.String(indexme='yes') | |
| 2666 if not properties.has_key('messages'): | |
| 2667 properties['messages'] = hyperdb.Multilink("msg") | |
| 2668 if not properties.has_key('files'): | |
| 2669 properties['files'] = hyperdb.Multilink("file") | |
| 2670 if not properties.has_key('nosy'): | |
| 2671 # note: journalling is turned off as it really just wastes | |
| 2672 # space. this behaviour may be overridden in an instance | |
| 2673 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
| 2674 if not properties.has_key('superseder'): | |
| 2675 properties['superseder'] = hyperdb.Multilink(classname) | |
| 2676 Class.__init__(self, db, classname, **properties) | |
| 2677 | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
2678 # vim: set et sts=4 sw=4 : |
