annotate roundup/backends/rdbms_common.py @ 3634:57c66056ffe4

Implemented what I'll call for now "transitive searching"... ...using the filter method. The first idea was mentioned on the roundup-users mailing list: http://article.gmane.org/gmane.comp.bug-tracking.roundup.user/6909 We can now search for items which link transitively to other classes using filter. An example is searching for all items where a certain user has added a message in the last week: db.issue.filter (None, {'messages.author' : '42', 'messages.date' : '.-1w;'}) or more readable (but not exactly semantically equivalent, if you're searching for multiple users in this way it will fail, because string searches are ANDed): {'messages.author.username':'ralf', ... We can even extend this further, look for all items that were changed by users belonging to a certain department (having the same supervisor -- a property that is not in the user class in standard roundup) in the last week, the filterspec would be: {'messages.author.supervisor' : '42', 'messages.date' : '.-1w;'} If anybody wants to suggest another name instead of transitive searching, you're welcome. I've implemented a generic method for this in hyperdb.py -- the backend now implements _filter in this case. With the generic method, anydbm and metakit should work (anydbm is tested, metakit breaks for other reasons). A backend may chose to implement the real transitive filter itself. This was done for rdbms_common.py. It now has an implementation of filter that supports transitive searching by creating one big join in the generated SQL query. I've added several new regression tests to test for the new features. All the tests (not just the new ones) run through on python2.3 and python2.4 with postgres, mysql, sqlite, anydbm -- but metakit was already broken when I started. I've generated a tag before commit called 'rsc_before_transitive_search' and will create the 'after' tag after this commit, so you can merge out my changes if you don't like them -- if you like them I can remove the tags. .-- Ralf
author Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
date Sat, 08 Jul 2006 18:28:18 +0000
parents a292054b4393
children 53987aa153d2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
1 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
4 # under the same terms as Python, so long as this copyright message and
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
5 # disclaimer are retained in their original form.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
6 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
11 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
17 #
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
18 #$Id: rdbms_common.py,v 1.172 2006-07-08 18:28:18 schlatterbeck Exp $
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
19 ''' Relational database (SQL) backend common code.
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
20
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
21 Basics:
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
22
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
23 - map roundup classes to relational tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
24 - automatically detect schema changes and modify the table schemas
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
25 appropriately (we store the "database version" of the schema in the
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
26 database itself as the only row of the "schema" table)
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
27 - multilinks (which represent a many-to-many relationship) are handled through
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
28 intermediate tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
29 - journals are stored adjunct to the per-class tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
30 - table names and columns have "_" prepended so the names can't clash with
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
31 restricted names (like "order")
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
32 - retirement is determined by the __retired__ column being true
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
33
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
34 Database-specific changes may generally be pushed out to the overridable
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
35 sql_* methods, since everything else should be fairly generic. There's
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
36 probably a bit of work to be done if a database is used that actually
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
37 honors column typing, since the initial databases don't (sqlite stores
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
38 everything as a string.)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
39
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
40 The schema of the hyperdb being mapped to the database is stored in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
41 database itself as a repr()'ed dictionary of information about each Class
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
42 that maps to a table. If that information differs from the hyperdb schema,
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
43 then we update it. We also store in the schema dict a version which
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
44 allows us to upgrade the database schema when necessary. See upgrade_db().
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
45 '''
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
46 __docformat__ = 'restructuredtext'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 # standard python modules
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
49 import sys, os, time, re, errno, weakref, copy, logging
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 # roundup modules
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
52 from roundup import hyperdb, date, password, roundupdb, security, support
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 from roundup.hyperdb import String, Password, Date, Interval, Link, \
1417
472c21af7f69 fixed error in indexargs_url (thanks Patrick Ohly)
Richard Jones <richard@users.sourceforge.net>
parents: 1415
diff changeset
54 Multilink, DatabaseError, Boolean, Number, Node
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
55 from roundup.backends import locking
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 # support
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 from blobfiles import FileStorage
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
59 try:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
60 from indexer_xapian import Indexer
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
61 except ImportError:
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
62 from indexer_rdbms import Indexer
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
63 from sessions_rdbms import Sessions, OneTimeKeys
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
64 from roundup.date import Range
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
66 # number of rows to keep in memory
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
67 ROW_CACHE_SIZE = 100
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
68
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
69 # dummy value meaning "argument not passed"
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
70 _marker = []
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
71
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
72 def _num_cvt(num):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
73 num = str(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
74 try:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
75 return int(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
76 except:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
77 return float(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
78
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
79 def _bool_cvt(value):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
80 if value in ('TRUE', 'FALSE'):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
81 return {'TRUE': 1, 'FALSE': 0}[value]
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
82 # assume it's a number returned from the db API
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
83 return int(value)
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
84
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
85 def connection_dict(config, dbnamestr=None):
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
86 ''' Used by Postgresql and MySQL to detemine the keyword args for
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
87 opening the database connection.'''
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
88 d = { }
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
89 if dbnamestr:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
90 d[dbnamestr] = config.RDBMS_NAME
3099
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
91 for name in ('host', 'port', 'password', 'user', 'read_default_group',
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
92 'read_default_file'):
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
93 cvar = 'RDBMS_'+name.upper()
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
94 if config[cvar] is not None:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
95 d[name] = config[cvar]
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
96 return d
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
97
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
99 ''' Wrapper around an SQL database that presents a hyperdb interface.
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
100
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
101 - some functionality is specific to the actual SQL database, hence
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
102 the sql_* methods that are NotImplemented
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
103 - we keep a cache of the latest ROW_CACHE_SIZE row fetches.
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
104 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 def __init__(self, config, journaltag=None):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 ''' Open the database and load the schema from it.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 self.config, self.journaltag = config, journaltag
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 self.dir = config.DATABASE
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 self.classes = {}
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
111 self.indexer = Indexer(self)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 self.security = security.Security(self)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 # additional transaction support for external files and the like
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
117 # keep a cache of the N most recently retrieved rows of any kind
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
118 # (classname, nodeid) = row
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
119 self.cache = {}
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
120 self.cache_lru = []
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
121 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
122 'filtering': 0}
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
123
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
124 # database lock
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
125 self.lockfile = None
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
126
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 # open a connection to the database, creating the "conn" attribute
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
128 self.open_connection()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
130 def clearCache(self):
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
131 self.cache = {}
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
132 self.cache_lru = []
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
133
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
134 def getSessionManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
135 return Sessions(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
136
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
137 def getOTKManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
138 return OneTimeKeys(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
139
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
140 def open_connection(self):
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
141 ''' Open a connection to the database, creating it if necessary.
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
142
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
143 Must call self.load_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 raise NotImplemented
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
147 def sql(self, sql, args=None):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 ''' Execute the sql with the optional args.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
150 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
151 logging.getLogger('hyperdb').debug('SQL %r %r'%(sql, args))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
152 if args:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
153 self.cursor.execute(sql, args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 else:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
155 self.cursor.execute(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
157 def sql_fetchone(self):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 ''' Fetch a single row. If there's nothing to fetch, return None.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
160 return self.cursor.fetchone()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
161
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
162 def sql_fetchall(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
163 ''' Fetch all rows. If there's nothing to fetch, return [].
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
164 '''
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
165 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
166
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
167 def sql_stringquote(self, value):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
168 ''' Quote the string so it's safe to put in the 'sql quotes'
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
169 '''
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
170 return re.sub("'", "''", str(value))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
171
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
172 def init_dbschema(self):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
173 self.database_schema = {
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
174 'version': self.current_db_version,
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
175 'tables': {}
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
176 }
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
177
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
178 def load_dbschema(self):
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
179 ''' Load the schema definition that the database currently implements
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
180 '''
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
181 self.cursor.execute('select schema from schema')
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
182 schema = self.cursor.fetchone()
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
183 if schema:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
184 self.database_schema = eval(schema[0])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
185 else:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
186 self.database_schema = {}
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
187
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
188 def save_dbschema(self):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
189 ''' Save the schema definition that the database currently implements
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
191 s = repr(self.database_schema)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
192 self.sql('delete from schema')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
193 self.sql('insert into schema values (%s)', (s,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 def post_init(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 ''' Called once the schema initialisation has finished.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198 We should now confirm that the schema defined by our "classes"
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 attribute actually matches the schema in the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
200 '''
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
201 save = 0
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
202
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
203 # handle changes in the schema
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
204 tables = self.database_schema['tables']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 for classname, spec in self.classes.items():
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
206 if tables.has_key(classname):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
207 dbspec = tables[classname]
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
208 if self.update_class(spec, dbspec):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
209 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
210 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 self.create_class(spec)
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
213 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
214 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
216 for classname, spec in tables.items():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 if not self.classes.has_key(classname):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
218 self.drop_class(classname, tables[classname])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
219 del tables[classname]
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
220 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
222 # now upgrade the database for column type changes, new internal
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
223 # tables, etc.
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
224 save = save | self.upgrade_db()
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
225
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 # update the database version of the schema
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
227 if save:
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
228 self.save_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
229
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230 # reindex the db if necessary
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 if self.indexer.should_reindex():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232 self.reindex()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
233
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 # commit
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
235 self.sql_commit()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
237 # update this number when we need to make changes to the SQL structure
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
238 # of the backen database
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
239 current_db_version = 4
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
240 def upgrade_db(self):
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
241 ''' Update the SQL database to reflect changes in the backend code.
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
242
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
243 Return boolean whether we need to save the schema.
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
244 '''
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
245 version = self.database_schema.get('version', 1)
2081
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
246 if version == self.current_db_version:
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
247 # nothing to do
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
248 return 0
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
249
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
250 if version < 2:
2456
1cd69db95b23 fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents: 2429
diff changeset
251 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
252 logging.getLogger('hyperdb').info('upgrade to version 2')
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
253 # change the schema structure
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
254 self.database_schema = {'tables': self.database_schema}
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
255
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
256 # version 1 didn't have the actor column (note that in
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
257 # MySQL this will also transition the tables to typed columns)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
258 self.add_new_columns_v2()
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
259
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
260 # version 1 doesn't have the OTK, session and indexing in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
261 # database
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
262 self.create_version_2_tables()
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
263
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
264 if version < 3:
2456
1cd69db95b23 fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents: 2429
diff changeset
265 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
266 logging.getLogger('hyperdb').info('upgrade to version 3')
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
267 self.fix_version_2_tables()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
268
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
269 if version < 4:
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
270 self.fix_version_3_tables()
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
271
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
272 self.database_schema['version'] = self.current_db_version
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
273 return 1
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
274
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
275 def fix_version_3_tables(self):
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
276 # drop the shorter VARCHAR OTK column and add a new TEXT one
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
277 for name in ('otk', 'session'):
2755
e3cd28cec23d Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents: 2733
diff changeset
278 self.sql('DELETE FROM %ss'%name)
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
279 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
280 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
281
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
282 def fix_version_2_tables(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
283 '''Default (used by sqlite): NOOP'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
284 pass
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
285
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
286 def _convert_journal_tables(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
287 '''Get current journal table contents, drop the table and re-create'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
288 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
289 cols = ','.join('nodeid date tag action params'.split())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
290 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
291 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
292 sql = 'select %s from %s__journal order by date'%(cols,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
293 klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
294 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
295 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
296 self.drop_journal_table_indexes(klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
297 c.execute('drop table %s__journal'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
298
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
299 # re-create and re-populate
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
300 self.create_journal_table(klass)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
301 a = self.arg
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
302 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
303 klass.classname, cols, a, a, a, a, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
304 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
305 # no data conversion needed
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
306 self.cursor.execute(sql, row)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
307
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
308 def _convert_string_properties(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
309 '''Get current Class tables that contain String properties, and
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
310 convert the VARCHAR columns to TEXT'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
311 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
312 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
313 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
314 cols, mls = self.determine_columns(klass.properties.items())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
315 scols = ','.join([i[0] for i in cols])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
316 sql = 'select id,%s from _%s'%(scols, klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
317 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
318 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
319 self.drop_class_table_indexes(klass.classname, klass.getkey())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
320 c.execute('drop table _%s'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
321
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
322 # re-create and re-populate
2424
74474ec41050 argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents: 2416
diff changeset
323 self.create_class_table(klass, create_sequence=0)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
324 a = ','.join([self.arg for i in range(len(cols)+1)])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
325 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
326 scols, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
327 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
328 l = []
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
329 for entry in row:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
330 # mysql will already be a string - psql needs "help"
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
331 if entry is not None and not isinstance(entry, type('')):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
332 entry = str(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
333 l.append(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
334 self.cursor.execute(sql, l)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
335
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
336 def refresh_database(self):
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
337 self.post_init()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
338
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
339
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
340 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
341 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
342 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
343 else:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
344 classes = self.classes.values()
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
345 for klass in classes:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
346 if show_progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
347 for nodeid in support.Progress('Reindex %s'%klass.classname,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
348 klass.list()):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
349 klass.index(nodeid)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
350 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
351 for nodeid in klass.list():
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
352 klass.index(nodeid)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
353 self.indexer.save_index()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
354
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
355 hyperdb_to_sql_datatypes = {
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
356 hyperdb.String : 'TEXT',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
357 hyperdb.Date : 'TIMESTAMP',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
358 hyperdb.Link : 'INTEGER',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
359 hyperdb.Interval : 'VARCHAR(255)',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
360 hyperdb.Password : 'VARCHAR(255)',
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
361 hyperdb.Boolean : 'BOOLEAN',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
362 hyperdb.Number : 'REAL',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
363 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
364 def determine_columns(self, properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 ''' Figure the column names and multilink properties from the spec
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
366
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367 "properties" is a list of (name, prop) where prop may be an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
368 instance of a hyperdb "type" _or_ a string repr of that type.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369 '''
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
370 cols = [
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
371 ('_actor', self.hyperdb_to_sql_datatypes[hyperdb.Link]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
372 ('_activity', self.hyperdb_to_sql_datatypes[hyperdb.Date]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
373 ('_creator', self.hyperdb_to_sql_datatypes[hyperdb.Link]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
374 ('_creation', self.hyperdb_to_sql_datatypes[hyperdb.Date]),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
375 ]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
376 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
377 # add the multilinks separately
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378 for col, prop in properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 mls.append(col)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
381 continue
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
382
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
383 if isinstance(prop, type('')):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
384 raise ValueError, "string property spec!"
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
385 #and prop.find('Multilink') != -1:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
386 #mls.append(col)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
387
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
388 datatype = self.hyperdb_to_sql_datatypes[prop.__class__]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
389 cols.append(('_'+col, datatype))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
390
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
391 # Intervals stored as two columns
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
392 if isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
393 cols.append(('__'+col+'_int__', 'BIGINT'))
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
394
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396 return cols, mls
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
398 def update_class(self, spec, old_spec, force=0):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
399 ''' Determine the differences between the current spec and the
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
400 database version of the spec, and update where necessary.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
401
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
402 If 'force' is true, update the database anyway.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
404 new_has = spec.properties.has_key
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
405 new_spec = spec.schema()
1515
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
406 new_spec[1].sort()
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
407 old_spec[1].sort()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
408 if not force and new_spec == old_spec:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
409 # no changes
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
410 return 0
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
411
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
412 logger = logging.getLogger('hyperdb')
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
413 logger.info('update_class %s'%spec.classname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
414
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
415 logger.debug('old_spec %r'%(old_spec,))
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
416 logger.debug('new_spec %r'%(new_spec,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
417
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
418 # detect key prop change for potential index change
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
419 keyprop_changes = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
420 if new_spec[0] != old_spec[0]:
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
421 if old_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
422 keyprop_changes['remove'] = old_spec[0]
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
423 if new_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
424 keyprop_changes['add'] = new_spec[0]
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
425
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
426 # detect multilinks that have been removed, and drop their table
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
427 old_has = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
428 for name, prop in old_spec[1]:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
429 old_has[name] = 1
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
430 if new_has(name):
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
431 continue
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
432
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
433 if prop.find('Multilink to') != -1:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
434 # first drop indexes.
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
435 self.drop_multilink_table_indexes(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
436
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
437 # now the multilink table itself
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
438 sql = 'drop table %s_%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
439 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
440 # if this is the key prop, drop the index first
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
441 if old_spec[0] == prop:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
442 self.drop_class_table_key_index(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
443 del keyprop_changes['remove']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
444
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
445 # drop the column
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
446 sql = 'alter table _%s drop column _%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
447
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
448 self.sql(sql)
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
449 old_has = old_has.has_key
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
450
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
451 # if we didn't remove the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
452 # changed, we still need to remove the old index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
453 if keyprop_changes.has_key('remove'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
454 self.drop_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
455 keyprop_changes['remove'])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
456
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
457 # add new columns
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
458 for propname, prop in new_spec[1]:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
459 if old_has(propname):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
460 continue
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
461 prop = spec.properties[propname]
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
462 if isinstance(prop, Multilink):
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
463 self.create_multilink_table(spec, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
464 else:
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
465 # add the column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
466 coltype = self.hyperdb_to_sql_datatypes[prop.__class__]
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
467 sql = 'alter table _%s add column _%s %s'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
468 spec.classname, propname, coltype)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
469 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
470
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
471 # extra Interval column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
472 if isinstance(prop, Interval):
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
473 sql = 'alter table _%s add column __%s_int__ BIGINT'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
474 spec.classname, propname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
475 self.sql(sql)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
476
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
477 # if the new column is a key prop, we need an index!
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
478 if new_spec[0] == propname:
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
479 self.create_class_table_key_index(spec.classname, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
480 del keyprop_changes['add']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
481
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
482 # if we didn't add the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
483 # changed, we still need to add the new index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
484 if keyprop_changes.has_key('add'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
485 self.create_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
486 keyprop_changes['add'])
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
487
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
488 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
489
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
490 def determine_all_columns(self, spec):
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
491 """Figure out the columns from the spec and also add internal columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
492
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
493 """
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
494 cols, mls = self.determine_columns(spec.properties.items())
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
495
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
496 # add on our special columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
497 cols.append(('id', 'INTEGER PRIMARY KEY'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
498 cols.append(('__retired__', 'INTEGER DEFAULT 0'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
499 return cols, mls
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
500
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
501 def create_class_table(self, spec):
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
502 '''Create the class table for the given Class "spec". Creates the
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
503 indexes too.'''
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
504 cols, mls = self.determine_all_columns(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
505
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
506 # create the base table
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
507 scols = ','.join(['%s %s'%x for x in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
508 sql = 'create table _%s (%s)'%(spec.classname, scols)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
509 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
510
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
511 self.create_class_table_indexes(spec)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
512
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
513 return cols, mls
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
514
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
515 def create_class_table_indexes(self, spec):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
516 ''' create the class table for the given spec
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
517 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
518 # create __retired__ index
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
519 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%(
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
520 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
521 self.sql(index_sql2)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
522
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
523 # create index for key property
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
524 if spec.key:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
525 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%(
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
526 spec.classname, spec.key,
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
527 spec.classname, spec.key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
528 self.sql(index_sql3)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
529
2228
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
530 # TODO: create indexes on (selected?) Link property columns, as
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
531 # they're more likely to be used for lookup
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
532
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
533 def drop_class_table_indexes(self, cn, key):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
534 # drop the old table indexes first
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
535 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
536 if key:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
537 l.append('_%s_%s_idx'%(cn, key))
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
538
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
539 table_name = '_%s'%cn
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
540 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
541 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
542 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
543 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
544 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
545
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
546 def create_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
547 ''' create the class table for the given spec
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
548 '''
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
549 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
550 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
551
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
552 def drop_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
553 table_name = '_%s'%cn
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
554 index_name = '_%s_%s_idx'%(cn, key)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
555 if not self.sql_index_exists(table_name, index_name):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
556 return
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
557 sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
558 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
559
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
560 def create_journal_table(self, spec):
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
561 ''' create the journal table for a class given the spec and
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
562 already-determined cols
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
563 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
564 # journal table
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
565 cols = ','.join(['%s varchar'%x
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
566 for x in 'nodeid date tag action params'.split()])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
567 sql = '''create table %s__journal (
3224
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
568 nodeid integer, date %s, tag varchar(255),
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
569 action varchar(255), params text)''' % (spec.classname,
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
570 self.hyperdb_to_sql_datatypes[hyperdb.Date])
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
571 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
572 self.create_journal_table_indexes(spec)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
573
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
574 def create_journal_table_indexes(self, spec):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
575 # index on nodeid
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
576 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%(
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
577 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
578 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
579
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
580 def drop_journal_table_indexes(self, classname):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
581 index_name = '%s_journ_idx'%classname
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
582 if not self.sql_index_exists('%s__journal'%classname, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
583 return
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
584 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
585 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
586
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
587 def create_multilink_table(self, spec, ml):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
588 ''' Create a multilink table for the "ml" property of the class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
589 given by the spec
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
590 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
591 # create the table
2828
3a2f138fcb28 use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2755
diff changeset
592 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%(
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
593 spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
594 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
595 self.create_multilink_table_indexes(spec, ml)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
596
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
597 def create_multilink_table_indexes(self, spec, ml):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
598 # create index on linkid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
599 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
600 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
601 self.sql(index_sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
602
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
603 # create index on nodeid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
604 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
605 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
606 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
607
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
608 def drop_multilink_table_indexes(self, classname, ml):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
609 l = [
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
610 '%s_%s_l_idx'%(classname, ml),
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
611 '%s_%s_n_idx'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
612 ]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
613 table_name = '%s_%s'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
614 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
615 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
616 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
617 index_sql = 'drop index %s'%index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
618 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
619
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
620 def create_class(self, spec):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
621 ''' Create a database table according to the given spec.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
622 '''
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
623 cols, mls = self.create_class_table(spec)
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
624 self.create_journal_table(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
625
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
626 # now create the multilink tables
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
627 for ml in mls:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
628 self.create_multilink_table(spec, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
629
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
630 def drop_class(self, cn, spec):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
631 ''' Drop the given table from the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
632
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
633 Drop the journal and multilink tables too.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
634 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
635 properties = spec[1]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
636 # figure the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
637 mls = []
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
638 for propanme, prop in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
639 if isinstance(prop, Multilink):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
640 mls.append(propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
641
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
642 # drop class table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
643 self.drop_class_table_indexes(cn, spec[0])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
644
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
645 self.drop_class_table(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
646
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
647 # drop journal table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
648 self.drop_journal_table_indexes(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
649 sql = 'drop table %s__journal'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
650 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
651
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
652 for ml in mls:
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
653 # drop multilink table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
654 self.drop_multilink_table_indexes(cn, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
655 sql = 'drop table %s_%s'%(spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
656 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
657
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
658 def drop_class_table(self, cn):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
659 sql = 'drop table _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
660 self.sql(sql)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
661
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
662 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
663 # Classes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
664 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
665 def __getattr__(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
666 ''' A convenient way of calling self.getclass(classname).
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
667 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
668 if self.classes.has_key(classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
669 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
670 raise AttributeError, classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
671
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
672 def addclass(self, cl):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
673 ''' Add a Class to the hyperdatabase.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
674 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
675 cn = cl.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
676 if self.classes.has_key(cn):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
677 raise ValueError, cn
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
678 self.classes[cn] = cl
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
679
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
680 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
681 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
682 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
683 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
684 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
685 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
686 description="User is allowed to access "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
687
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
688 def getclasses(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
689 ''' Return a list of the names of all existing classes.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
690 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
691 l = self.classes.keys()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
692 l.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
693 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
694
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
695 def getclass(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
696 '''Get the Class object representing a particular class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
697
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
698 If 'classname' is not a valid class name, a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
699 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
700 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
701 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
702 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
703 raise KeyError, 'There is no class called "%s"'%classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
704
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
705 def clear(self):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
706 '''Delete all database contents.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
707
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
708 Note: I don't commit here, which is different behaviour to the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
709 "nuke from orbit" behaviour in the dbs.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
710 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
711 logging.getLogger('hyperdb').info('clear')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
712 for cn in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
713 sql = 'delete from _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
714 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
715
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
716 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
717 # Nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
718 #
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
719
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
720 hyperdb_to_sql_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
721 hyperdb.String : str,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
722 # fractional seconds by default
3612
afda59d5d546 prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
723 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
724 hyperdb.Link : int,
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
725 hyperdb.Interval : str,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
726 hyperdb.Password : str,
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
727 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
728 hyperdb.Number : lambda x: x,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
729 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
730 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
731 def addnode(self, classname, nodeid, node):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
732 ''' Add the specified node to its class's db.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
733 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
734 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
735 logging.getLogger('hyperdb').debug('addnode %s%s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
736 nodeid, node))
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
737
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
738 # determine the column definitions and multilink tables
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
739 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
740 cols, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
741
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
742 # we'll be supplied these props if we're doing an import
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
743 values = node.copy()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
744 if not values.has_key('creator'):
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
745 # add in the "calculated" properties (dupe so we don't affect
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
746 # calling code's node assumptions)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
747 values['creation'] = values['activity'] = date.Date()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
748 values['actor'] = values['creator'] = self.getuid()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
749
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
750 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
751 props = cl.getprops(protected=1)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
752 del props['id']
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
753
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
754 # default the non-multilink columns
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
755 for col, prop in props.items():
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
756 if not values.has_key(col):
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
757 if isinstance(prop, Multilink):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
758 values[col] = []
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
759 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
760 values[col] = None
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
761
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
762 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
763 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
764 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
765 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
766 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
767
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
768 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
769 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
770 for col,dt in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
771 # this is somewhat dodgy....
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
772 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
773 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
774 value = values[col[2:-6]]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
775 # this is an Interval special "int" column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
776 if value is not None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
777 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
778 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
779 vals.append(value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
780 continue
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
781
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
782 prop = props[col[1:]]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
783 value = values[col[1:]]
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
784 if value is not None:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
785 value = self.hyperdb_to_sql_value[prop.__class__](value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
786 vals.append(value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
787 vals.append(nodeid)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
788 vals = tuple(vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
789
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
790 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
791 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
792 cols = ','.join([col for col,dt in cols]) + ',id'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
793
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
794 # perform the inserts
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
795 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
796 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
797
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
798 # insert the multilink rows
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
799 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
800 t = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
801 for entry in node[col]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
802 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
803 self.arg, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
804 self.sql(sql, (entry, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
805
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
806 def setnode(self, classname, nodeid, values, multilink_changes={}):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
807 ''' Change the specified node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
808 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
809 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
810 logging.getLogger('hyperdb').debug('setnode %s%s %r'
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
811 % (classname, nodeid, values))
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
812
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
813 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
814 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
815 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
816 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
817 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
818
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
819 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
820 props = cl.getprops()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
821
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
822 cols = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
823 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
824 # add the multilinks separately
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
825 for col in values.keys():
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
826 prop = props[col]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
827 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
828 mls.append(col)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
829 elif isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
830 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
831 cols.append(col)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
832 # extra leading '_' added by code below
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
833 cols.append('_' +col + '_int__')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
834 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
835 cols.append(col)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
836 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
837
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
838 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
839 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
840 for col in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
841 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
842 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
843 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
844 col = col[1:-6]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
845 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
846 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
847 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
848 vals.append(None)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
849 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
850 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
851 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
852 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
853 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
854 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
855 e = None
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
856 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
857 e = self.hyperdb_to_sql_value[prop.__class__](value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
858 vals.append(e)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
859
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
860 vals.append(int(nodeid))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
861 vals = tuple(vals)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
862
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
863 # if there's any updates to regular columns, do them
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
864 if cols:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
865 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
866 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols])
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
867 cols = ','.join(cols)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
868
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
869 # perform the update
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
870 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
871 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
872
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
873 # we're probably coming from an import, not a change
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
874 if not multilink_changes:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
875 for name in mls:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
876 prop = props[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
877 value = values[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
878
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
879 t = '%s_%s'%(classname, name)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
880
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
881 # clear out previous values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
882 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
883 self.sql('delete from %s where nodeid=%s'%(t, self.arg),
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
884 (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
885
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
886 # insert the values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
887 for entry in values[name]:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
888 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
889 self.arg, self.arg)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
890 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
891 self.sql(sql, (entry, nodeid))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
892
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
893 # we have multilink changes to apply
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
894 for col, (add, remove) in multilink_changes.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
895 tn = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
896 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
897 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
898 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
899 for addid in add:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
900 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
901 self.sql(sql, (int(nodeid), int(addid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
902 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
903 sql = 'delete from %s where nodeid=%s and linkid=%s'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
904 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
905 for removeid in remove:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
906 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
907 self.sql(sql, (int(nodeid), int(removeid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
908
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
909 sql_to_hyperdb_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
910 hyperdb.String : str,
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
911 hyperdb.Date : lambda x:date.Date(str(x).replace(' ', '.')),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
912 # hyperdb.Link : int, # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
913 hyperdb.Link : str,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
914 hyperdb.Interval : date.Interval,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
915 hyperdb.Password : lambda x: password.Password(encrypted=x),
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
916 hyperdb.Boolean : _bool_cvt,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
917 hyperdb.Number : _num_cvt,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
918 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
919 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
920 def getnode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
921 ''' Get a node from the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
922 '''
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
923 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
924 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
925 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
926 # push us back to the top of the LRU
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
927 self.cache_lru.remove(key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
928 self.cache_lru.insert(0, key)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
929 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
930 self.stats['cache_hits'] += 1
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
931 # return the cached information
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
932 return self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
933
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
934 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
935 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
936 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
937
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
938 # figure the columns we're fetching
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
939 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
940 cols, mls = self.determine_columns(cl.properties.items())
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
941 scols = ','.join([col for col,dt in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
942
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
943 # perform the basic property fetch
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
944 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
945 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
946
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
947 values = self.sql_fetchone()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
948 if values is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
949 raise IndexError, 'no such %s node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
950
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
951 # make up the node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
952 node = {}
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
953 props = cl.getprops(protected=1)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
954 for col in range(len(cols)):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
955 name = cols[col][0][1:]
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
956 if name.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
957 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
958 # ignore the special Interval-as-seconds column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
959 continue
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
960 value = values[col]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
961 if value is not None:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
962 value = self.sql_to_hyperdb_value[props[name].__class__](value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
963 node[name] = value
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
964
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
965
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
966 # now the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
967 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
968 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
969 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
970 self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
971 self.cursor.execute(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
972 # extract the first column from the result
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
973 # XXX numeric ids
2887
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
974 items = [int(x[0]) for x in self.cursor.fetchall()]
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
975 items.sort ()
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
976 node[col] = [str(x) for x in items]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
977
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
978 # save off in the cache
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
979 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
980 self.cache[key] = node
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
981 # update the LRU
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
982 self.cache_lru.insert(0, key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
983 if len(self.cache_lru) > ROW_CACHE_SIZE:
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
984 del self.cache[self.cache_lru.pop()]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
985
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
986 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
987 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
988
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
989 return node
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
990
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
991 def destroynode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
992 '''Remove a node from the database. Called exclusively by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
993 destroy() method on Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
994 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
995 logging.getLogger('hyperdb').info('destroynode %s%s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
996
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
997 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
998 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
999 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1000
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1001 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1002 if self.cache.has_key((classname, nodeid)):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1003 del self.cache[(classname, nodeid)]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1004
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1005 # see if there's any obvious commit actions that we should get rid of
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1006 for entry in self.transactions[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1007 if entry[1][:2] == (classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1008 self.transactions.remove(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1009
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1010 # now do the SQL
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1011 sql = 'delete from _%s where id=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1012 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1013
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1014 # remove from multilnks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1015 cl = self.getclass(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1016 x, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1017 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1018 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1019 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
1020 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1021
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1022 # remove journal entries
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1023 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1024 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1025
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1026 def hasnode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1027 ''' Determine if the database has a given node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1028 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1029 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1030 self.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1031 return int(self.cursor.fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1032
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1033 def countnodes(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1034 ''' Count the number of nodes that exist for a particular Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1035 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1036 sql = 'select count(*) from _%s'%classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1037 self.sql(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1038 return self.cursor.fetchone()[0]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1039
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1040 def addjournal(self, classname, nodeid, action, params, creator=None,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1041 creation=None):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1042 ''' Journal the Action
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1043 'action' may be:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1044
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1045 'create' or 'set' -- 'params' is a dictionary of property values
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1046 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1047 'retire' -- 'params' is None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1048 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1049 # handle supply of the special journalling parameters (usually
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1050 # supplied on importing an existing database)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1051 if creator:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1052 journaltag = creator
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1053 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1054 journaltag = self.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1055 if creation:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1056 journaldate = creation
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1057 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1058 journaldate = date.Date()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1059
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1060 # create the journal entry
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1061 cols = 'nodeid,date,tag,action,params'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1062
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1063 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1064 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1065 nodeid, journaldate, journaltag, action, params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1066
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1067 # make the journalled data marshallable
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1068 if isinstance(params, type({})):
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1069 self._journal_marshal(params, classname)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1070
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1071 params = repr(params)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1072
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1073 dc = self.hyperdb_to_sql_value[hyperdb.Date]
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1074 journaldate = dc(journaldate)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1075
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1076 self.save_journal(classname, cols, nodeid, journaldate,
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1077 journaltag, action, params)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1078
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1079 def setjournal(self, classname, nodeid, journal):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1080 '''Set the journal to the "journal" list.'''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1081 # clear out any existing entries
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1082 self.sql('delete from %s__journal where nodeid=%s'%(classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1083 self.arg), (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1084
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1085 # create the journal entry
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1086 cols = 'nodeid,date,tag,action,params'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1087
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1088 dc = self.hyperdb_to_sql_value[hyperdb.Date]
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1089 for nodeid, journaldate, journaltag, action, params in journal:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1090 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1091 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1092 classname, nodeid, journaldate, journaltag, action,
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1093 params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1094
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1095 # make the journalled data marshallable
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1096 if isinstance(params, type({})):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1097 self._journal_marshal(params, classname)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1098 params = repr(params)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1099
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1100 self.save_journal(classname, cols, nodeid, dc(journaldate),
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1101 journaltag, action, params)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1102
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1103 def _journal_marshal(self, params, classname):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1104 '''Convert the journal params values into safely repr'able and
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1105 eval'able values.'''
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1106 properties = self.getclass(classname).getprops()
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1107 for param, value in params.items():
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1108 if not value:
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1109 continue
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1110 property = properties[param]
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1111 cvt = self.hyperdb_to_sql_value[property.__class__]
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1112 if isinstance(property, Password):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1113 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1114 elif isinstance(property, Date):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1115 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1116 elif isinstance(property, Interval):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1117 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1118 elif isinstance(property, Boolean):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1119 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1120
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1121 def getjournal(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1122 ''' get the journal for id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1123 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1124 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1125 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1126 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1127
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1128 cols = ','.join('nodeid date tag action params'.split())
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1129 journal = self.load_journal(classname, cols, nodeid)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1130
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1131 # now unmarshal the data
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1132 dc = self.sql_to_hyperdb_value[hyperdb.Date]
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1133 res = []
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1134 properties = self.getclass(classname).getprops()
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1135 for nodeid, date_stamp, user, action, params in journal:
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1136 params = eval(params)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1137 if isinstance(params, type({})):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1138 for param, value in params.items():
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1139 if not value:
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1140 continue
2731
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1141 property = properties.get(param, None)
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1142 if property is None:
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1143 # deleted property
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1144 continue
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1145 cvt = self.sql_to_hyperdb_value[property.__class__]
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1146 if isinstance(property, Password):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1147 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1148 elif isinstance(property, Date):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1149 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1150 elif isinstance(property, Interval):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1151 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1152 elif isinstance(property, Boolean):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1153 params[param] = cvt(value)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1154 # XXX numeric ids
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1155 res.append((str(nodeid), dc(date_stamp), user, action, params))
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1156 return res
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1157
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1158 def save_journal(self, classname, cols, nodeid, journaldate,
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1159 journaltag, action, params):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1160 ''' Save the journal entry to the database
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1161 '''
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1162 entry = (nodeid, journaldate, journaltag, action, params)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1163
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1164 # do the insert
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1165 a = self.arg
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1166 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1167 classname, cols, a, a, a, a, a)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1168 self.sql(sql, entry)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1169
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1170 def load_journal(self, classname, cols, nodeid):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1171 ''' Load the journal from the database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1172 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1173 # now get the journal entries
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1174 sql = 'select %s from %s__journal where nodeid=%s order by date'%(
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1175 cols, classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1176 self.sql(sql, (nodeid,))
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1177 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1178
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1179 def pack(self, pack_before):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1180 ''' Delete all journal entries except "create" before 'pack_before'.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1181 '''
2416
Richard Jones <richard@users.sourceforge.net>
parents: 2413
diff changeset
1182 date_stamp = self.hyperdb_to_sql_value[Date](pack_before)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1183
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1184 # do the delete
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1185 for classname in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1186 sql = "delete from %s__journal where date<%s and "\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1187 "action<>'create'"%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1188 self.sql(sql, (date_stamp,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1189
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1190 def sql_commit(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1191 ''' Actually commit to the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1192 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1193 logging.getLogger('hyperdb').info('commit')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1194 self.conn.commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1195
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1196 # open a new cursor for subsequent work
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1197 self.cursor = self.conn.cursor()
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1198
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1199 def commit(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1200 ''' Commit the current transactions.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1201
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1202 Save all data changed since the database was opened or since the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1203 last commit() or rollback().
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1204 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1205 # commit the database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1206 self.sql_commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1207
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1208 # now, do all the other transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1209 for method, args in self.transactions:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1210 method(*args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1211
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1212 # save the indexer
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1213 self.indexer.save_index()
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1214
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1215 # clear out the transactions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1216 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1217
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1218 def sql_rollback(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1219 self.conn.rollback()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1220
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1221 def rollback(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1222 ''' Reverse all actions from the current transaction.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1223
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1224 Undo all the changes made since the database was opened or the last
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1225 commit() or rollback() was performed.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1226 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1227 logging.getLogger('hyperdb').info('rollback')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1228
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1229 self.sql_rollback()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1230
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1231 # roll back "other" transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1232 for method, args in self.transactions:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1233 # delete temporary files
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1234 if method == self.doStoreFile:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1235 self.rollbackStoreFile(*args)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1236 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1237
1492
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1238 # clear the cache
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1239 self.clearCache()
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1240
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1241 def sql_close(self):
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1242 logging.getLogger('hyperdb').info('close')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1243 self.conn.close()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1244
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1245 def close(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1246 ''' Close off the connection.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1247 '''
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
1248 self.indexer.close()
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1249 self.sql_close()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1250
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1251 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1252 # The base Class class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1253 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1254 class Class(hyperdb.Class):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1255 ''' The handle to a particular class of nodes in a hyperdatabase.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1256
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1257 All methods except __repr__ and getnode must be implemented by a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1258 concrete backend Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1259 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1260
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1261 def schema(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1262 ''' A dumpable version of the schema that we can store in the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1263 database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1264 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1265 return (self.key, [(x, repr(y)) for x,y in self.properties.items()])
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1266
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1267 def enableJournalling(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1268 '''Turn journalling on for this class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1269 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1270 self.do_journal = 1
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1271
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1272 def disableJournalling(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1273 '''Turn journalling off for this class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1274 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1275 self.do_journal = 0
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1276
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1277 # Editing nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1278 def create(self, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1279 ''' Create a new node of this class and return its id.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1280
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1281 The keyword arguments in 'propvalues' map property names to values.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1282
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1283 The values of arguments must be acceptable for the types of their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1284 corresponding properties or a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1285
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1286 If this class has a key property, it must be present and its value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1287 must not collide with other key strings or a ValueError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1288
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1289 Any other properties on this class that are missing from the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1290 'propvalues' dictionary are set to None.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1291
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1292 If an id in a link or multilink property does not refer to a valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1293 node, an IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1294 '''
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1295 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1296 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1297 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1298 return newid
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1299
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1300 def create_inner(self, **propvalues):
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1301 ''' Called by create, in-between the audit and react calls.
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1302 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1303 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1304 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1305
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1306 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1307 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1308
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1309 if propvalues.has_key('creator') or propvalues.has_key('actor') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1310 propvalues.has_key('creation') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1311 raise KeyError, '"creator", "actor", "creation" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1312 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1313
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1314 # new node's id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1315 newid = self.db.newid(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1316
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1317 # validate propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1318 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1319 for key, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1320 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1321 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1322 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1323 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1324 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1325 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1326 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1327
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1328 # try to handle this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1329 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1330 prop = self.properties[key]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1331 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1332 raise KeyError, '"%s" has no property "%s"'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1333 key)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1334
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1335 if value is not None and isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1336 if type(value) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1337 raise ValueError, 'link value must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1338 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1339 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1340 if not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1341 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1342 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1343 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1344 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1345 key, value, link_class)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1346 elif not self.db.getclass(link_class).hasnode(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1347 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1348
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1349 # save off the value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1350 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1351
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1352 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1353 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1354 self.db.addjournal(link_class, value, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1355 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1356
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1357 elif isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1358 if type(value) != type([]):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1359 raise TypeError, 'new property "%s" not a list of ids'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1360
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1361 # clean up and validate the list of links
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1362 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1363 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1364 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1365 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1366 raise ValueError, '"%s" multilink value (%r) '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1367 'must contain Strings'%(key, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1368 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1369 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1370 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1371 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1372 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1373 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1374 key, entry, self.properties[key].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1375 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1376 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1377 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1378
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1379 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1380 for nodeid in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1381 if not self.db.getclass(link_class).hasnode(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1382 raise IndexError, '%s has no node %s'%(link_class,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1383 nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1384 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1385 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1386 self.db.addjournal(link_class, nodeid, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1387 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1388
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1389 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1390 if type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1391 raise TypeError, 'new property "%s" not a string'%key
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1392 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1393 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1394 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1395
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1396 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1397 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1398 raise TypeError, 'new property "%s" not a Password'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1399
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1400 elif isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1401 if value is not None and not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1402 raise TypeError, 'new property "%s" not a Date'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1403
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1404 elif isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1405 if value is not None and not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1406 raise TypeError, 'new property "%s" not an Interval'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1407
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1408 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1409 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1410 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1411 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1412 raise TypeError, 'new property "%s" not numeric'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1413
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1414 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1415 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1416 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1417 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1418 raise TypeError, 'new property "%s" not boolean'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1419
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1420 # make sure there's data where there needs to be
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1421 for key, prop in self.properties.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1422 if propvalues.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1423 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1424 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1425 raise ValueError, 'key property "%s" is required'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1426 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1427 propvalues[key] = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1428 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1429 propvalues[key] = None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1430
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1431 # done
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1432 self.db.addnode(self.classname, newid, propvalues)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1433 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1434 self.db.addjournal(self.classname, newid, ''"create", {})
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1435
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1436 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1437 return str(newid)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1438
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1439 def get(self, nodeid, propname, default=_marker, cache=1):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1440 '''Get the value of a property on an existing node of this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1441
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1442 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1443 IndexError is raised. 'propname' must be the name of a property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1444 of this class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1445
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1446 'cache' exists for backwards compatibility, and is not used.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1447 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1448 if propname == 'id':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1449 return nodeid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1450
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1451 # get the node's dict
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1452 d = self.db.getnode(self.classname, nodeid)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1453
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1454 if propname == 'creation':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1455 if d.has_key('creation'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1456 return d['creation']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1457 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1458 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1459 if propname == 'activity':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1460 if d.has_key('activity'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1461 return d['activity']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1462 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1463 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1464 if propname == 'creator':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1465 if d.has_key('creator'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1466 return d['creator']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1467 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1468 return self.db.getuid()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1469 if propname == 'actor':
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1470 if d.has_key('actor'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1471 return d['actor']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1472 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1473 return self.db.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1474
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1475 # get the property (raises KeyErorr if invalid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1476 prop = self.properties[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1477
2690
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1478 # XXX may it be that propname is valid property name
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1479 # (above error is not raised) and not d.has_key(propname)???
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1480 if (not d.has_key(propname)) or (d[propname] is None):
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
1481 if default is _marker:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1482 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1483 return []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1484 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1485 return None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1486 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1487 return default
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1488
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1489 # don't pass our list to other code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1490 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1491 return d[propname][:]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1492
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1493 return d[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1494
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1495 def set(self, nodeid, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1496 '''Modify a property on an existing node of this class.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1497
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1498 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1499 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1500
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1501 Each key in 'propvalues' must be the name of a property of this
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1502 class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1503
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1504 All values in 'propvalues' must be acceptable types for their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1505 corresponding properties or a TypeError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1506
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1507 If the value of the key property is set, it must not collide with
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1508 other key strings or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1509
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1510 If the value of a Link or Multilink property contains an invalid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1511 node id, a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1512 '''
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1513 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1514 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1515 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1516 self.fireReactors('set', nodeid, oldvalues)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1517 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1518
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1519 def set_inner(self, nodeid, **propvalues):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1520 ''' Called by set, in-between the audit and react calls.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1521 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1522 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1523 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1524
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1525 if propvalues.has_key('creation') or propvalues.has_key('creator') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1526 propvalues.has_key('actor') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1527 raise KeyError, '"creation", "creator", "actor" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1528 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1529
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1530 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1531 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1532
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1533 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1534 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1535
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1536 node = self.db.getnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1537 if self.is_retired(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1538 raise IndexError, 'Requested item is retired'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1539 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1540
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1541 # make a copy of the values dictionary - we'll modify the contents
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1542 propvalues = propvalues.copy()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1543
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1544 # if the journal value is to be different, store it in here
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1545 journalvalues = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1546
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1547 # remember the add/remove stuff for multilinks, making it easier
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1548 # for the Database layer to do its stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1549 multilink_changes = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1550
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1551 for propname, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1552 # check to make sure we're not duplicating an existing key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1553 if propname == self.key and node[propname] != value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1554 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1555 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1556 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1557 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1558 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1559 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1560
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1561 # this will raise the KeyError if the property isn't valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1562 # ... we don't use getprops() here because we only care about
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1563 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1564 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1565 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1566 except KeyError:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1567 raise KeyError, '"%s" has no property named "%s"'%(
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1568 self.classname, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1569
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1570 # if the value's the same as the existing value, no sense in
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1571 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1572 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1573 if value == current:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1574 del propvalues[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1575 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1576 journalvalues[propname] = current
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1577
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1578 # do stuff based on the prop type
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1579 if isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1580 link_class = prop.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1581 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1582 if value is not None and not isinstance(value, type('')):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1583 raise ValueError, 'property "%s" link value be a string'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1584 propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1585 if isinstance(value, type('')) and not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1586 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1587 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1588 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1589 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1590 propname, value, prop.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1591
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1592 if (value is not None and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1593 not self.db.getclass(link_class).hasnode(value)):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1594 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1595
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1596 if self.do_journal and prop.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1597 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1598 if node[propname] is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1599 self.db.addjournal(link_class, node[propname],
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1600 ''"unlink", (self.classname, nodeid, propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1601
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1602 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1603 if value is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1604 self.db.addjournal(link_class, value, ''"link",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1605 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1606
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1607 elif isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1608 if type(value) != type([]):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1609 raise TypeError, 'new property "%s" not a list of'\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1610 ' ids'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1611 link_class = self.properties[propname].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1612 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1613 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1614 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1615 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1616 raise ValueError, 'new property "%s" link value ' \
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1617 'must be a string'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1618 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1619 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1620 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1621 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1622 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1623 propname, entry,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1624 self.properties[propname].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1625 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1626 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1627 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1628
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1629 # figure the journal entry for this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1630 add = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1631 remove = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1632
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1633 # handle removals
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1634 if node.has_key(propname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1635 l = node[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1636 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1637 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1638 for id in l[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1639 if id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1640 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1641 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1642 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1643 self.db.addjournal(link_class, id, 'unlink',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1644 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1645 l.remove(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1646 remove.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1647
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1648 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1649 for id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1650 if not self.db.getclass(link_class).hasnode(id):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1651 raise IndexError, '%s has no node %s'%(link_class, id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1652 if id in l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1653 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1654 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1655 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1656 self.db.addjournal(link_class, id, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1657 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1658 l.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1659 add.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1660
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1661 # figure the journal entry
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1662 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1663 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1664 l.append(('+', add))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1665 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1666 l.append(('-', remove))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1667 multilink_changes[propname] = (add, remove)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1668 if l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1669 journalvalues[propname] = tuple(l)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1670
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1671 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1672 if value is not None and type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1673 raise TypeError, 'new property "%s" not a string'%propname
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1674 if prop.indexme:
3399
3a0d4e4a0f34 merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
1675 if value is None: value = ''
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1676 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1677 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1678
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1679 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1680 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1681 raise TypeError, 'new property "%s" not a Password'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1682 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1683
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1684 elif value is not None and isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1685 if not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1686 raise TypeError, 'new property "%s" not a Date'% propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1687 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1688
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1689 elif value is not None and isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1690 if not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1691 raise TypeError, 'new property "%s" not an '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1692 'Interval'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1693 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1694
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1695 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1696 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1697 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1698 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1699 raise TypeError, 'new property "%s" not numeric'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1700
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1701 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1702 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1703 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1704 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1705 raise TypeError, 'new property "%s" not boolean'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1706
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1707 # nothing to do?
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1708 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1709 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1710
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1711 # update the activity time
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1712 propvalues['activity'] = date.Date()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1713 propvalues['actor'] = self.db.getuid()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1714
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1715 # do the set
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1716 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1717
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1718 # remove the activity props now they're handled
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1719 del propvalues['activity']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1720 del propvalues['actor']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1721
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1722 # journal the set
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1723 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1724 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1725
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1726 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1727
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1728 def retire(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1729 '''Retire a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1730
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1731 The properties on the node remain available from the get() method,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1732 and the node's id is never reused.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1733
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1734 Retired nodes are not returned by the find(), list(), or lookup()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1735 methods, and other nodes may reuse the values of their key properties.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1736 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1737 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1738 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1739
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1740 self.fireAuditors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1741
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1742 # use the arg for __retired__ to cope with any odd database type
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1743 # conversion (hello, sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1744 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1745 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1746 self.db.sql(sql, (1, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1747 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1748 self.db.addjournal(self.classname, nodeid, ''"retired", None)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1749
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1750 self.fireReactors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1751
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1752 def restore(self, nodeid):
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
1753 '''Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1754
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1755 Make node available for all operations like it was before retirement.
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1756 '''
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1757 if self.db.journaltag is None:
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1758 raise DatabaseError, 'Database open read-only'
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1759
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1760 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1761 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1762 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1763 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1764 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1765 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1766 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1767 else:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1768 raise KeyError, "Key property (%s) of retired node clashes with \
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1769 existing one (%s)" % (key, node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1770
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1771 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1772 # use the arg for __retired__ to cope with any odd database type
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1773 # conversion (hello, sqlite)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1774 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1775 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1776 self.db.sql(sql, (0, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1777 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1778 self.db.addjournal(self.classname, nodeid, ''"restored", None)
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1779
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1780 self.fireReactors('restore', nodeid, None)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1781
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1782 def is_retired(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1783 '''Return true if the node is rerired
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1784 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1785 sql = 'select __retired__ from _%s where id=%s'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1786 self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1787 self.db.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1788 return int(self.db.sql_fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1789
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1790 def destroy(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1791 '''Destroy a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1792
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1793 WARNING: this method should never be used except in extremely rare
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1794 situations where there could never be links to the node being
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1795 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1796
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1797 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1798
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1799 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1800
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1801 WARNING: really, use retire() instead
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1802
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1803 Well, I think that's enough warnings. This method exists mostly to
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1804 support the session storage of the cgi interface.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1805
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1806 The node is completely removed from the hyperdb, including all journal
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1807 entries. It will no longer be available, and will generally break code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1808 if there are any references to the node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1809 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1810 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1811 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1812 self.db.destroynode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1813
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1814 def history(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1815 '''Retrieve the journal of edits on a particular node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1816
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1817 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1818 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1819
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1820 The returned list contains tuples of the form
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1821
1409
8dc60d87ab42 Fixed a backlog of bug reports, and worked on python 2.3 compatibility:
Richard Jones <richard@users.sourceforge.net>
parents: 1383
diff changeset
1822 (nodeid, date, tag, action, params)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1823
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1824 'date' is a Timestamp object specifying the time of the change and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1825 'tag' is the journaltag specified when the database was opened.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1826 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1827 if not self.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1828 raise ValueError, 'Journalling is disabled for this class'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1829 return self.db.getjournal(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1830
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1831 # Locating nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1832 def hasnode(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1833 '''Determine if the given nodeid actually exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1834 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1835 return self.db.hasnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1836
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1837 def setkey(self, propname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1838 '''Select a String property of this class to be the key property.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1839
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1840 'propname' must be the name of a String property of this class or
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1841 None, or a TypeError is raised. The values of the key property on
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1842 all existing nodes must be unique or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1843 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1844 prop = self.getprops()[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1845 if not isinstance(prop, String):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1846 raise TypeError, 'key properties must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1847 self.key = propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1848
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1849 def getkey(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1850 '''Return the name of the key property for this class or None.'''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1851 return self.key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1852
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1853 def lookup(self, keyvalue):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1854 '''Locate a particular node by its key property and return its id.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1855
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1856 If this class has no key property, a TypeError is raised. If the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1857 'keyvalue' matches one of the values for the key property among
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1858 the nodes in this class, the matching node's id is returned;
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1859 otherwise a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1860 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1861 if not self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1862 raise TypeError, 'No key property set for class %s'%self.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1863
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1864 # use the arg to handle any odd database type conversion (hello,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1865 # sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1866 sql = "select id from _%s where _%s=%s and __retired__ <> %s"%(
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1867 self.classname, self.key, self.db.arg, self.db.arg)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1868 self.db.sql(sql, (keyvalue, 1))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1869
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1870 # see if there was a result that's not retired
1203
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1871 row = self.db.sql_fetchone()
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1872 if not row:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1873 raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key,
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1874 keyvalue, self.classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1875
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1876 # return the id
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1877 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1878 return str(row[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1879
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1880 def find(self, **propspec):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1881 '''Get the ids of nodes in this class which link to the given nodes.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1882
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1883 'propspec' consists of keyword args propname=nodeid or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1884 propname={nodeid:1, }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1885 'propname' must be the name of a property in this class, or a
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1886 KeyError is raised. That property must be a Link or
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1887 Multilink property, or a TypeError is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1888
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1889 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1890 the nodeids will be returned. Examples::
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1891
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1892 db.issue.find(messages='1')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1893 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1894 '''
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1895 # shortcut
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1896 if not propspec:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1897 return []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1898
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1899 # validate the args
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1900 props = self.getprops()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1901 propspec = propspec.items()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1902 for propname, nodeids in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1903 # check the prop is OK
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1904 prop = props[propname]
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1905 if not isinstance(prop, Link) and not isinstance(prop, Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1906 raise TypeError, "'%s' not a Link/Multilink property"%propname
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1907
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1908 # first, links
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1909 a = self.db.arg
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1910 allvalues = ()
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1911 sql = []
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1912 where = []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1913 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1914 if not isinstance(props[prop], hyperdb.Link):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1915 continue
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1916 if type(values) is type({}) and len(values) == 1:
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1917 values = values.keys()[0]
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1918 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1919 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1920 where.append('_%s = %s'%(prop, a))
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
1921 elif values is None:
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
1922 where.append('_%s is NULL'%prop)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1923 else:
2494
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1924 values = values.keys()
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1925 s = ''
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1926 if None in values:
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1927 values.remove(None)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1928 s = '_%s is NULL or '%prop
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1929 allvalues += tuple(values)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1930 s += '_%s in (%s)'%(prop, ','.join([a]*len(values)))
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
1931 where.append('(' + s +')')
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1932 if where:
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1933 allvalues = (1, ) + allvalues
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1934 sql.append('''select id from _%s where __retired__ <> %s
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1935 and %s'''%(self.classname, a, ' and '.join(where)))
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1936
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1937 # now multilinks
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1938 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1939 if not isinstance(props[prop], hyperdb.Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1940 continue
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1941 if not values:
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1942 continue
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1943 allvalues += (1, )
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1944 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1945 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1946 s = a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1947 else:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1948 allvalues += tuple(values.keys())
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1949 s = ','.join([a]*len(values))
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1950 tn = '%s_%s'%(self.classname, prop)
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1951 sql.append('''select id from _%s, %s where __retired__ <> %s
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1952 and id = %s.nodeid and %s.linkid in (%s)'''%(self.classname,
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1953 tn, a, tn, tn, s))
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1954
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1955 if not sql:
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1956 return []
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1957 sql = ' union '.join(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1958 self.db.sql(sql, allvalues)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1959 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1960 l = [str(x[0]) for x in self.db.sql_fetchall()]
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1961 return l
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1962
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1963 def stringFind(self, **requirements):
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1964 '''Locate a particular node by matching a set of its String
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1965 properties in a caseless search.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1966
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1967 If the property is not a String property, a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1968
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1969 The return is a list of the id of all nodes that match.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1970 '''
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1971 where = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1972 args = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1973 for propname in requirements.keys():
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1974 prop = self.properties[propname]
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
1975 if not isinstance(prop, String):
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1976 raise TypeError, "'%s' not a String property"%propname
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1977 where.append(propname)
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1978 args.append(requirements[propname].lower())
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1979
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1980 # generate the where clause
1549
a53a7e197360 fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents: 1539
diff changeset
1981 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where])
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1982 sql = 'select id from _%s where %s and __retired__<>%s'%(
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1983 self.classname, s, self.db.arg)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1984 args.append(1)
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1985 self.db.sql(sql, tuple(args))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1986 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1987 l = [str(x[0]) for x in self.db.sql_fetchall()]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1988 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1989
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1990 def list(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1991 ''' Return a list of the ids of the active nodes in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1992 '''
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1993 return self.getnodeids(retired=0)
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1994
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1995 def getnodeids(self, retired=None):
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1996 ''' Retrieve all the ids of the nodes for a particular Class.
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1997
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1998 Set retired=None to get all nodes. Otherwise it'll get all the
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1999 retired or non-retired nodes, depending on the flag.
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2000 '''
1707
3d627e34f18e sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1599
diff changeset
2001 # flip the sense of the 'retired' flag if we don't want all of them
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2002 if retired is not None:
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2003 if retired:
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2004 args = (0, )
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2005 else:
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2006 args = (1, )
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2007 sql = 'select id from _%s where __retired__ <> %s'%(self.classname,
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2008 self.db.arg)
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2009 else:
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2010 args = ()
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2011 sql = 'select id from _%s'%self.classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2012 self.db.sql(sql, args)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2013 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2014 ids = [str(x[0]) for x in self.db.cursor.fetchall()]
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2015 return ids
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2016
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2017 def _subselect (self, cn, tn) :
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2018 '''Create a subselect. This is factored out because some
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2019 databases (hmm only one, so far) doesn't support subselects
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2020 look for "I can't believe it's not a toy RDBMS" in the mysql
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2021 backend.
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2022 '''
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2023 return '_%s.id not in (select nodeid from %s)'%(cn, tn)
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2024
1249
6c24a86a12ae Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents: 1244
diff changeset
2025 def filter(self, search_matches, filterspec, sort=(None,None),
6c24a86a12ae Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents: 1244
diff changeset
2026 group=(None,None)):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2027 '''Return a list of the ids of the active nodes in this class that
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2028 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2029 sort spec
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2030
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2031 "filterspec" is {propname: value(s)}
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2032
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2033 "sort" and "group" are (dir, prop) where dir is '+', '-' or None
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2034 and prop is a prop name or None
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2035
2362
10fc45eea226 fix SearchAction use of Class.filter(), and clarify API docs for same
Richard Jones <richard@users.sourceforge.net>
parents: 2319
diff changeset
2036 "search_matches" is {nodeid: marker} or None
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2037
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2038 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2039 value to match is a list:
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2040
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2041 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2042 2. Other properties must match any of the elements in the list.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2043 '''
2578
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2044 # we can't match anything if search_matches is empty
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2045 if search_matches == {}:
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2046 return []
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2047
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2048 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2049 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2050
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2051 icn = self.classname
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2052
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2053 # vars to hold the components of the SQL statement
2586
3c67f4bfa225 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2578
diff changeset
2054 frum = [] # FROM clauses
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2055 loj = [] # LEFT OUTER JOIN clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2056 where = [] # WHERE clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2057 args = [] # *any* positional arguments
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2058 a = self.db.arg
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2059
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2060 # figure the WHERE clause from the filterspec
2256
0b198ed096af fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents: 2248
diff changeset
2061 mlfilt = 0 # are we joining with Multilink tables?
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2062 proptree = self._proptree (filterspec)
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2063 for p in proptree :
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2064 cn = p.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2065 ln = p.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2066 pln = p.parent.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2067 pcn = p.parent.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2068 k = p.name
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2069 v = p.val
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2070 propclass = p.prcls
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2071 if p.children :
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2072 if isinstance (propclass, Multilink) :
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2073 mlfilt = 1
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2074 mn = '%s_%s'%(pcn, p.name)
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2075 frum.append(mn)
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2076 frum.append('_%s as _%s' % (cn, ln))
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2077 where.append('_%s.id=%s.nodeid and %s.linkid=_%s.id'%(pln,
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2078 mn, mn, ln))
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2079 else :
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2080 if not isinstance (propclass, Link) :
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2081 raise ValueError,"%s must be Link/Multilink property"%k
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2082 frum.append('_%s as _%s' % (cn, ln))
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2083 where.append('_%s._%s=_%s.id'%(pln, k, ln))
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2084 continue
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2085 # now do other where clause stuff
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2086 elif isinstance(propclass, Multilink):
2256
0b198ed096af fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents: 2248
diff changeset
2087 mlfilt = 1
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2088 tn = '%s_%s'%(pcn, k)
1556
9e89f5394f6c handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents: 1555
diff changeset
2089 if v in ('-1', ['-1']):
9e89f5394f6c handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents: 1555
diff changeset
2090 # only match rows that have count(linkid)=0 in the
9e89f5394f6c handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents: 1555
diff changeset
2091 # corresponding multilink table)
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2092 where.append(self._subselect(pcn, tn))
1556
9e89f5394f6c handle myriad of argument types to filter()
Richard Jones <richard@users.sourceforge.net>
parents: 1555
diff changeset
2093 elif isinstance(v, type([])):
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1549
diff changeset
2094 frum.append(tn)
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
2095 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2096 where.append('_%s.id=%s.nodeid and %s.linkid in (%s)'%(pln,
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2097 tn, tn, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2098 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2099 else:
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1549
diff changeset
2100 frum.append(tn)
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2101 where.append('_%s.id=%s.nodeid and %s.linkid=%s'%(pln, tn,
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2102 tn, a))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2103 args.append(v)
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2104 elif k == 'id':
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2105 if isinstance(v, type([])):
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2106 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2107 where.append('_%s.%s in (%s)'%(pln, k, s))
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2108 args = args + v
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2109 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2110 where.append('_%s.%s=%s'%(pln, k, a))
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2111 args.append(v)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2112 elif isinstance(propclass, String):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2113 if not isinstance(v, type([])):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2114 v = [v]
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2115
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2116 # Quote the bits in the string that need it and then embed
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2117 # in a "substring" search. Note - need to quote the '%' so
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2118 # they make it through the python layer happily
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2119 v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v]
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2120
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2121 # now add to the where clause
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2122 where.append('('
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2123 +' and '.join(["_%s._%s LIKE '%s'"%(pln, k, s) for s in v])
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2124 +')')
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2125 # note: args are embedded in the query string now
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2126 elif isinstance(propclass, Link):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2127 if isinstance(v, type([])):
2508
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2128 d = {}
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2129 for entry in v:
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2130 if entry == '-1':
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2131 entry = None
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2132 d[entry] = entry
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2133 l = []
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2134 if d.has_key(None) or not d:
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2135 del d[None]
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2136 l.append('_%s._%s is NULL'%(pln, k))
2508
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2137 if d:
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2138 v = d.keys()
1295
9c3459cb8ab6 work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents: 1252
diff changeset
2139 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2140 l.append('(_%s._%s in (%s))'%(pln, k, s))
1295
9c3459cb8ab6 work-around for sqlite bug
Richard Jones <richard@users.sourceforge.net>
parents: 1252
diff changeset
2141 args = args + v
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2142 if l:
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2143 where.append('(' + ' or '.join(l) +')')
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2144 else:
2508
8e1c1623710b rdbms backends not filtering correctly on link=None
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
2145 if v in ('-1', None):
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2146 v = None
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2147 where.append('_%s._%s is NULL'%(pln, k))
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2148 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2149 where.append('_%s._%s=%s'%(pln, k, a))
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2150 args.append(v)
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2151 elif isinstance(propclass, Date):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2152 dc = self.db.hyperdb_to_sql_value[hyperdb.Date]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2153 if isinstance(v, type([])):
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2154 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2155 where.append('_%s._%s in (%s)'%(pln, k, s))
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2156 args = args + [dc(date.Date(x)) for x in v]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2157 else:
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2158 try:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2159 # Try to filter on range of dates
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3554
diff changeset
2160 date_rng = propclass.range_from_raw(v, self.db)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2161 if date_rng.from_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2162 where.append('_%s._%s >= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2163 args.append(dc(date_rng.from_value))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2164 if date_rng.to_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2165 where.append('_%s._%s <= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2166 args.append(dc(date_rng.to_value))
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2167 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2168 # If range creation fails - ignore that search parameter
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2169 pass
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2170 elif isinstance(propclass, Interval):
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2171 # filter using the __<prop>_int__ column
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2172 if isinstance(v, type([])):
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2173 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2174 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s))
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2175 args = args + [date.Interval(x).as_seconds() for x in v]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2176 else:
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1578
diff changeset
2177 try:
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1578
diff changeset
2178 # Try to filter on range of intervals
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1578
diff changeset
2179 date_rng = Range(v, date.Interval)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2180 if date_rng.from_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2181 where.append('_%s.__%s_int__ >= %s'%(pln, k, a))
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2182 args.append(date_rng.from_value.as_seconds())
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2183 if date_rng.to_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2184 where.append('_%s.__%s_int__ <= %s'%(pln, k, a))
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2185 args.append(date_rng.to_value.as_seconds())
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1578
diff changeset
2186 except ValueError:
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1578
diff changeset
2187 # If range creation fails - ignore that search parameter
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2188 pass
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2189 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2190 if isinstance(v, type([])):
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2191 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2192 where.append('_%s._%s in (%s)'%(pln, k, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2193 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2194 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2195 where.append('_%s._%s=%s'%(pln, k, a))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2196 args.append(v)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2197
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2198 props = self.getprops()
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2199
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2200 # don't match retired nodes
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2201 where.append('_%s.__retired__ <> 1'%icn)
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2202
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2203 # add results of full text search
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2204 if search_matches is not None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2205 v = search_matches.keys()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2206 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2207 where.append('_%s.id in (%s)'%(icn, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2208 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2209
2720
15d945dff53d add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents: 2692
diff changeset
2210 # sanity check: sorting *and* grouping on the same property?
15d945dff53d add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents: 2692
diff changeset
2211 if group[1] == sort[1]:
15d945dff53d add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents: 2692
diff changeset
2212 sort = (None, None)
15d945dff53d add sanity check for sort and group on same property [SF#1033477]
Richard Jones <richard@users.sourceforge.net>
parents: 2692
diff changeset
2213
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2214 # "grouping" is just the first-order sorting in the SQL fetch
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2215 orderby = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2216 ordercols = []
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2217 mlsort = []
3525
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2218 rhsnum = 0
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2219 for sortby in group, sort:
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2220 sdir, prop = sortby
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2221 if sdir and prop:
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2222 if isinstance(props[prop], Multilink):
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2223 mlsort.append(sortby)
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2224 continue
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2225 elif isinstance(props[prop], Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2226 # use the int column for sorting
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2227 o = '__'+prop+'_int__'
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2228 ordercols.append(o)
2318
fa2f7ba34399 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2275
diff changeset
2229 elif isinstance(props[prop], Link):
fa2f7ba34399 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2275
diff changeset
2230 # determine whether the linked Class has an order property
fa2f7ba34399 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2275
diff changeset
2231 lcn = props[prop].classname
fa2f7ba34399 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2275
diff changeset
2232 link = self.db.classes[lcn]
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2233 o = '_%s._%s'%(icn, prop)
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
2234 op = link.orderprop()
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
2235 if op != 'id':
2318
fa2f7ba34399 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2275
diff changeset
2236 tn = '_' + lcn
3525
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2237 rhs = 'rhs%s_'%rhsnum
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2238 rhsnum += 1
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2239 loj.append('LEFT OUTER JOIN %s as %s on %s=%s.id'%(
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2240 tn, rhs, o, rhs))
7be25d75c3d5 Fixed SQL generation for sort/group by separate Link properties [SF#1417565]
Richard Jones <richard@users.sourceforge.net>
parents: 3489
diff changeset
2241 o = '%s._%s'%(rhs, op)
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
2242 ordercols.append(o)
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2243 elif prop == 'id':
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2244 o = '_%s.id'%icn
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2245 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2246 o = '_%s._%s'%(icn, prop)
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2247 ordercols.append(o)
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2248 if sdir == '-':
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2249 o += ' desc'
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2250 orderby.append(o)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2251
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2252 # construct the SQL
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2253 frum.append('_'+icn)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2254 frum = ','.join(frum)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2255 if where:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2256 where = ' where ' + (' and '.join(where))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2257 else:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2258 where = ''
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2259 if mlfilt:
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2260 # we're joining tables on the id, so we will get dupes if we
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2261 # don't distinct()
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2262 cols = ['distinct(_%s.id)'%icn]
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2263 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2264 cols = ['_%s.id'%icn]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2265 if orderby:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2266 cols = cols + ordercols
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2267 order = ' order by %s'%(','.join(orderby))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2268 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2269 order = ''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2270 cols = ','.join(cols)
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2271 loj = ' '.join(loj)
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2272 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2273 args = tuple(args)
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2274 __traceback_info__ = (sql, args)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2275 self.db.sql(sql, args)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
2276 l = self.db.sql_fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2277
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2278 # return the IDs (the first column)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2279 # XXX numeric ids
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2280 l = [str(row[0]) for row in l]
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2281
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2282 if not mlsort:
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2283 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2284 self.db.stats['filtering'] += (time.time() - start_t)
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2285 return l
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2286
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2287 # ergh. someone wants to sort by a multilink.
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2288 r = []
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2289 for id in l:
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2290 m = []
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2291 for ml in mlsort:
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2292 m.append(self.get(id, ml[1]))
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2293 r.append((id, m))
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2294 i = 0
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2295 for sortby in mlsort:
2234
70d21059aa18 fix nested scope bug in multilink sorting in rdbms filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2228
diff changeset
2296 def sortfun(a, b, dir=sortby[i], i=i):
2185
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2297 if dir == '-':
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2298 return cmp(b[1][i], a[1][i])
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2299 else:
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2300 return cmp(a[1][i], b[1][i])
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2301 r.sort(sortfun)
c52a931879c4 sort/group by multilink in RDBMS
Richard Jones <richard@users.sourceforge.net>
parents: 2175
diff changeset
2302 i += 1
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2303 r = [i[0] for i in r]
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2304
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2305 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2306 self.db.stats['filtering'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2307
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2308 return r
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2309
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2310 def count(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2311 '''Get the number of nodes in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2312
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2313 If the returned integer is 'numnodes', the ids of all the nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2314 in this class run from 1 to numnodes, and numnodes+1 will be the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2315 id of the next node to be created in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2316 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2317 return self.db.countnodes(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2318
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2319 # Manipulating properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2320 def getprops(self, protected=1):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2321 '''Return a dictionary mapping property names to property objects.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2322 If the "protected" flag is true, we include protected properties -
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2323 those which may not be modified.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2324 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2325 d = self.properties.copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2326 if protected:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2327 d['id'] = String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2328 d['creation'] = hyperdb.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2329 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1175
diff changeset
2330 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2331 d['actor'] = hyperdb.Link('user')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2332 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2333
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2334 def addprop(self, **properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2335 '''Add properties to this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2336
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2337 The keyword arguments in 'properties' must map names to property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2338 objects, or a TypeError is raised. None of the keys in 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2339 may collide with the names of existing properties, or a ValueError
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2340 is raised before any properties have been added.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2341 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2342 for key in properties.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2343 if self.properties.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2344 raise ValueError, key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2345 self.properties.update(properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2346
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2347 def index(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2348 '''Add (or refresh) the node to search indexes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2349 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2350 # find all the String properties that have indexme
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2351 for prop, propclass in self.getprops().items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2352 if isinstance(propclass, String) and propclass.indexme:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2353 self.db.indexer.add_text((self.classname, nodeid, prop),
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2354 str(self.get(nodeid, prop)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2355
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2356 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2357 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2358 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2359 def export_list(self, propnames, nodeid):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2360 ''' Export a node - generate a list of CSV-able data in the order
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2361 specified by propnames for the given node.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2362 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2363 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2364 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2365 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2366 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2367 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2368 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2369 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2370 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2371 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2372 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2373 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2374 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2375 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2376 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2377 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2378 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2379 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2380
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2381 def import_list(self, propnames, proplist):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2382 ''' Import a node - all information including "id" is present and
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2383 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2384 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2385 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2386
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2387 Return the nodeid of the node imported.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2388 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2389 if self.db.journaltag is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2390 raise DatabaseError, 'Database open read-only'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2391 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2392
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2393 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2394 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2395 retire = 0
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2396 if not "id" in propnames:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2397 newid = self.db.newid(self.classname)
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2398 else:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2399 newid = eval(proplist[propnames.index("id")])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2400 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2401 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2402 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2403
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2404 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2405 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2406
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2407 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2408 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2409 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2410 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2411 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2412 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2413 retire = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2414 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2415 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2416 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2417 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2418
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2419 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2420 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2421 # don't set Nones
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2422 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2423 elif isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2424 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2425 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2426 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2427 elif isinstance(prop, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2428 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2429 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2430 value = pwd
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2431 elif isinstance(prop, String):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2432 if isinstance(value, unicode):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2433 value = value.encode('utf8')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2434 if not isinstance(value, str):
2691
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2435 raise TypeError, \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2436 'new property "%(propname)s" not a string: %(value)r' \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2437 % locals()
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2438 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2439 self.db.indexer.add_text((self.classname, newid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2440 value)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2441 d[propname] = value
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2442
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2443 # get a new id if necessary
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2444 if newid is None:
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2445 newid = self.db.newid(self.classname)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2446
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2447 # insert new node or update existing?
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2448 if not self.hasnode(newid):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2449 self.db.addnode(self.classname, newid, d) # insert
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2450 else:
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2451 self.db.setnode(self.classname, newid, d) # update
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2452
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2453 # retire?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2454 if retire:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2455 # use the arg for __retired__ to cope with any odd database type
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2456 # conversion (hello, sqlite)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2457 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2458 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2459 self.db.sql(sql, (1, newid))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2460 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2461
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2462 def export_journals(self):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2463 '''Export a class's journal - generate a list of lists of
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2464 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2465
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2466 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2467
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2468 No heading here - the columns are fixed.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2469 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2470 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2471 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2472 for nodeid in self.getnodeids():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2473 for nodeid, date, user, action, params in self.history(nodeid):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2474 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2475 if action == 'set':
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2476 export_data = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2477 for propname, value in params.items():
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2478 if not properties.has_key(propname):
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2479 # property no longer in the schema
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2480 continue
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2481
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2482 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2483 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2484 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2485 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2486 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2487 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2488 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2489 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2490 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2491 value = str(value)
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2492 export_data[propname] = value
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2493 params = export_data
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2494 l = [nodeid, date, user, action, params]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2495 r.append(map(repr, l))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2496 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2497
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2498 def import_journals(self, entries):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2499 '''Import a class's journal.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2500
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2501 Uses setjournal() to set the journal for each item.'''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2502 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2503 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2504 for l in entries:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2505 l = map(eval, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2506 nodeid, jdate, user, action, params = l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2507 r = d.setdefault(nodeid, [])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2508 if action == 'set':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2509 for propname, value in params.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2510 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2511 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2512 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2513 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2514 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2515 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2516 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2517 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2518 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2519 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2520 value = pwd
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2521 params[propname] = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2522 r.append((nodeid, date.Date(jdate), user, action, params))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2523
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2524 for nodeid, l in d.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2525 self.db.setjournal(self.classname, nodeid, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2526
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2586
diff changeset
2527 class FileClass(hyperdb.FileClass, Class):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2528 '''This class defines a large chunk of data. To support this, it has a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2529 mandatory String property "content" which is typically saved off
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2530 externally to the hyperdb.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2531
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2532 The default MIME type of this data is defined by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2533 "default_mime_type" class attribute, which may be overridden by each
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2534 node if the class defines a "type" String property.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2535 '''
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2536 def __init__(self, db, classname, **properties):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2537 '''The newly-created class automatically includes the "content"
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2538 and "type" properties.
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2539 '''
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2540 if not properties.has_key('content'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2541 properties['content'] = hyperdb.String(indexme='yes')
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2542 if not properties.has_key('type'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2543 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2544 Class.__init__(self, db, classname, **properties)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2545
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2546 def create(self, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2547 ''' snaffle the file propvalue and store in a file
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2548 '''
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2549 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2550 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2551 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2552
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2553 # now remove the content property so it's not stored in the db
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2554 content = propvalues['content']
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2555 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2556
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2557 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2558 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2559
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2560 # figure the mime type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2561 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2562
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2563 # and index!
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2564 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2565 self.db.indexer.add_text((self.classname, newid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2566 content, mime_type)
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2567
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2568 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2569 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2570
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2571 # store off the content as a file
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2572 self.db.storefile(self.classname, newid, None, content)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2573 return newid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2574
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2575 def get(self, nodeid, propname, default=_marker, cache=1):
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2576 ''' Trap the content propname and get it from the file
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2577
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2578 'cache' exists for backwards compatibility, and is not used.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2579 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2580 poss_msg = 'Possibly a access right configuration problem.'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2581 if propname == 'content':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2582 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2583 return self.db.getfile(self.classname, nodeid, None)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2584 except IOError, (strerror):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2585 # BUG: by catching this we donot see an error in the log.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2586 return 'ERROR reading file: %s%s\n%s\n%s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2587 self.classname, nodeid, poss_msg, strerror)
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2588 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2589 return Class.get(self, nodeid, propname, default)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2590 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2591 return Class.get(self, nodeid, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2592
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2593 def getprops(self, protected=1):
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2594 '''In addition to the actual properties on the node, these methods
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2595 provide the "content" property. If the "protected" flag is true,
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2596 we include protected properties - those which may not be
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2597 modified.
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2598
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2599 Note that the content prop is indexed separately, hence no indexme.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2600 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2601 d = Class.getprops(self, protected=protected).copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2602 d['content'] = hyperdb.String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2603 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2604
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2605 def set(self, itemid, **propvalues):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2606 ''' Snarf the "content" propvalue and update it in a file
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2607 '''
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2608 self.fireAuditors('set', itemid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2609 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2610
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2611 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2612 content = None
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2613 if propvalues.has_key('content'):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2614 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2615 del propvalues['content']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2616
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2617 # do the database create
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2618 propvalues = self.set_inner(itemid, **propvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2619
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2620 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2621 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2622 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2623 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2624 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2625 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2626 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2627 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2628 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2629
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2630 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2631 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2632 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2633
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2634 def index(self, nodeid):
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2635 ''' Add (or refresh) the node to search indexes.
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2636
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2637 Use the content-type property for the content property.
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2638 '''
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2639 # find all the String properties that have indexme
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2640 for prop, propclass in self.getprops().items():
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2641 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2642 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2643 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2644 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2645 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2646 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2647 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2648 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2649 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2650 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2651 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2652 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2653
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2654 # XXX deviation from spec - was called ItemClass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2655 class IssueClass(Class, roundupdb.IssueClass):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2656 # Overridden methods:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2657 def __init__(self, db, classname, **properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2658 '''The newly-created class automatically includes the "messages",
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2659 "files", "nosy", and "superseder" properties. If the 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2660 dictionary attempts to specify any of these properties or a
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2661 "creation", "creator", "activity" or "actor" property, a ValueError
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2662 is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2663 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2664 if not properties.has_key('title'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2665 properties['title'] = hyperdb.String(indexme='yes')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2666 if not properties.has_key('messages'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2667 properties['messages'] = hyperdb.Multilink("msg")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2668 if not properties.has_key('files'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2669 properties['files'] = hyperdb.Multilink("file")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2670 if not properties.has_key('nosy'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2671 # note: journalling is turned off as it really just wastes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2672 # space. this behaviour may be overridden in an instance
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2673 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2674 if not properties.has_key('superseder'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2675 properties['superseder'] = hyperdb.Multilink(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2676 Class.__init__(self, db, classname, **properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2677
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2678 # vim: set et sts=4 sw=4 :

Roundup Issue Tracker: http://roundup-tracker.org/