annotate roundup/backends/rdbms_common.py @ 3682:193f316dbbe9

More transitive-property support. - Implemented transitive properties in sort and group specs. Sort/group specs can now be lists of specs. - All regression tests except for one metakit backend test related to metakit having no representation of NULL pass - Fixed more PEP 8 whitespace peeves (and probably introduced some new ones :-) - Moved Proptree from support.py to hyperdb.py due to circular import - Moved some proptree-specific methods from Class to Proptree - Added a test for sorting by ids -> should be numeric sort (which now really works for all backends) - Added "required" attribute to all property classes in hyperdb (e.g., String, Link,...), see Feature Requests [SF#539081] -> factored common stuff to _Type. Note that I also converted to a new-style class when I was at it. Bad: The repr changes for new-style classes which made some SQL backends break (!) because the repr of Multilink is used in the schema storage. Fixed the repr to be independent of the class type. - Added get_required_props to Class. Todo: should also automagically make the key property required... - Add a sort_repr method to property classes. This defines the sort-order. Individual backends may use diffent routines if the outcome is the same. This one has a special case for id properties to make the sorting numeric. Using these methods isn't mandatory in backends as long as the sort-order is correct. - Multilink sorting takes orderprop into account. It used to sort by ids. You can restore the old behaviour by specifying id as the orderprop of the Multilink if you really need that. - If somebody specified a Link or Multilink as orderprop, we sort by labelprop of that class -- not transitively by orderprop. I've resited the tempation to implement recursive orderprop here: There could even be loops if several classes specify a Link or Multilink as the orderprop... - Fixed a bug in Metakit-Backend: When sorting by Links, the backend would do a natural join to the Link class. It would rename the "id" attribute before joining but *not* all the other attributes of the joined class. So in one test-case we had a name-clash with priority.name and status.name when sorting *and* grouping by these attributes. Depending on the order of joining this would produce a name-clash with broken sort-results (and broken display if the original class has an attribute that clashes). I'm now doing the sorting of Links in the generic filter method for the metakit backend. I've left the dead code in the metakit-backend since correctly implementing this in the backend will probably be more efficient. - updated doc/design.html with the new docstring of filter.
author Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
date Mon, 21 Aug 2006 12:19:48 +0000
parents fa7becc62534
children ad5e1d75e028
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
1 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
4 # under the same terms as Python, so long as this copyright message and
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
5 # disclaimer are retained in their original form.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
6 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
11 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
17 #
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
18 #$Id: rdbms_common.py,v 1.175 2006-08-21 12:19:48 schlatterbeck Exp $
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
19 ''' Relational database (SQL) backend common code.
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
20
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
21 Basics:
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
22
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
23 - map roundup classes to relational tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
24 - automatically detect schema changes and modify the table schemas
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
25 appropriately (we store the "database version" of the schema in the
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
26 database itself as the only row of the "schema" table)
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
27 - multilinks (which represent a many-to-many relationship) are handled through
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
28 intermediate tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
29 - journals are stored adjunct to the per-class tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
30 - table names and columns have "_" prepended so the names can't clash with
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
31 restricted names (like "order")
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
32 - retirement is determined by the __retired__ column being true
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
33
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
34 Database-specific changes may generally be pushed out to the overridable
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
35 sql_* methods, since everything else should be fairly generic. There's
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
36 probably a bit of work to be done if a database is used that actually
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
37 honors column typing, since the initial databases don't (sqlite stores
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
38 everything as a string.)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
39
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
40 The schema of the hyperdb being mapped to the database is stored in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
41 database itself as a repr()'ed dictionary of information about each Class
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
42 that maps to a table. If that information differs from the hyperdb schema,
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
43 then we update it. We also store in the schema dict a version which
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
44 allows us to upgrade the database schema when necessary. See upgrade_db().
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
45 '''
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
46 __docformat__ = 'restructuredtext'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 # standard python modules
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
49 import sys, os, time, re, errno, weakref, copy, logging
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 # roundup modules
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
52 from roundup import hyperdb, date, password, roundupdb, security, support
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 from roundup.hyperdb import String, Password, Date, Interval, Link, \
1417
472c21af7f69 fixed error in indexargs_url (thanks Patrick Ohly)
Richard Jones <richard@users.sourceforge.net>
parents: 1415
diff changeset
54 Multilink, DatabaseError, Boolean, Number, Node
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
55 from roundup.backends import locking
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
56 from roundup.support import reversed
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 # support
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 from blobfiles import FileStorage
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
60 try:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
61 from indexer_xapian import Indexer
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
62 except ImportError:
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
63 from indexer_rdbms import Indexer
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
64 from sessions_rdbms import Sessions, OneTimeKeys
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
65 from roundup.date import Range
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
67 # number of rows to keep in memory
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
68 ROW_CACHE_SIZE = 100
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
69
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
70 # dummy value meaning "argument not passed"
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
71 _marker = []
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
72
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
73 def _num_cvt(num):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
74 num = str(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
75 try:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
76 return int(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
77 except:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
78 return float(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
79
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
80 def _bool_cvt(value):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
81 if value in ('TRUE', 'FALSE'):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
82 return {'TRUE': 1, 'FALSE': 0}[value]
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
83 # assume it's a number returned from the db API
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
84 return int(value)
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
85
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
86 def connection_dict(config, dbnamestr=None):
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
87 ''' Used by Postgresql and MySQL to detemine the keyword args for
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
88 opening the database connection.'''
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
89 d = { }
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
90 if dbnamestr:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
91 d[dbnamestr] = config.RDBMS_NAME
3099
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
92 for name in ('host', 'port', 'password', 'user', 'read_default_group',
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
93 'read_default_file'):
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
94 cvar = 'RDBMS_'+name.upper()
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
95 if config[cvar] is not None:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
96 d[name] = config[cvar]
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
97 return d
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
98
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
100 ''' Wrapper around an SQL database that presents a hyperdb interface.
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
101
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
102 - some functionality is specific to the actual SQL database, hence
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
103 the sql_* methods that are NotImplemented
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
104 - we keep a cache of the latest ROW_CACHE_SIZE row fetches.
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
105 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 def __init__(self, config, journaltag=None):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 ''' Open the database and load the schema from it.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 self.config, self.journaltag = config, journaltag
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 self.dir = config.DATABASE
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 self.classes = {}
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
112 self.indexer = Indexer(self)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 self.security = security.Security(self)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 # additional transaction support for external files and the like
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
118 # keep a cache of the N most recently retrieved rows of any kind
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
119 # (classname, nodeid) = row
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
120 self.cache = {}
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
121 self.cache_lru = []
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
122 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
123 'filtering': 0}
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
124
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
125 # database lock
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
126 self.lockfile = None
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
127
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 # open a connection to the database, creating the "conn" attribute
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
129 self.open_connection()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
131 def clearCache(self):
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
132 self.cache = {}
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
133 self.cache_lru = []
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
134
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
135 def getSessionManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
136 return Sessions(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
137
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
138 def getOTKManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
139 return OneTimeKeys(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
140
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
141 def open_connection(self):
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
142 ''' Open a connection to the database, creating it if necessary.
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
143
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
144 Must call self.load_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 raise NotImplemented
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
148 def sql(self, sql, args=None):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 ''' Execute the sql with the optional args.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
150 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
151 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
152 logging.getLogger('hyperdb').debug('SQL %r %r'%(sql, args))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153 if args:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
154 self.cursor.execute(sql, args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 else:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
156 self.cursor.execute(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
158 def sql_fetchone(self):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 ''' Fetch a single row. If there's nothing to fetch, return None.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
161 return self.cursor.fetchone()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
162
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
163 def sql_fetchall(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
164 ''' Fetch all rows. If there's nothing to fetch, return [].
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
165 '''
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
166 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
167
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
168 def sql_stringquote(self, value):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
169 ''' Quote the string so it's safe to put in the 'sql quotes'
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
170 '''
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
171 return re.sub("'", "''", str(value))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
172
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
173 def init_dbschema(self):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
174 self.database_schema = {
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
175 'version': self.current_db_version,
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
176 'tables': {}
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
177 }
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
178
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
179 def load_dbschema(self):
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
180 ''' Load the schema definition that the database currently implements
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
181 '''
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
182 self.cursor.execute('select schema from schema')
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
183 schema = self.cursor.fetchone()
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
184 if schema:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
185 self.database_schema = eval(schema[0])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
186 else:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
187 self.database_schema = {}
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
188
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
189 def save_dbschema(self):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 ''' Save the schema definition that the database currently implements
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
191 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
192 s = repr(self.database_schema)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
193 self.sql('delete from schema')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
194 self.sql('insert into schema values (%s)', (s,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 def post_init(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 ''' Called once the schema initialisation has finished.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 We should now confirm that the schema defined by our "classes"
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
200 attribute actually matches the schema in the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201 '''
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
202 save = 0
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
203
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
204 # handle changes in the schema
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
205 tables = self.database_schema['tables']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 for classname, spec in self.classes.items():
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
207 if tables.has_key(classname):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
208 dbspec = tables[classname]
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
209 if self.update_class(spec, dbspec):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
210 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
211 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 self.create_class(spec)
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
214 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
215 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
216
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
217 for classname, spec in tables.items():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 if not self.classes.has_key(classname):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
219 self.drop_class(classname, tables[classname])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
220 del tables[classname]
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
221 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
223 # now upgrade the database for column type changes, new internal
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
224 # tables, etc.
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
225 save = save | self.upgrade_db()
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
226
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 # update the database version of the schema
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
228 if save:
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
229 self.save_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 # reindex the db if necessary
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232 if self.indexer.should_reindex():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
233 self.reindex()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235 # commit
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
236 self.sql_commit()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
238 # update this number when we need to make changes to the SQL structure
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
239 # of the backen database
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
240 current_db_version = 4
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
241 def upgrade_db(self):
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
242 ''' Update the SQL database to reflect changes in the backend code.
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
243
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
244 Return boolean whether we need to save the schema.
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
245 '''
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
246 version = self.database_schema.get('version', 1)
2081
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
247 if version == self.current_db_version:
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
248 # nothing to do
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
249 return 0
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
250
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
251 if version < 2:
2456
1cd69db95b23 fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents: 2429
diff changeset
252 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
253 logging.getLogger('hyperdb').info('upgrade to version 2')
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
254 # change the schema structure
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
255 self.database_schema = {'tables': self.database_schema}
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
256
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
257 # version 1 didn't have the actor column (note that in
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
258 # MySQL this will also transition the tables to typed columns)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
259 self.add_new_columns_v2()
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
260
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
261 # version 1 doesn't have the OTK, session and indexing in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
262 # database
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
263 self.create_version_2_tables()
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
264
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
265 if version < 3:
2456
1cd69db95b23 fixed some more mysql 0.6->0.7 upgrade bugs [SF#950410]
Richard Jones <richard@users.sourceforge.net>
parents: 2429
diff changeset
266 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
267 logging.getLogger('hyperdb').info('upgrade to version 3')
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
268 self.fix_version_2_tables()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
269
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
270 if version < 4:
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
271 self.fix_version_3_tables()
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
272
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
273 self.database_schema['version'] = self.current_db_version
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
274 return 1
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
275
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
276 def fix_version_3_tables(self):
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
277 # drop the shorter VARCHAR OTK column and add a new TEXT one
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
278 for name in ('otk', 'session'):
2755
e3cd28cec23d Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents: 2733
diff changeset
279 self.sql('DELETE FROM %ss'%name)
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
280 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
281 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
282
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
283 def fix_version_2_tables(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
284 '''Default (used by sqlite): NOOP'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
285 pass
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
286
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
287 def _convert_journal_tables(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
288 '''Get current journal table contents, drop the table and re-create'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
289 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
290 cols = ','.join('nodeid date tag action params'.split())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
291 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
292 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
293 sql = 'select %s from %s__journal order by date'%(cols,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
294 klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
295 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
296 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
297 self.drop_journal_table_indexes(klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
298 c.execute('drop table %s__journal'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
299
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
300 # re-create and re-populate
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
301 self.create_journal_table(klass)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
302 a = self.arg
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
303 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
304 klass.classname, cols, a, a, a, a, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
305 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
306 # no data conversion needed
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
307 self.cursor.execute(sql, row)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
308
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
309 def _convert_string_properties(self):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
310 '''Get current Class tables that contain String properties, and
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
311 convert the VARCHAR columns to TEXT'''
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
312 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
313 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
314 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
315 cols, mls = self.determine_columns(klass.properties.items())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
316 scols = ','.join([i[0] for i in cols])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
317 sql = 'select id,%s from _%s'%(scols, klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
318 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
319 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
320 self.drop_class_table_indexes(klass.classname, klass.getkey())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
321 c.execute('drop table _%s'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
322
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
323 # re-create and re-populate
2424
74474ec41050 argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents: 2416
diff changeset
324 self.create_class_table(klass, create_sequence=0)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
325 a = ','.join([self.arg for i in range(len(cols)+1)])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
326 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
327 scols, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
328 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
329 l = []
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
330 for entry in row:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
331 # mysql will already be a string - psql needs "help"
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
332 if entry is not None and not isinstance(entry, type('')):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
333 entry = str(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
334 l.append(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
335 self.cursor.execute(sql, l)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
336
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
337 def refresh_database(self):
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
338 self.post_init()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
339
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
340
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
341 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
342 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
343 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
344 else:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
345 classes = self.classes.values()
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
346 for klass in classes:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
347 if show_progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
348 for nodeid in support.Progress('Reindex %s'%klass.classname,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
349 klass.list()):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
350 klass.index(nodeid)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
351 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
352 for nodeid in klass.list():
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
353 klass.index(nodeid)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
354 self.indexer.save_index()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
355
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
356 hyperdb_to_sql_datatypes = {
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
357 hyperdb.String : 'TEXT',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
358 hyperdb.Date : 'TIMESTAMP',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
359 hyperdb.Link : 'INTEGER',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
360 hyperdb.Interval : 'VARCHAR(255)',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
361 hyperdb.Password : 'VARCHAR(255)',
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
362 hyperdb.Boolean : 'BOOLEAN',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
363 hyperdb.Number : 'REAL',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
364 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 def determine_columns(self, properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
366 ''' Figure the column names and multilink properties from the spec
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
368 "properties" is a list of (name, prop) where prop may be an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369 instance of a hyperdb "type" _or_ a string repr of that type.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370 '''
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
371 cols = [
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
372 ('_actor', self.hyperdb_to_sql_datatypes[hyperdb.Link]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
373 ('_activity', self.hyperdb_to_sql_datatypes[hyperdb.Date]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
374 ('_creator', self.hyperdb_to_sql_datatypes[hyperdb.Link]),
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
375 ('_creation', self.hyperdb_to_sql_datatypes[hyperdb.Date]),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
376 ]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
377 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378 # add the multilinks separately
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 for col, prop in properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
381 mls.append(col)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
382 continue
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
383
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
384 if isinstance(prop, type('')):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
385 raise ValueError, "string property spec!"
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
386 #and prop.find('Multilink') != -1:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
387 #mls.append(col)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
388
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
389 datatype = self.hyperdb_to_sql_datatypes[prop.__class__]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
390 cols.append(('_'+col, datatype))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
391
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
392 # Intervals stored as two columns
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
393 if isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
394 cols.append(('__'+col+'_int__', 'BIGINT'))
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
395
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 return cols, mls
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
398
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
399 def update_class(self, spec, old_spec, force=0):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400 ''' Determine the differences between the current spec and the
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
401 database version of the spec, and update where necessary.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
402
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
403 If 'force' is true, update the database anyway.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
405 new_has = spec.properties.has_key
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
406 new_spec = spec.schema()
1515
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
407 new_spec[1].sort()
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
408 old_spec[1].sort()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
409 if not force and new_spec == old_spec:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
410 # no changes
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
411 return 0
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
412
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
413 logger = logging.getLogger('hyperdb')
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
414 logger.info('update_class %s'%spec.classname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
415
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
416 logger.debug('old_spec %r'%(old_spec,))
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
417 logger.debug('new_spec %r'%(new_spec,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
418
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
419 # detect key prop change for potential index change
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
420 keyprop_changes = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
421 if new_spec[0] != old_spec[0]:
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
422 if old_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
423 keyprop_changes['remove'] = old_spec[0]
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
424 if new_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
425 keyprop_changes['add'] = new_spec[0]
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
426
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
427 # detect multilinks that have been removed, and drop their table
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
428 old_has = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
429 for name, prop in old_spec[1]:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
430 old_has[name] = 1
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
431 if new_has(name):
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
432 continue
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
433
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
434 if prop.find('Multilink to') != -1:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
435 # first drop indexes.
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
436 self.drop_multilink_table_indexes(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
437
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
438 # now the multilink table itself
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
439 sql = 'drop table %s_%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
440 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
441 # if this is the key prop, drop the index first
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
442 if old_spec[0] == prop:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
443 self.drop_class_table_key_index(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
444 del keyprop_changes['remove']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
445
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
446 # drop the column
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
447 sql = 'alter table _%s drop column _%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
448
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
449 self.sql(sql)
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
450 old_has = old_has.has_key
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
451
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
452 # if we didn't remove the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
453 # changed, we still need to remove the old index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
454 if keyprop_changes.has_key('remove'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
455 self.drop_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
456 keyprop_changes['remove'])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
457
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
458 # add new columns
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
459 for propname, prop in new_spec[1]:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
460 if old_has(propname):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
461 continue
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
462 prop = spec.properties[propname]
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
463 if isinstance(prop, Multilink):
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
464 self.create_multilink_table(spec, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
465 else:
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
466 # add the column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
467 coltype = self.hyperdb_to_sql_datatypes[prop.__class__]
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
468 sql = 'alter table _%s add column _%s %s'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
469 spec.classname, propname, coltype)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
470 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
471
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
472 # extra Interval column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
473 if isinstance(prop, Interval):
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
474 sql = 'alter table _%s add column __%s_int__ BIGINT'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
475 spec.classname, propname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
476 self.sql(sql)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
477
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
478 # if the new column is a key prop, we need an index!
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
479 if new_spec[0] == propname:
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
480 self.create_class_table_key_index(spec.classname, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
481 del keyprop_changes['add']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
482
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
483 # if we didn't add the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
484 # changed, we still need to add the new index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
485 if keyprop_changes.has_key('add'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
486 self.create_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
487 keyprop_changes['add'])
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
488
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
489 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
490
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
491 def determine_all_columns(self, spec):
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
492 """Figure out the columns from the spec and also add internal columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
493
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
494 """
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
495 cols, mls = self.determine_columns(spec.properties.items())
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
496
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
497 # add on our special columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
498 cols.append(('id', 'INTEGER PRIMARY KEY'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
499 cols.append(('__retired__', 'INTEGER DEFAULT 0'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
500 return cols, mls
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
501
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
502 def create_class_table(self, spec):
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
503 '''Create the class table for the given Class "spec". Creates the
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
504 indexes too.'''
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
505 cols, mls = self.determine_all_columns(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
506
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
507 # create the base table
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
508 scols = ','.join(['%s %s'%x for x in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
509 sql = 'create table _%s (%s)'%(spec.classname, scols)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
510 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
511
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
512 self.create_class_table_indexes(spec)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
513
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
514 return cols, mls
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
515
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
516 def create_class_table_indexes(self, spec):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
517 ''' create the class table for the given spec
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
518 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
519 # create __retired__ index
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
520 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%(
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
521 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
522 self.sql(index_sql2)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
523
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
524 # create index for key property
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
525 if spec.key:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
526 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%(
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
527 spec.classname, spec.key,
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
528 spec.classname, spec.key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
529 self.sql(index_sql3)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
530
2228
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
531 # TODO: create indexes on (selected?) Link property columns, as
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
532 # they're more likely to be used for lookup
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
533
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
534 def drop_class_table_indexes(self, cn, key):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
535 # drop the old table indexes first
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
536 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
537 if key:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
538 l.append('_%s_%s_idx'%(cn, key))
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
539
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
540 table_name = '_%s'%cn
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
541 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
542 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
543 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
544 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
545 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
546
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
547 def create_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
548 ''' create the class table for the given spec
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
549 '''
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
550 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
551 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
552
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
553 def drop_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
554 table_name = '_%s'%cn
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
555 index_name = '_%s_%s_idx'%(cn, key)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
556 if not self.sql_index_exists(table_name, index_name):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
557 return
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
558 sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
559 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
560
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
561 def create_journal_table(self, spec):
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
562 ''' create the journal table for a class given the spec and
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
563 already-determined cols
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
564 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
565 # journal table
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
566 cols = ','.join(['%s varchar'%x
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
567 for x in 'nodeid date tag action params'.split()])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
568 sql = '''create table %s__journal (
3224
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
569 nodeid integer, date %s, tag varchar(255),
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
570 action varchar(255), params text)''' % (spec.classname,
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
571 self.hyperdb_to_sql_datatypes[hyperdb.Date])
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
572 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
573 self.create_journal_table_indexes(spec)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
574
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
575 def create_journal_table_indexes(self, spec):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
576 # index on nodeid
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
577 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%(
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
578 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
579 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
580
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
581 def drop_journal_table_indexes(self, classname):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
582 index_name = '%s_journ_idx'%classname
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
583 if not self.sql_index_exists('%s__journal'%classname, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
584 return
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
585 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
586 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
587
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
588 def create_multilink_table(self, spec, ml):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
589 ''' Create a multilink table for the "ml" property of the class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
590 given by the spec
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
591 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
592 # create the table
2828
3a2f138fcb28 use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2755
diff changeset
593 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%(
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
594 spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
595 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
596 self.create_multilink_table_indexes(spec, ml)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
597
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
598 def create_multilink_table_indexes(self, spec, ml):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
599 # create index on linkid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
600 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
601 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
602 self.sql(index_sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
603
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
604 # create index on nodeid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
605 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
606 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
607 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
608
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
609 def drop_multilink_table_indexes(self, classname, ml):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
610 l = [
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
611 '%s_%s_l_idx'%(classname, ml),
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
612 '%s_%s_n_idx'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
613 ]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
614 table_name = '%s_%s'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
615 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
616 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
617 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
618 index_sql = 'drop index %s'%index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
619 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
620
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
621 def create_class(self, spec):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
622 ''' Create a database table according to the given spec.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
623 '''
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
624 cols, mls = self.create_class_table(spec)
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
625 self.create_journal_table(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
626
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
627 # now create the multilink tables
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
628 for ml in mls:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
629 self.create_multilink_table(spec, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
630
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
631 def drop_class(self, cn, spec):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
632 ''' Drop the given table from the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
633
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
634 Drop the journal and multilink tables too.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
635 '''
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
636 properties = spec[1]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
637 # figure the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
638 mls = []
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
639 for propanme, prop in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
640 if isinstance(prop, Multilink):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
641 mls.append(propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
642
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
643 # drop class table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
644 self.drop_class_table_indexes(cn, spec[0])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
645
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
646 self.drop_class_table(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
647
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
648 # drop journal table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
649 self.drop_journal_table_indexes(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
650 sql = 'drop table %s__journal'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
651 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
652
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
653 for ml in mls:
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
654 # drop multilink table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
655 self.drop_multilink_table_indexes(cn, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
656 sql = 'drop table %s_%s'%(spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
657 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
658
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
659 def drop_class_table(self, cn):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
660 sql = 'drop table _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
661 self.sql(sql)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
662
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
663 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
664 # Classes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
665 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
666 def __getattr__(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
667 ''' A convenient way of calling self.getclass(classname).
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
668 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
669 if self.classes.has_key(classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
670 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
671 raise AttributeError, classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
672
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
673 def addclass(self, cl):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
674 ''' Add a Class to the hyperdatabase.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
675 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
676 cn = cl.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
677 if self.classes.has_key(cn):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
678 raise ValueError, cn
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
679 self.classes[cn] = cl
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
680
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
681 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
682 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
683 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
684 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
685 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
686 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
687 description="User is allowed to access "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
688
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
689 def getclasses(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
690 ''' Return a list of the names of all existing classes.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
691 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
692 l = self.classes.keys()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
693 l.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
694 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
695
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
696 def getclass(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
697 '''Get the Class object representing a particular class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
698
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
699 If 'classname' is not a valid class name, a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
700 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
701 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
702 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
703 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
704 raise KeyError, 'There is no class called "%s"'%classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
705
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
706 def clear(self):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
707 '''Delete all database contents.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
708
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
709 Note: I don't commit here, which is different behaviour to the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
710 "nuke from orbit" behaviour in the dbs.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
711 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
712 logging.getLogger('hyperdb').info('clear')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
713 for cn in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
714 sql = 'delete from _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
715 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
716
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
717 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
718 # Nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
719 #
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
720
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
721 hyperdb_to_sql_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
722 hyperdb.String : str,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
723 # fractional seconds by default
3612
afda59d5d546 prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
724 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
725 hyperdb.Link : int,
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
726 hyperdb.Interval : str,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
727 hyperdb.Password : str,
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
728 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
729 hyperdb.Number : lambda x: x,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
730 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
731 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
732 def addnode(self, classname, nodeid, node):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
733 ''' Add the specified node to its class's db.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
734 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
735 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
736 logging.getLogger('hyperdb').debug('addnode %s%s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
737 nodeid, node))
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
738
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
739 # determine the column definitions and multilink tables
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
740 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
741 cols, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
742
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
743 # we'll be supplied these props if we're doing an import
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
744 values = node.copy()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
745 if not values.has_key('creator'):
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
746 # add in the "calculated" properties (dupe so we don't affect
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
747 # calling code's node assumptions)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
748 values['creation'] = values['activity'] = date.Date()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
749 values['actor'] = values['creator'] = self.getuid()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
750
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
751 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
752 props = cl.getprops(protected=1)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
753 del props['id']
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
754
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
755 # default the non-multilink columns
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
756 for col, prop in props.items():
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
757 if not values.has_key(col):
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
758 if isinstance(prop, Multilink):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
759 values[col] = []
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
760 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
761 values[col] = None
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
762
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
763 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
764 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
765 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
766 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
767 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
768
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
769 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
770 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
771 for col,dt in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
772 # this is somewhat dodgy....
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
773 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
774 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
775 value = values[col[2:-6]]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
776 # this is an Interval special "int" column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
777 if value is not None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
778 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
779 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
780 vals.append(value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
781 continue
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
782
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
783 prop = props[col[1:]]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
784 value = values[col[1:]]
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
785 if value is not None:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
786 value = self.hyperdb_to_sql_value[prop.__class__](value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
787 vals.append(value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
788 vals.append(nodeid)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
789 vals = tuple(vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
790
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
791 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
792 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
793 cols = ','.join([col for col,dt in cols]) + ',id'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
794
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
795 # perform the inserts
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
796 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
797 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
798
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
799 # insert the multilink rows
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
800 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
801 t = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
802 for entry in node[col]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
803 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
804 self.arg, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
805 self.sql(sql, (entry, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
806
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
807 def setnode(self, classname, nodeid, values, multilink_changes={}):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
808 ''' Change the specified node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
809 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
810 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
811 logging.getLogger('hyperdb').debug('setnode %s%s %r'
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
812 % (classname, nodeid, values))
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
813
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
814 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
815 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
816 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
817 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
818 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
819
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
820 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
821 props = cl.getprops()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
822
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
823 cols = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
824 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
825 # add the multilinks separately
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
826 for col in values.keys():
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
827 prop = props[col]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
828 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
829 mls.append(col)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
830 elif isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
831 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
832 cols.append(col)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
833 # extra leading '_' added by code below
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
834 cols.append('_' +col + '_int__')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
835 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
836 cols.append(col)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
837 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
838
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
839 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
840 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
841 for col in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
842 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
843 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
844 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
845 col = col[1:-6]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
846 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
847 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
848 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
849 vals.append(None)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
850 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
851 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
852 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
853 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
854 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
855 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
856 e = None
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
857 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
858 e = self.hyperdb_to_sql_value[prop.__class__](value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
859 vals.append(e)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
860
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
861 vals.append(int(nodeid))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
862 vals = tuple(vals)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
863
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
864 # if there's any updates to regular columns, do them
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
865 if cols:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
866 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
867 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols])
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
868 cols = ','.join(cols)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
869
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
870 # perform the update
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
871 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
872 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
873
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
874 # we're probably coming from an import, not a change
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
875 if not multilink_changes:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
876 for name in mls:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
877 prop = props[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
878 value = values[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
879
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
880 t = '%s_%s'%(classname, name)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
881
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
882 # clear out previous values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
883 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
884 self.sql('delete from %s where nodeid=%s'%(t, self.arg),
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
885 (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
886
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
887 # insert the values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
888 for entry in values[name]:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
889 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
890 self.arg, self.arg)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
891 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
892 self.sql(sql, (entry, nodeid))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
893
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
894 # we have multilink changes to apply
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
895 for col, (add, remove) in multilink_changes.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
896 tn = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
897 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
898 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
899 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
900 for addid in add:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
901 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
902 self.sql(sql, (int(nodeid), int(addid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
903 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
904 sql = 'delete from %s where nodeid=%s and linkid=%s'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
905 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
906 for removeid in remove:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
907 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
908 self.sql(sql, (int(nodeid), int(removeid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
909
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
910 sql_to_hyperdb_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
911 hyperdb.String : str,
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
912 hyperdb.Date : lambda x:date.Date(str(x).replace(' ', '.')),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
913 # hyperdb.Link : int, # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
914 hyperdb.Link : str,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
915 hyperdb.Interval : date.Interval,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
916 hyperdb.Password : lambda x: password.Password(encrypted=x),
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
917 hyperdb.Boolean : _bool_cvt,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
918 hyperdb.Number : _num_cvt,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
919 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
920 }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
921 def getnode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
922 ''' Get a node from the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
923 '''
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
924 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
925 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
926 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
927 # push us back to the top of the LRU
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
928 self.cache_lru.remove(key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
929 self.cache_lru.insert(0, key)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
930 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
931 self.stats['cache_hits'] += 1
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
932 # return the cached information
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
933 return self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
934
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
935 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
936 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
937 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
938
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
939 # figure the columns we're fetching
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
940 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
941 cols, mls = self.determine_columns(cl.properties.items())
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
942 scols = ','.join([col for col,dt in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
943
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
944 # perform the basic property fetch
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
945 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
946 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
947
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
948 values = self.sql_fetchone()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
949 if values is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
950 raise IndexError, 'no such %s node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
951
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
952 # make up the node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
953 node = {}
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
954 props = cl.getprops(protected=1)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
955 for col in range(len(cols)):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
956 name = cols[col][0][1:]
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
957 if name.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
958 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
959 # ignore the special Interval-as-seconds column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
960 continue
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
961 value = values[col]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
962 if value is not None:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
963 value = self.sql_to_hyperdb_value[props[name].__class__](value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
964 node[name] = value
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
965
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
966
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
967 # now the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
968 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
969 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
970 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
971 self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
972 self.cursor.execute(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
973 # extract the first column from the result
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
974 # XXX numeric ids
2887
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
975 items = [int(x[0]) for x in self.cursor.fetchall()]
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
976 items.sort ()
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
977 node[col] = [str(x) for x in items]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
978
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
979 # save off in the cache
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
980 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
981 self.cache[key] = node
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
982 # update the LRU
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
983 self.cache_lru.insert(0, key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
984 if len(self.cache_lru) > ROW_CACHE_SIZE:
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
985 del self.cache[self.cache_lru.pop()]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
986
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
987 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
988 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
989
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
990 return node
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
991
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
992 def destroynode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
993 '''Remove a node from the database. Called exclusively by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
994 destroy() method on Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
995 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
996 logging.getLogger('hyperdb').info('destroynode %s%s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
997
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
998 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
999 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1000 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1001
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1002 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1003 if self.cache.has_key((classname, nodeid)):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1004 del self.cache[(classname, nodeid)]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1005
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1006 # see if there's any obvious commit actions that we should get rid of
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1007 for entry in self.transactions[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1008 if entry[1][:2] == (classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1009 self.transactions.remove(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1010
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1011 # now do the SQL
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1012 sql = 'delete from _%s where id=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1013 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1014
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1015 # remove from multilnks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1016 cl = self.getclass(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1017 x, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1018 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1019 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1020 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
1021 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1022
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1023 # remove journal entries
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1024 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1025 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1026
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1027 def hasnode(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1028 ''' Determine if the database has a given node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1029 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1030 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1031 self.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1032 return int(self.cursor.fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1033
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1034 def countnodes(self, classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1035 ''' Count the number of nodes that exist for a particular Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1036 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1037 sql = 'select count(*) from _%s'%classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1038 self.sql(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1039 return self.cursor.fetchone()[0]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1040
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1041 def addjournal(self, classname, nodeid, action, params, creator=None,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1042 creation=None):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1043 ''' Journal the Action
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1044 'action' may be:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1045
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1046 'create' or 'set' -- 'params' is a dictionary of property values
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1047 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1048 'retire' -- 'params' is None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1049 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1050 # handle supply of the special journalling parameters (usually
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1051 # supplied on importing an existing database)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1052 if creator:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1053 journaltag = creator
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1054 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1055 journaltag = self.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1056 if creation:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1057 journaldate = creation
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1058 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1059 journaldate = date.Date()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1060
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1061 # create the journal entry
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1062 cols = 'nodeid,date,tag,action,params'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1063
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1064 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1065 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1066 nodeid, journaldate, journaltag, action, params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1067
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1068 # make the journalled data marshallable
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1069 if isinstance(params, type({})):
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1070 self._journal_marshal(params, classname)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1071
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1072 params = repr(params)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1073
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1074 dc = self.hyperdb_to_sql_value[hyperdb.Date]
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1075 journaldate = dc(journaldate)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1076
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1077 self.save_journal(classname, cols, nodeid, journaldate,
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1078 journaltag, action, params)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1079
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1080 def setjournal(self, classname, nodeid, journal):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1081 '''Set the journal to the "journal" list.'''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1082 # clear out any existing entries
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1083 self.sql('delete from %s__journal where nodeid=%s'%(classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1084 self.arg), (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1085
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1086 # create the journal entry
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1087 cols = 'nodeid,date,tag,action,params'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1088
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1089 dc = self.hyperdb_to_sql_value[hyperdb.Date]
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1090 for nodeid, journaldate, journaltag, action, params in journal:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1091 if __debug__:
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1092 logging.getLogger('hyperdb').debug('addjournal %s%s %r %s %s %r'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1093 classname, nodeid, journaldate, journaltag, action,
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1094 params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1095
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1096 # make the journalled data marshallable
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1097 if isinstance(params, type({})):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1098 self._journal_marshal(params, classname)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1099 params = repr(params)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1100
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1101 self.save_journal(classname, cols, nodeid, dc(journaldate),
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1102 journaltag, action, params)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1103
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1104 def _journal_marshal(self, params, classname):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1105 '''Convert the journal params values into safely repr'able and
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1106 eval'able values.'''
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1107 properties = self.getclass(classname).getprops()
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1108 for param, value in params.items():
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1109 if not value:
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1110 continue
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1111 property = properties[param]
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1112 cvt = self.hyperdb_to_sql_value[property.__class__]
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1113 if isinstance(property, Password):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1114 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1115 elif isinstance(property, Date):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1116 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1117 elif isinstance(property, Interval):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1118 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1119 elif isinstance(property, Boolean):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1120 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1121
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1122 def getjournal(self, classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1123 ''' get the journal for id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1124 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1125 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1126 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1127 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1128
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1129 cols = ','.join('nodeid date tag action params'.split())
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1130 journal = self.load_journal(classname, cols, nodeid)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1131
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1132 # now unmarshal the data
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1133 dc = self.sql_to_hyperdb_value[hyperdb.Date]
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1134 res = []
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1135 properties = self.getclass(classname).getprops()
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1136 for nodeid, date_stamp, user, action, params in journal:
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1137 params = eval(params)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1138 if isinstance(params, type({})):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1139 for param, value in params.items():
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1140 if not value:
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1141 continue
2731
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1142 property = properties.get(param, None)
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1143 if property is None:
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1144 # deleted property
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1145 continue
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1146 cvt = self.sql_to_hyperdb_value[property.__class__]
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1147 if isinstance(property, Password):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1148 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1149 elif isinstance(property, Date):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1150 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1151 elif isinstance(property, Interval):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1152 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1153 elif isinstance(property, Boolean):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1154 params[param] = cvt(value)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1155 # XXX numeric ids
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1156 res.append((str(nodeid), dc(date_stamp), user, action, params))
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1157 return res
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1158
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1159 def save_journal(self, classname, cols, nodeid, journaldate,
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1160 journaltag, action, params):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1161 ''' Save the journal entry to the database
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1162 '''
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1163 entry = (nodeid, journaldate, journaltag, action, params)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1164
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1165 # do the insert
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1166 a = self.arg
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1167 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1168 classname, cols, a, a, a, a, a)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1169 self.sql(sql, entry)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1170
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1171 def load_journal(self, classname, cols, nodeid):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1172 ''' Load the journal from the database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1173 '''
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1174 # now get the journal entries
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1175 sql = 'select %s from %s__journal where nodeid=%s order by date'%(
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1176 cols, classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1177 self.sql(sql, (nodeid,))
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1178 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1179
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1180 def pack(self, pack_before):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1181 ''' Delete all journal entries except "create" before 'pack_before'.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1182 '''
2416
Richard Jones <richard@users.sourceforge.net>
parents: 2413
diff changeset
1183 date_stamp = self.hyperdb_to_sql_value[Date](pack_before)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1184
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1185 # do the delete
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1186 for classname in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1187 sql = "delete from %s__journal where date<%s and "\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1188 "action<>'create'"%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1189 self.sql(sql, (date_stamp,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1190
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1191 def sql_commit(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1192 ''' Actually commit to the database.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1193 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1194 logging.getLogger('hyperdb').info('commit')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1195 self.conn.commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1196
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1197 # open a new cursor for subsequent work
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1198 self.cursor = self.conn.cursor()
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1199
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1200 def commit(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1201 ''' Commit the current transactions.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1202
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1203 Save all data changed since the database was opened or since the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1204 last commit() or rollback().
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1205 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1206 # commit the database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1207 self.sql_commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1208
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1209 # now, do all the other transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1210 for method, args in self.transactions:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1211 method(*args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1212
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1213 # save the indexer
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1214 self.indexer.save_index()
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1215
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1216 # clear out the transactions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1217 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1218
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1219 def sql_rollback(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1220 self.conn.rollback()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1221
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1222 def rollback(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1223 ''' Reverse all actions from the current transaction.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1224
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1225 Undo all the changes made since the database was opened or the last
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1226 commit() or rollback() was performed.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1227 '''
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1228 logging.getLogger('hyperdb').info('rollback')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1229
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1230 self.sql_rollback()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1231
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1232 # roll back "other" transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1233 for method, args in self.transactions:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1234 # delete temporary files
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1235 if method == self.doStoreFile:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1236 self.rollbackStoreFile(*args)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1237 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1238
1492
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1239 # clear the cache
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1240 self.clearCache()
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1241
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1242 def sql_close(self):
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1243 logging.getLogger('hyperdb').info('close')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1244 self.conn.close()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1245
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1246 def close(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1247 ''' Close off the connection.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1248 '''
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
1249 self.indexer.close()
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1250 self.sql_close()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1251
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1252 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1253 # The base Class class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1254 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1255 class Class(hyperdb.Class):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1256 ''' The handle to a particular class of nodes in a hyperdatabase.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1257
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1258 All methods except __repr__ and getnode must be implemented by a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1259 concrete backend Class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1260 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1261
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1262 def schema(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1263 ''' A dumpable version of the schema that we can store in the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1264 database
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1265 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1266 return (self.key, [(x, repr(y)) for x,y in self.properties.items()])
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1267
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1268 def enableJournalling(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1269 '''Turn journalling on for this class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1270 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1271 self.do_journal = 1
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1272
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1273 def disableJournalling(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1274 '''Turn journalling off for this class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1275 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1276 self.do_journal = 0
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1277
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1278 # Editing nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1279 def create(self, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1280 ''' Create a new node of this class and return its id.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1281
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1282 The keyword arguments in 'propvalues' map property names to values.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1283
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1284 The values of arguments must be acceptable for the types of their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1285 corresponding properties or a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1286
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1287 If this class has a key property, it must be present and its value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1288 must not collide with other key strings or a ValueError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1289
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1290 Any other properties on this class that are missing from the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1291 'propvalues' dictionary are set to None.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1292
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1293 If an id in a link or multilink property does not refer to a valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1294 node, an IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1295 '''
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1296 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1297 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1298 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1299 return newid
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1300
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1301 def create_inner(self, **propvalues):
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1302 ''' Called by create, in-between the audit and react calls.
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1303 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1304 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1305 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1306
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1307 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1308 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1309
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1310 if propvalues.has_key('creator') or propvalues.has_key('actor') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1311 propvalues.has_key('creation') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1312 raise KeyError, '"creator", "actor", "creation" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1313 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1314
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1315 # new node's id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1316 newid = self.db.newid(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1317
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1318 # validate propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1319 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1320 for key, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1321 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1322 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1323 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1324 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1325 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1326 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1327 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1328
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1329 # try to handle this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1330 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1331 prop = self.properties[key]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1332 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1333 raise KeyError, '"%s" has no property "%s"'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1334 key)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1335
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1336 if value is not None and isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1337 if type(value) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1338 raise ValueError, 'link value must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1339 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1340 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1341 if not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1342 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1343 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1344 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1345 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1346 key, value, link_class)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1347 elif not self.db.getclass(link_class).hasnode(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1348 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1349
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1350 # save off the value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1351 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1352
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1353 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1354 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1355 self.db.addjournal(link_class, value, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1356 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1357
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1358 elif isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1359 if type(value) != type([]):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1360 raise TypeError, 'new property "%s" not a list of ids'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1361
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1362 # clean up and validate the list of links
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1363 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1364 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1365 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1366 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1367 raise ValueError, '"%s" multilink value (%r) '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1368 'must contain Strings'%(key, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1369 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1370 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1371 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1372 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1373 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1374 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1375 key, entry, self.properties[key].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1376 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1377 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1378 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1379
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1380 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1381 for nodeid in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1382 if not self.db.getclass(link_class).hasnode(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1383 raise IndexError, '%s has no node %s'%(link_class,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1384 nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1385 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1386 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1387 self.db.addjournal(link_class, nodeid, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1388 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1389
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1390 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1391 if type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1392 raise TypeError, 'new property "%s" not a string'%key
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1393 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1394 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1395 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1396
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1397 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1398 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1399 raise TypeError, 'new property "%s" not a Password'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1400
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1401 elif isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1402 if value is not None and not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1403 raise TypeError, 'new property "%s" not a Date'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1404
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1405 elif isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1406 if value is not None and not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1407 raise TypeError, 'new property "%s" not an Interval'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1408
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1409 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1410 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1411 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1412 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1413 raise TypeError, 'new property "%s" not numeric'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1414
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1415 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1416 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1417 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1418 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1419 raise TypeError, 'new property "%s" not boolean'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1420
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1421 # make sure there's data where there needs to be
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1422 for key, prop in self.properties.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1423 if propvalues.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1424 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1425 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1426 raise ValueError, 'key property "%s" is required'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1427 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1428 propvalues[key] = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1429 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1430 propvalues[key] = None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1431
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1432 # done
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1433 self.db.addnode(self.classname, newid, propvalues)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1434 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1435 self.db.addjournal(self.classname, newid, ''"create", {})
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1436
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1437 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1438 return str(newid)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1439
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1440 def get(self, nodeid, propname, default=_marker, cache=1):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1441 '''Get the value of a property on an existing node of this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1442
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1443 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1444 IndexError is raised. 'propname' must be the name of a property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1445 of this class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1446
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1447 'cache' exists for backwards compatibility, and is not used.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1448 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1449 if propname == 'id':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1450 return nodeid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1451
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1452 # get the node's dict
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1453 d = self.db.getnode(self.classname, nodeid)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1454
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1455 if propname == 'creation':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1456 if d.has_key('creation'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1457 return d['creation']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1458 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1459 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1460 if propname == 'activity':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1461 if d.has_key('activity'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1462 return d['activity']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1463 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1464 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1465 if propname == 'creator':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1466 if d.has_key('creator'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1467 return d['creator']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1468 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1469 return self.db.getuid()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1470 if propname == 'actor':
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1471 if d.has_key('actor'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1472 return d['actor']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1473 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1474 return self.db.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1475
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1476 # get the property (raises KeyErorr if invalid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1477 prop = self.properties[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1478
2690
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1479 # XXX may it be that propname is valid property name
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1480 # (above error is not raised) and not d.has_key(propname)???
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1481 if (not d.has_key(propname)) or (d[propname] is None):
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
1482 if default is _marker:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1483 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1484 return []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1485 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1486 return None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1487 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1488 return default
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1489
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1490 # don't pass our list to other code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1491 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1492 return d[propname][:]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1493
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1494 return d[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1495
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1496 def set(self, nodeid, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1497 '''Modify a property on an existing node of this class.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1498
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1499 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1500 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1501
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1502 Each key in 'propvalues' must be the name of a property of this
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1503 class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1504
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1505 All values in 'propvalues' must be acceptable types for their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1506 corresponding properties or a TypeError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1507
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1508 If the value of the key property is set, it must not collide with
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1509 other key strings or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1510
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1511 If the value of a Link or Multilink property contains an invalid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1512 node id, a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1513 '''
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1514 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1515 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1516 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1517 self.fireReactors('set', nodeid, oldvalues)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1518 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1519
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1520 def set_inner(self, nodeid, **propvalues):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1521 ''' Called by set, in-between the audit and react calls.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1522 '''
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1523 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1524 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1525
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1526 if propvalues.has_key('creation') or propvalues.has_key('creator') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1527 propvalues.has_key('actor') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1528 raise KeyError, '"creation", "creator", "actor" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1529 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1530
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1531 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1532 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1533
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1534 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1535 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1536
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1537 node = self.db.getnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1538 if self.is_retired(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1539 raise IndexError, 'Requested item is retired'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1540 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1541
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1542 # make a copy of the values dictionary - we'll modify the contents
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1543 propvalues = propvalues.copy()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1544
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1545 # if the journal value is to be different, store it in here
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1546 journalvalues = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1547
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1548 # remember the add/remove stuff for multilinks, making it easier
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1549 # for the Database layer to do its stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1550 multilink_changes = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1551
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1552 for propname, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1553 # check to make sure we're not duplicating an existing key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1554 if propname == self.key and node[propname] != value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1555 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1556 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1557 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1558 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1559 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1560 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1561
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1562 # this will raise the KeyError if the property isn't valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1563 # ... we don't use getprops() here because we only care about
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1564 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1565 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1566 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1567 except KeyError:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1568 raise KeyError, '"%s" has no property named "%s"'%(
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1569 self.classname, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1570
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1571 # if the value's the same as the existing value, no sense in
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1572 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1573 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1574 if value == current:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1575 del propvalues[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1576 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1577 journalvalues[propname] = current
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1578
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1579 # do stuff based on the prop type
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1580 if isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1581 link_class = prop.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1582 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1583 if value is not None and not isinstance(value, type('')):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1584 raise ValueError, 'property "%s" link value be a string'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1585 propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1586 if isinstance(value, type('')) and not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1587 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1588 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1589 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1590 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1591 propname, value, prop.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1592
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1593 if (value is not None and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1594 not self.db.getclass(link_class).hasnode(value)):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1595 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1596
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1597 if self.do_journal and prop.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1598 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1599 if node[propname] is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1600 self.db.addjournal(link_class, node[propname],
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1601 ''"unlink", (self.classname, nodeid, propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1602
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1603 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1604 if value is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1605 self.db.addjournal(link_class, value, ''"link",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1606 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1607
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1608 elif isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1609 if type(value) != type([]):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1610 raise TypeError, 'new property "%s" not a list of'\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1611 ' ids'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1612 link_class = self.properties[propname].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1613 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1614 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1615 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1616 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1617 raise ValueError, 'new property "%s" link value ' \
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1618 'must be a string'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1619 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1620 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1621 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1622 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1623 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1624 propname, entry,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1625 self.properties[propname].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1626 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1627 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1628 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1629
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1630 # figure the journal entry for this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1631 add = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1632 remove = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1633
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1634 # handle removals
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1635 if node.has_key(propname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1636 l = node[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1637 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1638 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1639 for id in l[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1640 if id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1641 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1642 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1643 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1644 self.db.addjournal(link_class, id, 'unlink',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1645 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1646 l.remove(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1647 remove.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1648
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1649 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1650 for id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1651 if not self.db.getclass(link_class).hasnode(id):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1652 raise IndexError, '%s has no node %s'%(link_class, id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1653 if id in l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1654 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1655 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1656 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1657 self.db.addjournal(link_class, id, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1658 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1659 l.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1660 add.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1661
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1662 # figure the journal entry
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1663 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1664 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1665 l.append(('+', add))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1666 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1667 l.append(('-', remove))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1668 multilink_changes[propname] = (add, remove)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1669 if l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1670 journalvalues[propname] = tuple(l)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1671
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1672 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1673 if value is not None and type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1674 raise TypeError, 'new property "%s" not a string'%propname
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1675 if prop.indexme:
3399
3a0d4e4a0f34 merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
1676 if value is None: value = ''
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1677 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1678 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1679
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1680 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1681 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1682 raise TypeError, 'new property "%s" not a Password'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1683 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1684
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1685 elif value is not None and isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1686 if not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1687 raise TypeError, 'new property "%s" not a Date'% propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1688 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1689
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1690 elif value is not None and isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1691 if not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1692 raise TypeError, 'new property "%s" not an '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1693 'Interval'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1694 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1695
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1696 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1697 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1698 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1699 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1700 raise TypeError, 'new property "%s" not numeric'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1701
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1702 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1703 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1704 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1705 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1706 raise TypeError, 'new property "%s" not boolean'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1707
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1708 # nothing to do?
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1709 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1710 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1711
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1712 # update the activity time
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1713 propvalues['activity'] = date.Date()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1714 propvalues['actor'] = self.db.getuid()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1715
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1716 # do the set
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1717 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1718
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1719 # remove the activity props now they're handled
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1720 del propvalues['activity']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1721 del propvalues['actor']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1722
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1723 # journal the set
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1724 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1725 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1726
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1727 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1728
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1729 def retire(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1730 '''Retire a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1731
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1732 The properties on the node remain available from the get() method,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1733 and the node's id is never reused.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1734
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1735 Retired nodes are not returned by the find(), list(), or lookup()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1736 methods, and other nodes may reuse the values of their key properties.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1737 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1738 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1739 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1740
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1741 self.fireAuditors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1742
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1743 # use the arg for __retired__ to cope with any odd database type
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1744 # conversion (hello, sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1745 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1746 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1747 self.db.sql(sql, (1, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1748 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1749 self.db.addjournal(self.classname, nodeid, ''"retired", None)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1750
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1751 self.fireReactors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1752
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1753 def restore(self, nodeid):
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
1754 '''Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1755
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1756 Make node available for all operations like it was before retirement.
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1757 '''
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1758 if self.db.journaltag is None:
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1759 raise DatabaseError, 'Database open read-only'
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1760
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1761 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1762 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1763 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1764 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1765 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1766 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1767 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1768 else:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1769 raise KeyError, "Key property (%s) of retired node clashes with \
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1770 existing one (%s)" % (key, node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1771
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1772 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1773 # use the arg for __retired__ to cope with any odd database type
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1774 # conversion (hello, sqlite)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1775 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1776 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1777 self.db.sql(sql, (0, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1778 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1779 self.db.addjournal(self.classname, nodeid, ''"restored", None)
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1780
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1781 self.fireReactors('restore', nodeid, None)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1782
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1783 def is_retired(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1784 '''Return true if the node is rerired
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1785 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1786 sql = 'select __retired__ from _%s where id=%s'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1787 self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1788 self.db.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1789 return int(self.db.sql_fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1790
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1791 def destroy(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1792 '''Destroy a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1793
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1794 WARNING: this method should never be used except in extremely rare
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1795 situations where there could never be links to the node being
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1796 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1797
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1798 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1799
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1800 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1801
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1802 WARNING: really, use retire() instead
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1803
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1804 Well, I think that's enough warnings. This method exists mostly to
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1805 support the session storage of the cgi interface.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1806
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1807 The node is completely removed from the hyperdb, including all journal
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1808 entries. It will no longer be available, and will generally break code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1809 if there are any references to the node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1810 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1811 if self.db.journaltag is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1812 raise DatabaseError, 'Database open read-only'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1813 self.db.destroynode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1814
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1815 def history(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1816 '''Retrieve the journal of edits on a particular node.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1817
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1818 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1819 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1820
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1821 The returned list contains tuples of the form
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1822
1409
8dc60d87ab42 Fixed a backlog of bug reports, and worked on python 2.3 compatibility:
Richard Jones <richard@users.sourceforge.net>
parents: 1383
diff changeset
1823 (nodeid, date, tag, action, params)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1824
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1825 'date' is a Timestamp object specifying the time of the change and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1826 'tag' is the journaltag specified when the database was opened.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1827 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1828 if not self.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1829 raise ValueError, 'Journalling is disabled for this class'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1830 return self.db.getjournal(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1831
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1832 # Locating nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1833 def hasnode(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1834 '''Determine if the given nodeid actually exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1835 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1836 return self.db.hasnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1837
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1838 def setkey(self, propname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1839 '''Select a String property of this class to be the key property.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1840
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1841 'propname' must be the name of a String property of this class or
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1842 None, or a TypeError is raised. The values of the key property on
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1843 all existing nodes must be unique or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1844 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1845 prop = self.getprops()[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1846 if not isinstance(prop, String):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1847 raise TypeError, 'key properties must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1848 self.key = propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1849
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1850 def getkey(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1851 '''Return the name of the key property for this class or None.'''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1852 return self.key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1853
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1854 def lookup(self, keyvalue):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1855 '''Locate a particular node by its key property and return its id.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1856
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1857 If this class has no key property, a TypeError is raised. If the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1858 'keyvalue' matches one of the values for the key property among
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1859 the nodes in this class, the matching node's id is returned;
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1860 otherwise a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1861 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1862 if not self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1863 raise TypeError, 'No key property set for class %s'%self.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1864
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1865 # use the arg to handle any odd database type conversion (hello,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1866 # sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1867 sql = "select id from _%s where _%s=%s and __retired__ <> %s"%(
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1868 self.classname, self.key, self.db.arg, self.db.arg)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1869 self.db.sql(sql, (keyvalue, 1))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1870
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1871 # see if there was a result that's not retired
1203
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1872 row = self.db.sql_fetchone()
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1873 if not row:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1874 raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key,
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1875 keyvalue, self.classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1876
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1877 # return the id
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1878 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1879 return str(row[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1880
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1881 def find(self, **propspec):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1882 '''Get the ids of nodes in this class which link to the given nodes.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1883
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1884 'propspec' consists of keyword args propname=nodeid or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1885 propname={nodeid:1, }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1886 'propname' must be the name of a property in this class, or a
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1887 KeyError is raised. That property must be a Link or
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1888 Multilink property, or a TypeError is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1889
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1890 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1891 the nodeids will be returned. Examples::
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1892
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1893 db.issue.find(messages='1')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1894 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1895 '''
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1896 # shortcut
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1897 if not propspec:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1898 return []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1899
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1900 # validate the args
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1901 props = self.getprops()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1902 propspec = propspec.items()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1903 for propname, nodeids in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1904 # check the prop is OK
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1905 prop = props[propname]
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1906 if not isinstance(prop, Link) and not isinstance(prop, Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1907 raise TypeError, "'%s' not a Link/Multilink property"%propname
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1908
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1909 # first, links
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1910 a = self.db.arg
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1911 allvalues = ()
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1912 sql = []
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1913 where = []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1914 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1915 if not isinstance(props[prop], hyperdb.Link):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1916 continue
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1917 if type(values) is type({}) and len(values) == 1:
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1918 values = values.keys()[0]
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1919 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1920 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1921 where.append('_%s = %s'%(prop, a))
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
1922 elif values is None:
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
1923 where.append('_%s is NULL'%prop)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1924 else:
2494
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1925 values = values.keys()
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1926 s = ''
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1927 if None in values:
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1928 values.remove(None)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1929 s = '_%s is NULL or '%prop
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1930 allvalues += tuple(values)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
1931 s += '_%s in (%s)'%(prop, ','.join([a]*len(values)))
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
1932 where.append('(' + s +')')
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1933 if where:
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1934 allvalues = (1, ) + allvalues
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1935 sql.append('''select id from _%s where __retired__ <> %s
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1936 and %s'''%(self.classname, a, ' and '.join(where)))
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1937
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1938 # now multilinks
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1939 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1940 if not isinstance(props[prop], hyperdb.Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1941 continue
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1942 if not values:
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1943 continue
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1944 allvalues += (1, )
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1945 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1946 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1947 s = a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1948 else:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1949 allvalues += tuple(values.keys())
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1950 s = ','.join([a]*len(values))
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1951 tn = '%s_%s'%(self.classname, prop)
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1952 sql.append('''select id from _%s, %s where __retired__ <> %s
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1953 and id = %s.nodeid and %s.linkid in (%s)'''%(self.classname,
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1954 tn, a, tn, tn, s))
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1955
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1956 if not sql:
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
1957 return []
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
1958 sql = ' union '.join(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1959 self.db.sql(sql, allvalues)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1960 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1961 l = [str(x[0]) for x in self.db.sql_fetchall()]
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1962 return l
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1963
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1964 def stringFind(self, **requirements):
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1965 '''Locate a particular node by matching a set of its String
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1966 properties in a caseless search.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1967
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1968 If the property is not a String property, a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1969
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1970 The return is a list of the id of all nodes that match.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1971 '''
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1972 where = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1973 args = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1974 for propname in requirements.keys():
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1975 prop = self.properties[propname]
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
1976 if not isinstance(prop, String):
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1977 raise TypeError, "'%s' not a String property"%propname
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1978 where.append(propname)
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1979 args.append(requirements[propname].lower())
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1980
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1981 # generate the where clause
1549
a53a7e197360 fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents: 1539
diff changeset
1982 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where])
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1983 sql = 'select id from _%s where %s and __retired__<>%s'%(
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1984 self.classname, s, self.db.arg)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1985 args.append(1)
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
1986 self.db.sql(sql, tuple(args))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1987 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1988 l = [str(x[0]) for x in self.db.sql_fetchall()]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1989 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1990
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1991 def list(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1992 ''' Return a list of the ids of the active nodes in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1993 '''
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1994 return self.getnodeids(retired=0)
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1995
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1996 def getnodeids(self, retired=None):
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1997 ''' Retrieve all the ids of the nodes for a particular Class.
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
1998
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1999 Set retired=None to get all nodes. Otherwise it'll get all the
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2000 retired or non-retired nodes, depending on the flag.
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2001 '''
1707
3d627e34f18e sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1599
diff changeset
2002 # flip the sense of the 'retired' flag if we don't want all of them
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2003 if retired is not None:
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2004 if retired:
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2005 args = (0, )
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2006 else:
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2007 args = (1, )
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2008 sql = 'select id from _%s where __retired__ <> %s'%(self.classname,
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2009 self.db.arg)
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2010 else:
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2011 args = ()
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2012 sql = 'select id from _%s'%self.classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2013 self.db.sql(sql, args)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2014 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2015 ids = [str(x[0]) for x in self.db.cursor.fetchall()]
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2016 return ids
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2017
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2018 def _subselect(self, classname, multilink_table):
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2019 '''Create a subselect. This is factored out because some
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2020 databases (hmm only one, so far) doesn't support subselects
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2021 look for "I can't believe it's not a toy RDBMS" in the mysql
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2022 backend.
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2023 '''
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2024 return '_%s.id not in (select nodeid from %s)'%(classname,
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2025 multilink_table)
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2026
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2027 def filter(self, search_matches, filterspec, sort=[], group=[]):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2028 '''Return a list of the ids of the active nodes in this class that
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2029 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2030 sort spec
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2031
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2032 "filterspec" is {propname: value(s)}
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2033
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2034 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-'
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2035 or None and prop is a prop name or None. Note that for
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2036 backward-compatibility reasons a single (dir, prop) tuple is
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2037 also allowed.
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2038
2362
10fc45eea226 fix SearchAction use of Class.filter(), and clarify API docs for same
Richard Jones <richard@users.sourceforge.net>
parents: 2319
diff changeset
2039 "search_matches" is {nodeid: marker} or None
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2040
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2041 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2042 value to match is a list:
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2043
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2044 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2045 2. Other properties must match any of the elements in the list.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2046 '''
2578
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2047 # we can't match anything if search_matches is empty
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2048 if search_matches == {}:
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2049 return []
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2050
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2051 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2052 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2053
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2054 icn = self.classname
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2055
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2056 # vars to hold the components of the SQL statement
2586
3c67f4bfa225 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2578
diff changeset
2057 frum = [] # FROM clauses
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2058 loj = [] # LEFT OUTER JOIN clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2059 where = [] # WHERE clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2060 args = [] # *any* positional arguments
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2061 a = self.db.arg
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2062
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2063 # figure the WHERE clause from the filterspec
2256
0b198ed096af fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents: 2248
diff changeset
2064 mlfilt = 0 # are we joining with Multilink tables?
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2065 sortattr = self._sortattr (group = group, sort = sort)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2066 proptree = self._proptree(filterspec, sortattr)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2067 mlseen = 0
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2068 for pt in reversed(proptree.sortattr):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2069 p = pt
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2070 while p.parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2071 if isinstance (p.propclass, Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2072 mlseen = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2073 if mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2074 p.sort_ids_needed = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2075 p.tree_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2076 p = p.parent
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2077 if not mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2078 pt.attr_sort_done = pt.tree_sort_done = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2079 proptree.compute_sort_done()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2080
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2081 orderby = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2082 ordercols = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2083 auxcols = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2084 mlsort = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2085 rhsnum = 0
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2086 for p in proptree:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2087 oc = None
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2088 cn = p.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2089 ln = p.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2090 pln = p.parent.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2091 pcn = p.parent.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2092 k = p.name
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2093 v = p.val
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2094 propclass = p.propclass
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2095 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2096 oc = ac = '_%s._%s'%(pln, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2097 if isinstance(propclass, Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2098 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2099 mlfilt = 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2100 tn = '%s_%s'%(pcn, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2101 if v in ('-1', ['-1']):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2102 # only match rows that have count(linkid)=0 in the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2103 # corresponding multilink table)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2104 where.append(self._subselect(pcn, tn))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2105 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2106 frum.append(tn)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2107 where.append('_%s.id=%s.nodeid'%(pln,tn))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2108 if p.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2109 frum.append('_%s as _%s' % (cn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2110 where.append('%s.linkid=_%s.id'%(tn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2111 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2112 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2113 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2114 where.append('%s.linkid in (%s)'%(tn, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2115 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2116 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2117 where.append('%s.linkid=%s'%(tn, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2118 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2119 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2120 assert not p.attr_sort_done and not p.sort_ids_needed
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2121 elif k == 'id':
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2122 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2123 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2124 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2125 where.append('_%s.%s in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2126 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2127 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2128 where.append('_%s.%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2129 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2130 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2131 oc = ac = '_%s.id'%pln
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2132 elif isinstance(propclass, String) and p.sort_type < 2:
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2133 if not isinstance(v, type([])):
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2134 v = [v]
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2135
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2136 # Quote the bits in the string that need it and then embed
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2137 # in a "substring" search. Note - need to quote the '%' so
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2138 # they make it through the python layer happily
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2139 v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v]
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2140
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2141 # now add to the where clause
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2142 where.append('('
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2143 +' and '.join(["_%s._%s LIKE '%s'"%(pln, k, s) for s in v])
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2144 +')')
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2145 # note: args are embedded in the query string now
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2146 elif isinstance(propclass, Link):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2147 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2148 if p.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2149 frum.append('_%s as _%s' % (cn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2150 where.append('_%s._%s=_%s.id'%(pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2151 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2152 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2153 d = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2154 for entry in v:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2155 if entry == '-1':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2156 entry = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2157 d[entry] = entry
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2158 l = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2159 if d.has_key(None) or not d:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2160 del d[None]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2161 l.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2162 if d:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2163 v = d.keys()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2164 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2165 l.append('(_%s._%s in (%s))'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2166 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2167 if l:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2168 where.append('(' + ' or '.join(l) +')')
3636
fa7becc62534 Of course for Links there is the same bug as I fixed before for multilinks.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3635
diff changeset
2169 else:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2170 if v in ('-1', None):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2171 v = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2172 where.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2173 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2174 where.append('_%s._%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2175 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2176 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2177 lp = p.cls.labelprop()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2178 oc = ac = '_%s._%s'%(pln, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2179 if lp != 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2180 if p.tree_sort_done and (p.sort_type == 2 or not
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2181 p.children):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2182 loj.append(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2183 'LEFT OUTER JOIN _%s as _%s on _%s._%s=_%s.id'%(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2184 cn, ln, pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2185 oc = '_%s._%s'%(ln, lp)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2186 elif isinstance(propclass, Date) and p.sort_type < 2:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2187 dc = self.db.hyperdb_to_sql_value[hyperdb.Date]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2188 if isinstance(v, type([])):
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2189 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2190 where.append('_%s._%s in (%s)'%(pln, k, s))
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2191 args = args + [dc(date.Date(x)) for x in v]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2192 else:
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2193 try:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2194 # Try to filter on range of dates
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3554
diff changeset
2195 date_rng = propclass.range_from_raw(v, self.db)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2196 if date_rng.from_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2197 where.append('_%s._%s >= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2198 args.append(dc(date_rng.from_value))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2199 if date_rng.to_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2200 where.append('_%s._%s <= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2201 args.append(dc(date_rng.to_value))
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2202 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2203 # If range creation fails - ignore that search parameter
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2204 pass
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2205 elif isinstance(propclass, Interval):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2206 # filter/sort using the __<prop>_int__ column
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2207 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2208 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2209 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2210 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2211 args = args + [date.Interval(x).as_seconds() for x in v]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2212 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2213 try:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2214 # Try to filter on range of intervals
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2215 date_rng = Range(v, date.Interval)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2216 if date_rng.from_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2217 where.append('_%s.__%s_int__ >= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2218 args.append(date_rng.from_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2219 if date_rng.to_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2220 where.append('_%s.__%s_int__ <= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2221 args.append(date_rng.to_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2222 except ValueError:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2223 # If range creation fails - ignore search parameter
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2224 pass
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2225 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2226 oc = ac = '_%s.__%s_int__'%(pln,k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2227 elif p.sort_type < 2:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2228 if isinstance(v, type([])):
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2229 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2230 where.append('_%s._%s in (%s)'%(pln, k, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2231 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2232 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2233 where.append('_%s._%s=%s'%(pln, k, a))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2234 args.append(v)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2235 if oc:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2236 if p.sort_ids_needed:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2237 auxcols[ac] = p
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2238 if p.tree_sort_done and p.sort_direction:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2239 # Don't select top-level id twice
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2240 if p.name != 'id' or p.parent != proptree:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2241 ordercols.append(oc)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2242 if p.sort_direction == '-':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2243 oc += ' desc'
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2244 orderby.append(oc)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2245
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2246 props = self.getprops()
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2247
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2248 # don't match retired nodes
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2249 where.append('_%s.__retired__ <> 1'%icn)
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2250
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2251 # add results of full text search
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2252 if search_matches is not None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2253 v = search_matches.keys()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2254 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2255 where.append('_%s.id in (%s)'%(icn, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2256 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2257
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2258 # construct the SQL
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2259 frum.append('_'+icn)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2260 frum = ','.join(frum)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2261 if where:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2262 where = ' where ' + (' and '.join(where))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2263 else:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2264 where = ''
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2265 if mlfilt:
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2266 # we're joining tables on the id, so we will get dupes if we
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2267 # don't distinct()
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2268 cols = ['distinct(_%s.id)'%icn]
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2269 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2270 cols = ['_%s.id'%icn]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2271 if orderby:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2272 cols = cols + ordercols
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2273 order = ' order by %s'%(','.join(orderby))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2274 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2275 order = ''
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2276 for o, p in auxcols.iteritems ():
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2277 cols.append (o)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2278 p.auxcol = len (cols) - 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2279
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2280 cols = ','.join(cols)
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2281 loj = ' '.join(loj)
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2282 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2283 args = tuple(args)
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2284 __traceback_info__ = (sql, args)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2285 self.db.sql(sql, args)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
2286 l = self.db.sql_fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2287
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2288 # Compute values needed for sorting in proptree.sort
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2289 for p in auxcols.itervalues():
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2290 p.sort_ids = p.sort_result = [row[p.auxcol] for row in l]
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2291 # return the IDs (the first column)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2292 # XXX numeric ids
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2293 l = [str(row[0]) for row in l]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2294 l = proptree.sort (l)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2295
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2296 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2297 self.db.stats['filtering'] += (time.time() - start_t)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2298 return l
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2299
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2300 def count(self):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2301 '''Get the number of nodes in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2302
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2303 If the returned integer is 'numnodes', the ids of all the nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2304 in this class run from 1 to numnodes, and numnodes+1 will be the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2305 id of the next node to be created in this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2306 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2307 return self.db.countnodes(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2308
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2309 # Manipulating properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2310 def getprops(self, protected=1):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2311 '''Return a dictionary mapping property names to property objects.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2312 If the "protected" flag is true, we include protected properties -
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2313 those which may not be modified.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2314 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2315 d = self.properties.copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2316 if protected:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2317 d['id'] = String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2318 d['creation'] = hyperdb.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2319 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1175
diff changeset
2320 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2321 d['actor'] = hyperdb.Link('user')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2322 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2323
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2324 def addprop(self, **properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2325 '''Add properties to this class.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2326
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2327 The keyword arguments in 'properties' must map names to property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2328 objects, or a TypeError is raised. None of the keys in 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2329 may collide with the names of existing properties, or a ValueError
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2330 is raised before any properties have been added.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2331 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2332 for key in properties.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2333 if self.properties.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2334 raise ValueError, key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2335 self.properties.update(properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2336
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2337 def index(self, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2338 '''Add (or refresh) the node to search indexes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2339 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2340 # find all the String properties that have indexme
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2341 for prop, propclass in self.getprops().items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2342 if isinstance(propclass, String) and propclass.indexme:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2343 self.db.indexer.add_text((self.classname, nodeid, prop),
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2344 str(self.get(nodeid, prop)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2345
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2346 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2347 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2348 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2349 def export_list(self, propnames, nodeid):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2350 ''' Export a node - generate a list of CSV-able data in the order
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2351 specified by propnames for the given node.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2352 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2353 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2354 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2355 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2356 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2357 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2358 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2359 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2360 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2361 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2362 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2363 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2364 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2365 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2366 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2367 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2368 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2369 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2370
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2371 def import_list(self, propnames, proplist):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2372 ''' Import a node - all information including "id" is present and
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2373 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2374 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2375 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2376
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2377 Return the nodeid of the node imported.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2378 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2379 if self.db.journaltag is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2380 raise DatabaseError, 'Database open read-only'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2381 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2382
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2383 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2384 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2385 retire = 0
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2386 if not "id" in propnames:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2387 newid = self.db.newid(self.classname)
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2388 else:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2389 newid = eval(proplist[propnames.index("id")])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2390 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2391 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2392 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2393
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2394 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2395 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2396
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2397 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2398 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2399 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2400 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2401 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2402 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2403 retire = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2404 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2405 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2406 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2407 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2408
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2409 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2410 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2411 # don't set Nones
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2412 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2413 elif isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2414 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2415 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2416 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2417 elif isinstance(prop, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2418 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2419 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2420 value = pwd
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2421 elif isinstance(prop, String):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2422 if isinstance(value, unicode):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2423 value = value.encode('utf8')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2424 if not isinstance(value, str):
2691
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2425 raise TypeError, \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2426 'new property "%(propname)s" not a string: %(value)r' \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2427 % locals()
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2428 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2429 self.db.indexer.add_text((self.classname, newid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2430 value)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2431 d[propname] = value
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2432
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2433 # get a new id if necessary
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2434 if newid is None:
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2435 newid = self.db.newid(self.classname)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2436
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2437 # insert new node or update existing?
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2438 if not self.hasnode(newid):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2439 self.db.addnode(self.classname, newid, d) # insert
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2440 else:
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2441 self.db.setnode(self.classname, newid, d) # update
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2442
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2443 # retire?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2444 if retire:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2445 # use the arg for __retired__ to cope with any odd database type
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2446 # conversion (hello, sqlite)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2447 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2448 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2449 self.db.sql(sql, (1, newid))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2450 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2451
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2452 def export_journals(self):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2453 '''Export a class's journal - generate a list of lists of
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2454 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2455
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2456 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2457
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2458 No heading here - the columns are fixed.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2459 '''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2460 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2461 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2462 for nodeid in self.getnodeids():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2463 for nodeid, date, user, action, params in self.history(nodeid):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2464 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2465 if action == 'set':
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2466 export_data = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2467 for propname, value in params.items():
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2468 if not properties.has_key(propname):
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2469 # property no longer in the schema
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2470 continue
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2471
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2472 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2473 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2474 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2475 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2476 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2477 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2478 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2479 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2480 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2481 value = str(value)
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2482 export_data[propname] = value
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2483 params = export_data
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2484 l = [nodeid, date, user, action, params]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2485 r.append(map(repr, l))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2486 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2487
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2488 def import_journals(self, entries):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2489 '''Import a class's journal.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2490
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2491 Uses setjournal() to set the journal for each item.'''
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2492 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2493 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2494 for l in entries:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2495 l = map(eval, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2496 nodeid, jdate, user, action, params = l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2497 r = d.setdefault(nodeid, [])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2498 if action == 'set':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2499 for propname, value in params.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2500 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2501 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2502 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2503 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2504 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2505 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2506 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2507 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2508 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2509 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2510 value = pwd
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2511 params[propname] = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2512 r.append((nodeid, date.Date(jdate), user, action, params))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2513
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2514 for nodeid, l in d.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2515 self.db.setjournal(self.classname, nodeid, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2516
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2586
diff changeset
2517 class FileClass(hyperdb.FileClass, Class):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2518 '''This class defines a large chunk of data. To support this, it has a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2519 mandatory String property "content" which is typically saved off
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2520 externally to the hyperdb.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2521
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2522 The default MIME type of this data is defined by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2523 "default_mime_type" class attribute, which may be overridden by each
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2524 node if the class defines a "type" String property.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2525 '''
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2526 def __init__(self, db, classname, **properties):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2527 '''The newly-created class automatically includes the "content"
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2528 and "type" properties.
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2529 '''
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2530 if not properties.has_key('content'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2531 properties['content'] = hyperdb.String(indexme='yes')
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2532 if not properties.has_key('type'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2533 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2534 Class.__init__(self, db, classname, **properties)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2535
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2536 def create(self, **propvalues):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2537 ''' snaffle the file propvalue and store in a file
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2538 '''
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2539 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2540 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2541 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2542
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2543 # now remove the content property so it's not stored in the db
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2544 content = propvalues['content']
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2545 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2546
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2547 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2548 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2549
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2550 # figure the mime type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2551 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2552
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2553 # and index!
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2554 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2555 self.db.indexer.add_text((self.classname, newid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2556 content, mime_type)
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2557
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2558 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2559 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2560
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2561 # store off the content as a file
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2562 self.db.storefile(self.classname, newid, None, content)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2563 return newid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2564
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2565 def get(self, nodeid, propname, default=_marker, cache=1):
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2566 ''' Trap the content propname and get it from the file
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2567
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2568 'cache' exists for backwards compatibility, and is not used.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2569 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2570 poss_msg = 'Possibly a access right configuration problem.'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2571 if propname == 'content':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2572 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2573 return self.db.getfile(self.classname, nodeid, None)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2574 except IOError, (strerror):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2575 # BUG: by catching this we donot see an error in the log.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2576 return 'ERROR reading file: %s%s\n%s\n%s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2577 self.classname, nodeid, poss_msg, strerror)
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2578 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2579 return Class.get(self, nodeid, propname, default)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2580 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2581 return Class.get(self, nodeid, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2582
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2583 def getprops(self, protected=1):
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2584 '''In addition to the actual properties on the node, these methods
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2585 provide the "content" property. If the "protected" flag is true,
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2586 we include protected properties - those which may not be
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2587 modified.
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2588
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2589 Note that the content prop is indexed separately, hence no indexme.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2590 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2591 d = Class.getprops(self, protected=protected).copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2592 d['content'] = hyperdb.String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2593 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2594
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2595 def set(self, itemid, **propvalues):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2596 ''' Snarf the "content" propvalue and update it in a file
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2597 '''
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2598 self.fireAuditors('set', itemid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2599 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2600
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2601 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2602 content = None
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2603 if propvalues.has_key('content'):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2604 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2605 del propvalues['content']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2606
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2607 # do the database create
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2608 propvalues = self.set_inner(itemid, **propvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2609
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2610 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2611 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2612 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2613 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2614 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2615 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2616 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2617 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2618 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2619
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2620 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2621 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2622 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2623
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2624 def index(self, nodeid):
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2625 ''' Add (or refresh) the node to search indexes.
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2626
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2627 Use the content-type property for the content property.
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2628 '''
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2629 # find all the String properties that have indexme
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2630 for prop, propclass in self.getprops().items():
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2631 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2632 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2633 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2634 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2635 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2636 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2637 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2638 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2639 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2640 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2641 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2642 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2643
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2644 # XXX deviation from spec - was called ItemClass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2645 class IssueClass(Class, roundupdb.IssueClass):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2646 # Overridden methods:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2647 def __init__(self, db, classname, **properties):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2648 '''The newly-created class automatically includes the "messages",
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2649 "files", "nosy", and "superseder" properties. If the 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2650 dictionary attempts to specify any of these properties or a
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2651 "creation", "creator", "activity" or "actor" property, a ValueError
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2652 is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2653 '''
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2654 if not properties.has_key('title'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2655 properties['title'] = hyperdb.String(indexme='yes')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2656 if not properties.has_key('messages'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2657 properties['messages'] = hyperdb.Multilink("msg")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2658 if not properties.has_key('files'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2659 properties['files'] = hyperdb.Multilink("file")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2660 if not properties.has_key('nosy'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2661 # note: journalling is turned off as it really just wastes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2662 # space. this behaviour may be overridden in an instance
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2663 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2664 if not properties.has_key('superseder'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2665 properties['superseder'] = hyperdb.Multilink(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2666 Class.__init__(self, db, classname, **properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2667
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2668 # vim: set et sts=4 sw=4 :

Roundup Issue Tracker: http://roundup-tracker.org/