Mercurial > p > roundup > code

annotate roundup/backends/rdbms_common.py @ 4060:2a68d7494bbc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Robustify SQL<->HyperDB data type conversion.
author Stefan Seefeld <stefan@seefeld.name>
date Fri, 20 Feb 2009 04:50:39 +0000
parents ef0b4396888a
children 52431f2ad76b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
1 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
4 # under the same terms as Python, so long as this copyright message and
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
5 # disclaimer are retained in their original form.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
6 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
11 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
17 #
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
18 """ Relational database (SQL) backend common code.
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
19
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
20 Basics:
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
21
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
22 - map roundup classes to relational tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
23 - automatically detect schema changes and modify the table schemas
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
24 appropriately (we store the "database version" of the schema in the
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
25 database itself as the only row of the "schema" table)
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
26 - multilinks (which represent a many-to-many relationship) are handled through
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
27 intermediate tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
28 - journals are stored adjunct to the per-class tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
29 - table names and columns have "_" prepended so the names can't clash with
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
30 restricted names (like "order")
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
31 - retirement is determined by the __retired__ column being > 0
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
32
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
33 Database-specific changes may generally be pushed out to the overridable
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
34 sql_* methods, since everything else should be fairly generic. There's
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
35 probably a bit of work to be done if a database is used that actually
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
36 honors column typing, since the initial databases don't (sqlite stores
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
37 everything as a string.)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
38
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
39 The schema of the hyperdb being mapped to the database is stored in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
40 database itself as a repr()'ed dictionary of information about each Class
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
41 that maps to a table. If that information differs from the hyperdb schema,
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
42 then we update it. We also store in the schema dict a version which
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
43 allows us to upgrade the database schema when necessary. See upgrade_db().
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
44
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
45 To force a unqiueness constraint on the key properties we put the item
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
46 id into the __retired__ column duing retirement (so it's 0 for "active"
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
47 items) and place a unqiueness constraint on key + __retired__. This is
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
48 particularly important for the users class where multiple users may
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
49 try to have the same username, with potentially many retired users with
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
50 the same name.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
51 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
52 __docformat__ = 'restructuredtext'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 # standard python modules
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
55 import sys, os, time, re, errno, weakref, copy, logging
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 # roundup modules
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
58 from roundup import hyperdb, date, password, roundupdb, security, support
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 from roundup.hyperdb import String, Password, Date, Interval, Link, \
1417
472c21af7f69 fixed error in indexargs_url (thanks Patrick Ohly)
Richard Jones <richard@users.sourceforge.net>
parents: 1415
diff changeset
60 Multilink, DatabaseError, Boolean, Number, Node
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
61 from roundup.backends import locking
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
62 from roundup.support import reversed
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
63 from roundup.i18n import _
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 # support
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 from blobfiles import FileStorage
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
67 try:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
68 from indexer_xapian import Indexer
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
69 except ImportError:
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
70 from indexer_rdbms import Indexer
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
71 from sessions_rdbms import Sessions, OneTimeKeys
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
72 from roundup.date import Range
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
74 # number of rows to keep in memory
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
75 ROW_CACHE_SIZE = 100
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
76
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
77 # dummy value meaning "argument not passed"
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
78 _marker = []
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
79
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
80 def _num_cvt(num):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
81 num = str(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
82 try:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
83 return int(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
84 except:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
85 return float(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
86
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
87 def _bool_cvt(value):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
88 if value in ('TRUE', 'FALSE'):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
89 return {'TRUE': 1, 'FALSE': 0}[value]
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
90 # assume it's a number returned from the db API
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
91 return int(value)
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
92
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
93 def connection_dict(config, dbnamestr=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
94 """ Used by Postgresql and MySQL to detemine the keyword args for
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
95 opening the database connection."""
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
96 d = { }
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
97 if dbnamestr:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
98 d[dbnamestr] = config.RDBMS_NAME
3099
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
99 for name in ('host', 'port', 'password', 'user', 'read_default_group',
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
100 'read_default_file'):
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
101 cvar = 'RDBMS_'+name.upper()
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
102 if config[cvar] is not None:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
103 d[name] = config[cvar]
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
104 return d
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
105
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
107 """ Wrapper around an SQL database that presents a hyperdb interface.
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
108
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
109 - some functionality is specific to the actual SQL database, hence
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
110 the sql_* methods that are NotImplemented
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
111 - we keep a cache of the latest ROW_CACHE_SIZE row fetches.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
112 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 def __init__(self, config, journaltag=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
114 """ Open the database and load the schema from it.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
115 """
3897
e2c2d91932ad respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3872
diff changeset
116 FileStorage.__init__(self, config.UMASK)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 self.config, self.journaltag = config, journaltag
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 self.dir = config.DATABASE
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 self.classes = {}
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
120 self.indexer = Indexer(self)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 self.security = security.Security(self)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 # additional transaction support for external files and the like
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
126 # keep a cache of the N most recently retrieved rows of any kind
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
127 # (classname, nodeid) = row
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
128 self.cache = {}
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
129 self.cache_lru = []
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
130 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
131 'filtering': 0}
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
132
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
133 # database lock
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
134 self.lockfile = None
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
135
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 # open a connection to the database, creating the "conn" attribute
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
137 self.open_connection()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
139 def clearCache(self):
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
140 self.cache = {}
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
141 self.cache_lru = []
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
142
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
143 def getSessionManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
144 return Sessions(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
145
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
146 def getOTKManager(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
147 return OneTimeKeys(self)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
148
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
149 def open_connection(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
150 """ Open a connection to the database, creating it if necessary.
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
151
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
152 Must call self.load_dbschema()
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
153 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 raise NotImplemented
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
156 def sql(self, sql, args=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
157 """ Execute the sql with the optional args.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
158 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
159 self.log_debug('SQL %r %r'%(sql, args))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 if args:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
161 self.cursor.execute(sql, args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
162 else:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
163 self.cursor.execute(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
165 def sql_fetchone(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
166 """ Fetch a single row. If there's nothing to fetch, return None.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
167 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
168 return self.cursor.fetchone()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
169
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
170 def sql_fetchall(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
171 """ Fetch all rows. If there's nothing to fetch, return [].
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
172 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
173 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
175 def sql_stringquote(self, value):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
176 """ Quote the string so it's safe to put in the 'sql quotes'
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
177 """
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
178 return re.sub("'", "''", str(value))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
179
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
180 def init_dbschema(self):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
181 self.database_schema = {
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
182 'version': self.current_db_version,
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
183 'tables': {}
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
184 }
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
185
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
186 def load_dbschema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
187 """ Load the schema definition that the database currently implements
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
188 """
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
189 self.cursor.execute('select schema from schema')
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
190 schema = self.cursor.fetchone()
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
191 if schema:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
192 self.database_schema = eval(schema[0])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
193 else:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
194 self.database_schema = {}
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
195
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
196 def save_dbschema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
197 """ Save the schema definition that the database currently implements
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
198 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
199 s = repr(self.database_schema)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
200 self.sql('delete from schema')
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3694
diff changeset
201 self.sql('insert into schema values (%s)'%self.arg, (s,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203 def post_init(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
204 """ Called once the schema initialisation has finished.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 We should now confirm that the schema defined by our "classes"
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
207 attribute actually matches the schema in the database.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
208 """
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
209 save = 0
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
210
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
211 # handle changes in the schema
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
212 tables = self.database_schema['tables']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 for classname, spec in self.classes.items():
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
214 if tables.has_key(classname):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
215 dbspec = tables[classname]
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
216 if self.update_class(spec, dbspec):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
217 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
218 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
219 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 self.create_class(spec)
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
221 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
222 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
224 for classname, spec in tables.items():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225 if not self.classes.has_key(classname):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
226 self.drop_class(classname, tables[classname])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
227 del tables[classname]
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
228 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
229
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
230 # now upgrade the database for column type changes, new internal
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
231 # tables, etc.
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
232 save = save | self.upgrade_db()
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
233
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 # update the database version of the schema
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
235 if save:
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
236 self.save_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 # reindex the db if necessary
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 if self.indexer.should_reindex():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 self.reindex()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242 # commit
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
243 self.sql_commit()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
244
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
245 # update this number when we need to make changes to the SQL structure
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
246 # of the backen database
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
247 current_db_version = 5
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
248 db_version_updated = False
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
249 def upgrade_db(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
250 """ Update the SQL database to reflect changes in the backend code.
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
251
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
252 Return boolean whether we need to save the schema.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
253 """
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
254 version = self.database_schema.get('version', 1)
3993
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
255 if version > self.current_db_version:
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
256 raise DatabaseError('attempting to run rev %d DATABASE with rev '
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
257 '%d CODE!'%(version, self.current_db_version))
2081
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
258 if version == self.current_db_version:
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
259 # nothing to do
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
260 return 0
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
261
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
262 if version < 2:
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
263 log_info('upgrade to version 2')
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
264 # change the schema structure
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
265 self.database_schema = {'tables': self.database_schema}
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
266
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
267 # version 1 didn't have the actor column (note that in
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
268 # MySQL this will also transition the tables to typed columns)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
269 self.add_new_columns_v2()
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
270
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
271 # version 1 doesn't have the OTK, session and indexing in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
272 # database
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
273 self.create_version_2_tables()
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
274
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
275 if version < 3:
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
276 log_info('upgrade to version 3')
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
277 self.fix_version_2_tables()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
278
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
279 if version < 4:
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
280 self.fix_version_3_tables()
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
281
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
282 if version < 5:
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
283 self.fix_version_4_tables()
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
284
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
285 self.database_schema['version'] = self.current_db_version
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
286 self.db_version_updated = True
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
287 return 1
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
288
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
289 def fix_version_3_tables(self):
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
290 # drop the shorter VARCHAR OTK column and add a new TEXT one
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
291 for name in ('otk', 'session'):
2755
e3cd28cec23d Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents: 2733
diff changeset
292 self.sql('DELETE FROM %ss'%name)
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
293 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
294 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
295
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
296 def fix_version_2_tables(self):
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
297 # Default (used by sqlite): NOOP
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
298 pass
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
299
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
300 def fix_version_4_tables(self):
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
301 # note this is an explicit call now
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
302 c = self.cursor
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
303 for cn, klass in self.classes.items():
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
304 c.execute('select id from _%s where __retired__<>0'%(cn,))
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
305 for (id,) in c.fetchall():
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
306 c.execute('update _%s set __retired__=%s where id=%s'%(cn,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
307 self.arg, self.arg), (id, id))
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
308
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
309 if klass.key:
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
310 self.add_class_key_required_unique_constraint(cn, klass.key)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
311
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
312 def _convert_journal_tables(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
313 """Get current journal table contents, drop the table and re-create"""
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
314 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
315 cols = ','.join('nodeid date tag action params'.split())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
316 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
317 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
318 sql = 'select %s from %s__journal order by date'%(cols,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
319 klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
320 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
321 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
322 self.drop_journal_table_indexes(klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
323 c.execute('drop table %s__journal'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
324
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
325 # re-create and re-populate
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
326 self.create_journal_table(klass)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
327 a = self.arg
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
328 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
329 klass.classname, cols, a, a, a, a, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
330 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
331 # no data conversion needed
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
332 self.cursor.execute(sql, row)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
333
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
334 def _convert_string_properties(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
335 """Get current Class tables that contain String properties, and
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
336 convert the VARCHAR columns to TEXT"""
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
337 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
338 for klass in self.classes.values():
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
339 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
340 cols, mls = self.determine_columns(klass.properties.items())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
341 scols = ','.join([i[0] for i in cols])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
342 sql = 'select id,%s from _%s'%(scols, klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
343 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
344 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
345 self.drop_class_table_indexes(klass.classname, klass.getkey())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
346 c.execute('drop table _%s'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
347
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
348 # re-create and re-populate
2424
74474ec41050 argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents: 2416
diff changeset
349 self.create_class_table(klass, create_sequence=0)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
350 a = ','.join([self.arg for i in range(len(cols)+1)])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
351 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
352 scols, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
353 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
354 l = []
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
355 for entry in row:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
356 # mysql will already be a string - psql needs "help"
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
357 if entry is not None and not isinstance(entry, type('')):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
358 entry = str(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
359 l.append(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
360 self.cursor.execute(sql, l)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
361
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
362 def refresh_database(self):
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
363 self.post_init()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
364
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
365
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
366 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
367 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
368 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
369 else:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
370 classes = self.classes.values()
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
371 for klass in classes:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
372 if show_progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
373 for nodeid in support.Progress('Reindex %s'%klass.classname,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
374 klass.list()):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
375 klass.index(nodeid)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
376 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
377 for nodeid in klass.list():
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
378 klass.index(nodeid)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
379 self.indexer.save_index()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
381 hyperdb_to_sql_datatypes = {
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
382 hyperdb.String : 'TEXT',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
383 hyperdb.Date : 'TIMESTAMP',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
384 hyperdb.Link : 'INTEGER',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
385 hyperdb.Interval : 'VARCHAR(255)',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
386 hyperdb.Password : 'VARCHAR(255)',
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
387 hyperdb.Boolean : 'BOOLEAN',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
388 hyperdb.Number : 'REAL',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
389 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
390
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
391 def hyperdb_to_sql_datatype(self, propclass):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
392
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
393 datatype = self.hyperdb_to_sql_datatypes.get(propclass)
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
394 if datatype:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
395 return datatype
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
396
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
397 for k, v in self.hyperdb_to_sql_datatypes.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
398 if issubclass(propclass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
399 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
400
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
401 raise ValueError, '%r is not a hyperdb property class' % propclass
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
402
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 def determine_columns(self, properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
404 """ Figure the column names and multilink properties from the spec
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
405
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
406 "properties" is a list of (name, prop) where prop may be an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
407 instance of a hyperdb "type" _or_ a string repr of that type.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
408 """
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
409 cols = [
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
410 ('_actor', self.hyperdb_to_sql_datatype(hyperdb.Link)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
411 ('_activity', self.hyperdb_to_sql_datatype(hyperdb.Date)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
412 ('_creator', self.hyperdb_to_sql_datatype(hyperdb.Link)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
413 ('_creation', self.hyperdb_to_sql_datatype(hyperdb.Date)),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
414 ]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
415 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
416 # add the multilinks separately
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
417 for col, prop in properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
418 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
419 mls.append(col)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
420 continue
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
421
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
422 if isinstance(prop, type('')):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
423 raise ValueError, "string property spec!"
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
424 #and prop.find('Multilink') != -1:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
425 #mls.append(col)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
426
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
427 datatype = self.hyperdb_to_sql_datatype(prop.__class__)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
428 cols.append(('_'+col, datatype))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
429
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
430 # Intervals stored as two columns
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
431 if isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
432 cols.append(('__'+col+'_int__', 'BIGINT'))
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
433
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
434 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
435 return cols, mls
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
436
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
437 def update_class(self, spec, old_spec, force=0):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
438 """ Determine the differences between the current spec and the
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
439 database version of the spec, and update where necessary.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
440
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
441 If 'force' is true, update the database anyway.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
442 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
443 new_has = spec.properties.has_key
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
444 new_spec = spec.schema()
1515
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
445 new_spec[1].sort()
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
446 old_spec[1].sort()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
447 if not force and new_spec == old_spec:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
448 # no changes
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
449 return 0
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
450
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
451 logger = logging.getLogger('hyperdb')
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
452 logger.info('update_class %s'%spec.classname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
453
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
454 logger.debug('old_spec %r'%(old_spec,))
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
455 logger.debug('new_spec %r'%(new_spec,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
456
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
457 # detect key prop change for potential index change
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
458 keyprop_changes = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
459 if new_spec[0] != old_spec[0]:
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
460 if old_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
461 keyprop_changes['remove'] = old_spec[0]
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
462 if new_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
463 keyprop_changes['add'] = new_spec[0]
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
464
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
465 # detect multilinks that have been removed, and drop their table
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
466 old_has = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
467 for name, prop in old_spec[1]:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
468 old_has[name] = 1
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
469 if new_has(name):
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
470 continue
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
471
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
472 if prop.find('Multilink to') != -1:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
473 # first drop indexes.
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
474 self.drop_multilink_table_indexes(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
475
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
476 # now the multilink table itself
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
477 sql = 'drop table %s_%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
478 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
479 # if this is the key prop, drop the index first
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
480 if old_spec[0] == prop:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
481 self.drop_class_table_key_index(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
482 del keyprop_changes['remove']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
483
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
484 # drop the column
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
485 sql = 'alter table _%s drop column _%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
486
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
487 self.sql(sql)
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
488 old_has = old_has.has_key
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
489
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
490 # if we didn't remove the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
491 # changed, we still need to remove the old index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
492 if keyprop_changes.has_key('remove'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
493 self.drop_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
494 keyprop_changes['remove'])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
495
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
496 # add new columns
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
497 for propname, prop in new_spec[1]:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
498 if old_has(propname):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
499 continue
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
500 prop = spec.properties[propname]
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
501 if isinstance(prop, Multilink):
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
502 self.create_multilink_table(spec, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
503 else:
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
504 # add the column
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
505 coltype = self.hyperdb_to_sql_datatype(prop.__class__)
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
506 sql = 'alter table _%s add column _%s %s'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
507 spec.classname, propname, coltype)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
508 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
509
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
510 # extra Interval column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
511 if isinstance(prop, Interval):
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
512 sql = 'alter table _%s add column __%s_int__ BIGINT'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
513 spec.classname, propname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
514 self.sql(sql)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
515
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
516 # if the new column is a key prop, we need an index!
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
517 if new_spec[0] == propname:
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
518 self.create_class_table_key_index(spec.classname, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
519 del keyprop_changes['add']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
520
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
521 # if we didn't add the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
522 # changed, we still need to add the new index
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
523 if keyprop_changes.has_key('add'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
524 self.create_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
525 keyprop_changes['add'])
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
526
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
527 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
528
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
529 def determine_all_columns(self, spec):
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
530 """Figure out the columns from the spec and also add internal columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
531
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
532 """
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
533 cols, mls = self.determine_columns(spec.properties.items())
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
534
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
535 # add on our special columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
536 cols.append(('id', 'INTEGER PRIMARY KEY'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
537 cols.append(('__retired__', 'INTEGER DEFAULT 0'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
538 return cols, mls
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
539
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
540 def create_class_table(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
541 """Create the class table for the given Class "spec". Creates the
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
542 indexes too."""
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
543 cols, mls = self.determine_all_columns(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
544
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
545 # create the base table
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
546 scols = ','.join(['%s %s'%x for x in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
547 sql = 'create table _%s (%s)'%(spec.classname, scols)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
548 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
549
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
550 self.create_class_table_indexes(spec)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
551
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
552 return cols, mls
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
553
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
554 def create_class_table_indexes(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
555 """ create the class table for the given spec
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
556 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
557 # create __retired__ index
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
558 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%(
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
559 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
560 self.sql(index_sql2)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
561
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
562 # create index for key property
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
563 if spec.key:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
564 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%(
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
565 spec.classname, spec.key,
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
566 spec.classname, spec.key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
567 self.sql(index_sql3)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
568
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
569 # and the unique index for key / retired(id)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
570 self.add_class_key_required_unique_constraint(spec.classname,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
571 spec.key)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
572
2228
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
573 # TODO: create indexes on (selected?) Link property columns, as
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
574 # they're more likely to be used for lookup
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
575
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
576 def add_class_key_required_unique_constraint(self, cn, key):
3969
905faf52a51f fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents: 3967
diff changeset
577 sql = '''create unique index _%s_key_retired_idx
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
578 on _%s(__retired__, _%s)'''%(cn, cn, key)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
579 self.sql(sql)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
580
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
581 def drop_class_table_indexes(self, cn, key):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
582 # drop the old table indexes first
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
583 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
584 if key:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
585 l.append('_%s_%s_idx'%(cn, key))
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
586
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
587 table_name = '_%s'%cn
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
588 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
589 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
590 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
591 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
592 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
593
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
594 def create_class_table_key_index(self, cn, key):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
595 """ create the class table for the given spec
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
596 """
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
597 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
598 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
599
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
600 def drop_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
601 table_name = '_%s'%cn
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
602 index_name = '_%s_%s_idx'%(cn, key)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
603 if self.sql_index_exists(table_name, index_name):
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
604 sql = 'drop index '+index_name
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
605 self.sql(sql)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
606
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
607 # and now the retired unique index too
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
608 index_name = '_%s_key_retired_idx'%cn
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
609 if self.sql_index_exists(table_name, index_name):
3967
b1e81ad3fa6a don't recompute the index name
Richard Jones <richard@users.sourceforge.net>
parents: 3963
diff changeset
610 sql = 'drop index '+index_name
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
611 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
612
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
613 def create_journal_table(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
614 """ create the journal table for a class given the spec and
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
615 already-determined cols
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
616 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
617 # journal table
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
618 cols = ','.join(['%s varchar'%x
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
619 for x in 'nodeid date tag action params'.split()])
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
620 sql = """create table %s__journal (
3224
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
621 nodeid integer, date %s, tag varchar(255),
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
622 action varchar(255), params text)""" % (spec.classname,
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
623 self.hyperdb_to_sql_datatype(hyperdb.Date))
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
624 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
625 self.create_journal_table_indexes(spec)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
626
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
627 def create_journal_table_indexes(self, spec):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
628 # index on nodeid
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
629 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%(
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
630 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
631 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
632
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
633 def drop_journal_table_indexes(self, classname):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
634 index_name = '%s_journ_idx'%classname
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
635 if not self.sql_index_exists('%s__journal'%classname, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
636 return
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
637 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
638 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
639
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
640 def create_multilink_table(self, spec, ml):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
641 """ Create a multilink table for the "ml" property of the class
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
642 given by the spec
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
643 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
644 # create the table
2828
3a2f138fcb28 use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2755
diff changeset
645 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%(
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
646 spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
647 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
648 self.create_multilink_table_indexes(spec, ml)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
649
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
650 def create_multilink_table_indexes(self, spec, ml):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
651 # create index on linkid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
652 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
653 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
654 self.sql(index_sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
655
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
656 # create index on nodeid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
657 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
658 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
659 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
660
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
661 def drop_multilink_table_indexes(self, classname, ml):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
662 l = [
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
663 '%s_%s_l_idx'%(classname, ml),
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
664 '%s_%s_n_idx'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
665 ]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
666 table_name = '%s_%s'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
667 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
668 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
669 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
670 index_sql = 'drop index %s'%index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
671 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
672
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
673 def create_class(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
674 """ Create a database table according to the given spec.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
675 """
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
676 cols, mls = self.create_class_table(spec)
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
677 self.create_journal_table(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
678
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
679 # now create the multilink tables
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
680 for ml in mls:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
681 self.create_multilink_table(spec, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
682
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
683 def drop_class(self, cn, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
684 """ Drop the given table from the database.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
685
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
686 Drop the journal and multilink tables too.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
687 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
688 properties = spec[1]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
689 # figure the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
690 mls = []
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3906
diff changeset
691 for propname, prop in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
692 if isinstance(prop, Multilink):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
693 mls.append(propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
694
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
695 # drop class table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
696 self.drop_class_table_indexes(cn, spec[0])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
697
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
698 self.drop_class_table(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
699
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
700 # drop journal table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
701 self.drop_journal_table_indexes(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
702 sql = 'drop table %s__journal'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
703 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
704
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
705 for ml in mls:
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
706 # drop multilink table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
707 self.drop_multilink_table_indexes(cn, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
708 sql = 'drop table %s_%s'%(spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
709 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
710
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
711 def drop_class_table(self, cn):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
712 sql = 'drop table _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
713 self.sql(sql)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
714
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
715 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
716 # Classes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
717 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
718 def __getattr__(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
719 """ A convenient way of calling self.getclass(classname).
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
720 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
721 if self.classes.has_key(classname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
722 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
723 raise AttributeError, classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
724
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
725 def addclass(self, cl):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
726 """ Add a Class to the hyperdatabase.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
727 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
728 cn = cl.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
729 if self.classes.has_key(cn):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
730 raise ValueError, cn
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
731 self.classes[cn] = cl
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
732
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
733 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
734 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
735 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
736 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
737 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
738 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
739 description="User is allowed to access "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
740
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
741 def getclasses(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
742 """ Return a list of the names of all existing classes.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
743 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
744 l = self.classes.keys()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
745 l.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
746 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
747
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
748 def getclass(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
749 """Get the Class object representing a particular class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
750
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
751 If 'classname' is not a valid class name, a KeyError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
752 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
753 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
754 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
755 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
756 raise KeyError, 'There is no class called "%s"'%classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
757
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
758 def clear(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
759 """Delete all database contents.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
760
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
761 Note: I don't commit here, which is different behaviour to the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
762 "nuke from orbit" behaviour in the dbs.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
763 """
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
764 logging.getLogger('hyperdb').info('clear')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
765 for cn in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
766 sql = 'delete from _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
767 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
768
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
769 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
770 # Nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
771 #
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
772
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
773 hyperdb_to_sql_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
774 hyperdb.String : str,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
775 # fractional seconds by default
3612
afda59d5d546 prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
776 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
777 hyperdb.Link : int,
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
778 hyperdb.Interval : str,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
779 hyperdb.Password : str,
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
780 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
781 hyperdb.Number : lambda x: x,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
782 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
783 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
784
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
785 def to_sql_value(self, propklass):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
786
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
787 fn = self.hyperdb_to_sql_value.get(propklass)
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
788 if fn:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
789 return fn
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
790
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
791 for k, v in self.hyperdb_to_sql_value.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
792 if issubclass(propklass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
793 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
794
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
795 raise ValueError, '%r is not a hyperdb property class' % propklass
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
796
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
797 def addnode(self, classname, nodeid, node):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
798 """ Add the specified node to its class's db.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
799 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
800 self.log_debug('addnode %s%s %r'%(classname,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
801 nodeid, node))
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
802
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
803 # determine the column definitions and multilink tables
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
804 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
805 cols, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
806
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
807 # we'll be supplied these props if we're doing an import
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
808 values = node.copy()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
809 if not values.has_key('creator'):
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
810 # add in the "calculated" properties (dupe so we don't affect
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
811 # calling code's node assumptions)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
812 values['creation'] = values['activity'] = date.Date()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
813 values['actor'] = values['creator'] = self.getuid()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
814
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
815 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
816 props = cl.getprops(protected=1)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
817 del props['id']
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
818
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
819 # default the non-multilink columns
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
820 for col, prop in props.items():
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
821 if not values.has_key(col):
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
822 if isinstance(prop, Multilink):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
823 values[col] = []
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
824 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
825 values[col] = None
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
826
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
827 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
828 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
829 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
830 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
831 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
832
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
833 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
834 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
835 for col,dt in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
836 # this is somewhat dodgy....
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
837 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
838 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
839 value = values[col[2:-6]]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
840 # this is an Interval special "int" column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
841 if value is not None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
842 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
843 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
844 vals.append(value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
845 continue
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
846
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
847 prop = props[col[1:]]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
848 value = values[col[1:]]
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
849 if value is not None:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
850 value = self.to_sql_value(prop.__class__)(value)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
851 vals.append(value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
852 vals.append(nodeid)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
853 vals = tuple(vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
854
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
855 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
856 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
857 cols = ','.join([col for col,dt in cols]) + ',id'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
858
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
859 # perform the inserts
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
860 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
861 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
862
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
863 # insert the multilink rows
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
864 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
865 t = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
866 for entry in node[col]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
867 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
868 self.arg, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
869 self.sql(sql, (entry, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
870
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
871 def setnode(self, classname, nodeid, values, multilink_changes={}):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
872 """ Change the specified node.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
873 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
874 self.log_debug('setnode %s%s %r'
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
875 % (classname, nodeid, values))
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
876
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
877 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
878 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
879 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
880 del self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
881 self.cache_lru.remove(key)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
882
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
883 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
884 props = cl.getprops()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
885
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
886 cols = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
887 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
888 # add the multilinks separately
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
889 for col in values.keys():
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
890 prop = props[col]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
891 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
892 mls.append(col)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
893 elif isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
894 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
895 cols.append(col)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
896 # extra leading '_' added by code below
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
897 cols.append('_' +col + '_int__')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
898 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
899 cols.append(col)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
900 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
901
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
902 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
903 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
904 for col in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
905 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
906 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
907 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
908 col = col[1:-6]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
909 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
910 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
911 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
912 vals.append(None)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
913 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
914 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
915 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
916 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
917 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
918 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
919 e = None
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
920 else:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
921 e = self.to_sql_value(prop.__class__)(value)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
922 vals.append(e)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
923
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
924 vals.append(int(nodeid))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
925 vals = tuple(vals)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
926
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
927 # if there's any updates to regular columns, do them
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
928 if cols:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
929 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
930 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols])
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
931 cols = ','.join(cols)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
932
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
933 # perform the update
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
934 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
935 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
936
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
937 # we're probably coming from an import, not a change
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
938 if not multilink_changes:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
939 for name in mls:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
940 prop = props[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
941 value = values[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
942
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
943 t = '%s_%s'%(classname, name)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
944
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
945 # clear out previous values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
946 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
947 self.sql('delete from %s where nodeid=%s'%(t, self.arg),
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
948 (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
949
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
950 # insert the values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
951 for entry in values[name]:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
952 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
953 self.arg, self.arg)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
954 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
955 self.sql(sql, (entry, nodeid))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
956
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
957 # we have multilink changes to apply
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
958 for col, (add, remove) in multilink_changes.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
959 tn = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
960 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
961 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
962 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
963 for addid in add:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
964 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
965 self.sql(sql, (int(nodeid), int(addid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
966 if remove:
4039
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
967 s = ','.join([self.arg]*len(remove))
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
968 sql = 'delete from %s where nodeid=%s and linkid in (%s)'%(tn,
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
969 self.arg, s)
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
970 # XXX numeric ids
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
971 self.sql(sql, [int(nodeid)] + remove)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
972
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
973 sql_to_hyperdb_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
974 hyperdb.String : str,
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
975 hyperdb.Date : lambda x:date.Date(str(x).replace(' ', '.')),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
976 # hyperdb.Link : int, # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
977 hyperdb.Link : str,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
978 hyperdb.Interval : date.Interval,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
979 hyperdb.Password : lambda x: password.Password(encrypted=x),
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
980 hyperdb.Boolean : _bool_cvt,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
981 hyperdb.Number : _num_cvt,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
982 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
983 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
984
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
985 def to_hyperdb_value(self, propklass):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
986
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
987 fn = self.sql_to_hyperdb_value.get(propklass)
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
988 if fn:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
989 return fn
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
990
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
991 for k, v in self.sql_to_hyperdb_value.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
992 if issubclass(propklass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
993 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
994
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
995 raise ValueError, '%r is not a hyperdb property class' % propklass
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
996
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
997 def getnode(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
998 """ Get a node from the database.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
999 """
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1000 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1001 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1002 if self.cache.has_key(key):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1003 # push us back to the top of the LRU
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1004 self.cache_lru.remove(key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
1005 self.cache_lru.insert(0, key)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1006 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1007 self.stats['cache_hits'] += 1
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1008 # return the cached information
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1009 return self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1010
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1011 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1012 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1013 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1014
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1015 # figure the columns we're fetching
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1016 cl = self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1017 cols, mls = self.determine_columns(cl.properties.items())
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1018 scols = ','.join([col for col,dt in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1019
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1020 # perform the basic property fetch
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1021 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1022 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1023
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1024 values = self.sql_fetchone()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1025 if values is None:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1026 raise IndexError, 'no such %s node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1027
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1028 # make up the node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1029 node = {}
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1030 props = cl.getprops(protected=1)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1031 for col in range(len(cols)):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1032 name = cols[col][0][1:]
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1033 if name.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1034 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1035 # ignore the special Interval-as-seconds column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1036 continue
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1037 value = values[col]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1038 if value is not None:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1039 value = self.to_hyperdb_value(props[name].__class__)(value)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1040 node[name] = value
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1041
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1042
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1043 # now the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1044 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1045 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1046 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1047 self.arg)
4038
f8bc05158f46 Execute cursor via Database.sql().
Stefan Seefeld <stefan@seefeld.name>
parents: 4031
diff changeset
1048 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1049 # extract the first column from the result
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1050 # XXX numeric ids
2887
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
1051 items = [int(x[0]) for x in self.cursor.fetchall()]
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
1052 items.sort ()
23ed4195e442 ensure multilink ordering in RDBMS backends (thanks Marcus Priesch)
Richard Jones <richard@users.sourceforge.net>
parents: 2828
diff changeset
1053 node[col] = [str(x) for x in items]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1054
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1055 # save off in the cache
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1056 key = (classname, nodeid)
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1057 self.cache[key] = node
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1058 # update the LRU
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1059 self.cache_lru.insert(0, key)
1185
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
1060 if len(self.cache_lru) > ROW_CACHE_SIZE:
3b0735ef8207 fix to LRU cache
Richard Jones <richard@users.sourceforge.net>
parents: 1183
diff changeset
1061 del self.cache[self.cache_lru.pop()]
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1062
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1063 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1064 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1065
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1066 return node
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1067
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1068 def destroynode(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1069 """Remove a node from the database. Called exclusively by the
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1070 destroy() method on Class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1071 """
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1072 logging.getLogger('hyperdb').info('destroynode %s%s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1073
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1074 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1075 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1076 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1077
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1078 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1079 if self.cache.has_key((classname, nodeid)):
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1080 del self.cache[(classname, nodeid)]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1081
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1082 # see if there's any obvious commit actions that we should get rid of
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1083 for entry in self.transactions[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1084 if entry[1][:2] == (classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1085 self.transactions.remove(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1086
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1087 # now do the SQL
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1088 sql = 'delete from _%s where id=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1089 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1090
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1091 # remove from multilnks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1092 cl = self.getclass(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1093 x, mls = self.determine_columns(cl.properties.items())
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1094 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1095 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1096 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
1097 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1098
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1099 # remove journal entries
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1100 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1101 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1102
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1103 # cleanup any blob filestorage when we commit
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1104 self.transactions.append((FileStorage.destroy, (self, classname, nodeid)))
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1105
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1106 def hasnode(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1107 """ Determine if the database has a given node.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1108 """
4031
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1109 # If this node is in the cache, then we do not need to go to
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1110 # the database. (We don't consider this an LRU hit, though.)
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1111 if self.cache.has_key((classname, nodeid)):
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1112 # Return 1, not True, to match the type of the result of
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1113 # the SQL operation below.
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1114 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1115 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1116 self.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1117 return int(self.cursor.fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1118
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1119 def countnodes(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1120 """ Count the number of nodes that exist for a particular Class.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1121 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1122 sql = 'select count(*) from _%s'%classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1123 self.sql(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1124 return self.cursor.fetchone()[0]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1125
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1126 def addjournal(self, classname, nodeid, action, params, creator=None,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1127 creation=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1128 """ Journal the Action
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1129 'action' may be:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1130
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1131 'create' or 'set' -- 'params' is a dictionary of property values
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1132 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1133 'retire' -- 'params' is None
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1134 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1135 # handle supply of the special journalling parameters (usually
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1136 # supplied on importing an existing database)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1137 if creator:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1138 journaltag = creator
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1139 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1140 journaltag = self.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1141 if creation:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1142 journaldate = creation
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1143 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1144 journaldate = date.Date()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1145
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1146 # create the journal entry
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1147 cols = 'nodeid,date,tag,action,params'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1148
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1149 self.log_debug('addjournal %s%s %r %s %s %r'%(classname,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1150 nodeid, journaldate, journaltag, action, params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1151
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1152 # make the journalled data marshallable
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1153 if isinstance(params, type({})):
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1154 self._journal_marshal(params, classname)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1155
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1156 params = repr(params)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1157
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1158 dc = self.to_sql_value(hyperdb.Date)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1159 journaldate = dc(journaldate)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1160
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1161 self.save_journal(classname, cols, nodeid, journaldate,
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1162 journaltag, action, params)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1163
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1164 def setjournal(self, classname, nodeid, journal):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1165 """Set the journal to the "journal" list."""
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1166 # clear out any existing entries
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1167 self.sql('delete from %s__journal where nodeid=%s'%(classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1168 self.arg), (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1169
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1170 # create the journal entry
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1171 cols = 'nodeid,date,tag,action,params'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1172
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1173 dc = self.to_sql_value(hyperdb.Date)
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1174 for nodeid, journaldate, journaltag, action, params in journal:
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1175 self.log_debug('addjournal %s%s %r %s %s %r'%(
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1176 classname, nodeid, journaldate, journaltag, action,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1177 params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1178
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1179 # make the journalled data marshallable
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1180 if isinstance(params, type({})):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1181 self._journal_marshal(params, classname)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1182 params = repr(params)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1183
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1184 self.save_journal(classname, cols, nodeid, dc(journaldate),
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1185 journaltag, action, params)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1186
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1187 def _journal_marshal(self, params, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1188 """Convert the journal params values into safely repr'able and
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1189 eval'able values."""
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1190 properties = self.getclass(classname).getprops()
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1191 for param, value in params.items():
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1192 if not value:
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1193 continue
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1194 property = properties[param]
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1195 cvt = self.to_sql_value(property.__class__)
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1196 if isinstance(property, Password):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1197 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1198 elif isinstance(property, Date):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1199 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1200 elif isinstance(property, Interval):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1201 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1202 elif isinstance(property, Boolean):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1203 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1204
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1205 def getjournal(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1206 """ get the journal for id
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1207 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1208 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1209 if not self.hasnode(classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1210 raise IndexError, '%s has no node %s'%(classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1211
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1212 cols = ','.join('nodeid date tag action params'.split())
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1213 journal = self.load_journal(classname, cols, nodeid)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1214
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1215 # now unmarshal the data
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1216 dc = self.to_hyperdb_value(hyperdb.Date)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1217 res = []
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1218 properties = self.getclass(classname).getprops()
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1219 for nodeid, date_stamp, user, action, params in journal:
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1220 params = eval(params)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1221 if isinstance(params, type({})):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1222 for param, value in params.items():
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1223 if not value:
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1224 continue
2731
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1225 property = properties.get(param, None)
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1226 if property is None:
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1227 # deleted property
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1228 continue
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1229 cvt = self.to_hyperdb_value(property.__class__)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1230 if isinstance(property, Password):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1231 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1232 elif isinstance(property, Date):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1233 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1234 elif isinstance(property, Interval):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1235 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1236 elif isinstance(property, Boolean):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1237 params[param] = cvt(value)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1238 # XXX numeric ids
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1239 res.append((str(nodeid), dc(date_stamp), user, action, params))
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1240 return res
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1241
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1242 def save_journal(self, classname, cols, nodeid, journaldate,
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1243 journaltag, action, params):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1244 """ Save the journal entry to the database
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1245 """
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1246 entry = (nodeid, journaldate, journaltag, action, params)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1247
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1248 # do the insert
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1249 a = self.arg
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1250 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1251 classname, cols, a, a, a, a, a)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1252 self.sql(sql, entry)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1253
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1254 def load_journal(self, classname, cols, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1255 """ Load the journal from the database
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1256 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1257 # now get the journal entries
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1258 sql = 'select %s from %s__journal where nodeid=%s order by date'%(
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1259 cols, classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1260 self.sql(sql, (nodeid,))
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1261 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1262
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1263 def pack(self, pack_before):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1264 """ Delete all journal entries except "create" before 'pack_before'.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1265 """
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1266 date_stamp = self.to_sql_value(Date)(pack_before)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1267
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1268 # do the delete
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1269 for classname in self.classes.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1270 sql = "delete from %s__journal where date<%s and "\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1271 "action<>'create'"%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1272 self.sql(sql, (date_stamp,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1273
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1274 def sql_commit(self, fail_ok=False):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1275 """ Actually commit to the database.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1276 """
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1277 logging.getLogger('hyperdb').info('commit')
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1278
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1279 self.conn.commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1280
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1281 # open a new cursor for subsequent work
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1282 self.cursor = self.conn.cursor()
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1283
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1284 def commit(self, fail_ok=False):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1285 """ Commit the current transactions.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1286
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1287 Save all data changed since the database was opened or since the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1288 last commit() or rollback().
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1289
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1290 fail_ok indicates that the commit is allowed to fail. This is used
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1291 in the web interface when committing cleaning of the session
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1292 database. We don't care if there's a concurrency issue there.
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1293
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1294 The only backend this seems to affect is postgres.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1295 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1296 # commit the database
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1297 self.sql_commit(fail_ok)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1298
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1299 # now, do all the other transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1300 for method, args in self.transactions:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1301 method(*args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1302
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1303 # save the indexer
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1304 self.indexer.save_index()
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1305
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1306 # clear out the transactions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1307 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1308
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1309 def sql_rollback(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1310 self.conn.rollback()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1311
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1312 def rollback(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1313 """ Reverse all actions from the current transaction.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1314
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1315 Undo all the changes made since the database was opened or the last
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1316 commit() or rollback() was performed.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1317 """
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1318 logging.getLogger('hyperdb').info('rollback')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1319
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1320 self.sql_rollback()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1321
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1322 # roll back "other" transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1323 for method, args in self.transactions:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1324 # delete temporary files
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1325 if method == self.doStoreFile:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1326 self.rollbackStoreFile(*args)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1327 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1328
1492
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1329 # clear the cache
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1330 self.clearCache()
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1331
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1332 def sql_close(self):
3155
57b60bda9473 Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
Richard Jones <richard@users.sourceforge.net>
parents: 3147
diff changeset
1333 logging.getLogger('hyperdb').info('close')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1334 self.conn.close()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1335
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1336 def close(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1337 """ Close off the connection.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1338 """
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
1339 self.indexer.close()
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1340 self.sql_close()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1341
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1342 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1343 # The base Class class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1344 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1345 class Class(hyperdb.Class):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1346 """ The handle to a particular class of nodes in a hyperdatabase.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1347
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1348 All methods except __repr__ and getnode must be implemented by a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1349 concrete backend Class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1350 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1351
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1352 def schema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1353 """ A dumpable version of the schema that we can store in the
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1354 database
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1355 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1356 return (self.key, [(x, repr(y)) for x,y in self.properties.items()])
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1357
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1358 def enableJournalling(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1359 """Turn journalling on for this class
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1360 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1361 self.do_journal = 1
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1362
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1363 def disableJournalling(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1364 """Turn journalling off for this class
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1365 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1366 self.do_journal = 0
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1367
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1368 # Editing nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1369 def create(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1370 """ Create a new node of this class and return its id.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1371
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1372 The keyword arguments in 'propvalues' map property names to values.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1373
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1374 The values of arguments must be acceptable for the types of their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1375 corresponding properties or a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1376
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1377 If this class has a key property, it must be present and its value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1378 must not collide with other key strings or a ValueError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1379
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1380 Any other properties on this class that are missing from the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1381 'propvalues' dictionary are set to None.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1382
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1383 If an id in a link or multilink property does not refer to a valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1384 node, an IndexError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1385 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1386 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1387 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1388 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1389 return newid
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1390
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1391 def create_inner(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1392 """ Called by create, in-between the audit and react calls.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1393 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1394 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1395 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1396
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1397 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
1398 raise DatabaseError, _('Database open read-only')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1399
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1400 if propvalues.has_key('creator') or propvalues.has_key('actor') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1401 propvalues.has_key('creation') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1402 raise KeyError, '"creator", "actor", "creation" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1403 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1404
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1405 # new node's id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1406 newid = self.db.newid(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1407
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1408 # validate propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1409 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1410 for key, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1411 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1412 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1413 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1414 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1415 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1416 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1417 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1418
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1419 # try to handle this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1420 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1421 prop = self.properties[key]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1422 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1423 raise KeyError, '"%s" has no property "%s"'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1424 key)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1425
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1426 if value is not None and isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1427 if type(value) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1428 raise ValueError, 'link value must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1429 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1430 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1431 if not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1432 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1433 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1434 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1435 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1436 key, value, link_class)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1437 elif not self.db.getclass(link_class).hasnode(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1438 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1439
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1440 # save off the value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1441 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1442
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1443 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1444 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1445 self.db.addjournal(link_class, value, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1446 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1447
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1448 elif isinstance(prop, Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1449 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1450 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1451 if not hasattr(value, '__iter__'):
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1452 raise TypeError, 'new property "%s" not an iterable of ids'%key
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1453
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1454 # clean up and validate the list of links
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1455 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1456 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1457 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1458 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1459 raise ValueError, '"%s" multilink value (%r) '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1460 'must contain Strings'%(key, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1461 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1462 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1463 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1464 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1465 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1466 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1467 key, entry, self.properties[key].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1468 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1469 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1470 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1471
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1472 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1473 for nodeid in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1474 if not self.db.getclass(link_class).hasnode(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1475 raise IndexError, '%s has no node %s'%(link_class,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1476 nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1477 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1478 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1479 self.db.addjournal(link_class, nodeid, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1480 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1481
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1482 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1483 if type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1484 raise TypeError, 'new property "%s" not a string'%key
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1485 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1486 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1487 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1488
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1489 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1490 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1491 raise TypeError, 'new property "%s" not a Password'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1492
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1493 elif isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1494 if value is not None and not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1495 raise TypeError, 'new property "%s" not a Date'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1496
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1497 elif isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1498 if value is not None and not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1499 raise TypeError, 'new property "%s" not an Interval'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1500
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1501 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1502 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1503 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1504 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1505 raise TypeError, 'new property "%s" not numeric'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1506
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1507 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1508 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1509 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1510 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1511 raise TypeError, 'new property "%s" not boolean'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1512
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1513 # make sure there's data where there needs to be
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1514 for key, prop in self.properties.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1515 if propvalues.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1516 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1517 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1518 raise ValueError, 'key property "%s" is required'%key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1519 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1520 propvalues[key] = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1521 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1522 propvalues[key] = None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1523
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1524 # done
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1525 self.db.addnode(self.classname, newid, propvalues)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1526 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1527 self.db.addjournal(self.classname, newid, ''"create", {})
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1528
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1529 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1530 return str(newid)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1531
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1532 def get(self, nodeid, propname, default=_marker, cache=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1533 """Get the value of a property on an existing node of this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1534
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1535 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1536 IndexError is raised. 'propname' must be the name of a property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1537 of this class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1538
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1539 'cache' exists for backwards compatibility, and is not used.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1540 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1541 if propname == 'id':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1542 return nodeid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1543
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1544 # get the node's dict
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1545 d = self.db.getnode(self.classname, nodeid)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1546
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1547 if propname == 'creation':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1548 if d.has_key('creation'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1549 return d['creation']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1550 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1551 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1552 if propname == 'activity':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1553 if d.has_key('activity'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1554 return d['activity']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1555 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1556 return date.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1557 if propname == 'creator':
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1558 if d.has_key('creator'):
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1559 return d['creator']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1560 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1561 return self.db.getuid()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1562 if propname == 'actor':
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1563 if d.has_key('actor'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1564 return d['actor']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1565 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1566 return self.db.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1567
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1568 # get the property (raises KeyErorr if invalid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1569 prop = self.properties[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1570
2690
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1571 # XXX may it be that propname is valid property name
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1572 # (above error is not raised) and not d.has_key(propname)???
9054d546fc9f Class.get(): return default if rdbms value is None (i.e. unset).
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1573 if (not d.has_key(propname)) or (d[propname] is None):
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
1574 if default is _marker:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1575 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1576 return []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1577 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1578 return None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1579 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1580 return default
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1581
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1582 # don't pass our list to other code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1583 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1584 return d[propname][:]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1585
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1586 return d[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1587
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1588 def set(self, nodeid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1589 """Modify a property on an existing node of this class.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1590
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1591 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1592 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1593
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1594 Each key in 'propvalues' must be the name of a property of this
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1595 class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1596
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1597 All values in 'propvalues' must be acceptable types for their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1598 corresponding properties or a TypeError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1599
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1600 If the value of the key property is set, it must not collide with
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1601 other key strings or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1602
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1603 If the value of a Link or Multilink property contains an invalid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1604 node id, a ValueError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1605 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1606 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1607 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1608 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1609 self.fireReactors('set', nodeid, oldvalues)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1610 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1611
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1612 def set_inner(self, nodeid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1613 """ Called by set, in-between the audit and react calls.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1614 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1615 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1616 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1617
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1618 if propvalues.has_key('creation') or propvalues.has_key('creator') or \
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1619 propvalues.has_key('actor') or propvalues.has_key('activity'):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1620 raise KeyError, '"creation", "creator", "actor" and '\
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1621 '"activity" are reserved'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1622
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1623 if propvalues.has_key('id'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1624 raise KeyError, '"id" is reserved'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1625
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1626 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
1627 raise DatabaseError, _('Database open read-only')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1628
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1629 node = self.db.getnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1630 if self.is_retired(nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1631 raise IndexError, 'Requested item is retired'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1632 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1633
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1634 # make a copy of the values dictionary - we'll modify the contents
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1635 propvalues = propvalues.copy()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1636
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1637 # if the journal value is to be different, store it in here
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1638 journalvalues = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1639
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1640 # remember the add/remove stuff for multilinks, making it easier
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1641 # for the Database layer to do its stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1642 multilink_changes = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1643
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1644 for propname, value in propvalues.items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1645 # check to make sure we're not duplicating an existing key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1646 if propname == self.key and node[propname] != value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1647 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1648 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1649 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1650 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1651 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1652 raise ValueError, 'node with key "%s" exists'%value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1653
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1654 # this will raise the KeyError if the property isn't valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1655 # ... we don't use getprops() here because we only care about
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1656 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1657 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1658 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1659 except KeyError:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1660 raise KeyError, '"%s" has no property named "%s"'%(
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1661 self.classname, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1662
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1663 # if the value's the same as the existing value, no sense in
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1664 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1665 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1666 if value == current:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1667 del propvalues[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1668 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1669 journalvalues[propname] = current
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1670
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1671 # do stuff based on the prop type
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1672 if isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1673 link_class = prop.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1674 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1675 if value is not None and not isinstance(value, type('')):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1676 raise ValueError, 'property "%s" link value be a string'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1677 propname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1678 if isinstance(value, type('')) and not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1679 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1680 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1681 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1682 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1683 propname, value, prop.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1684
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1685 if (value is not None and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1686 not self.db.getclass(link_class).hasnode(value)):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1687 raise IndexError, '%s has no node %s'%(link_class, value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1688
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1689 if self.do_journal and prop.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1690 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1691 if node[propname] is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1692 self.db.addjournal(link_class, node[propname],
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1693 ''"unlink", (self.classname, nodeid, propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1694
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1695 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1696 if value is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1697 self.db.addjournal(link_class, value, ''"link",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1698 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1699
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1700 elif isinstance(prop, Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1701 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1702 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1703 if not hasattr(value, '__iter__'):
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1704 raise TypeError, 'new property "%s" not an iterable of'\
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1705 ' ids'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1706 link_class = self.properties[propname].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1707 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1708 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1709 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1710 if type(entry) != type(''):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1711 raise ValueError, 'new property "%s" link value ' \
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1712 'must be a string'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1713 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1714 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1715 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1716 except (TypeError, KeyError):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1717 raise IndexError, 'new property "%s": %s not a %s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1718 propname, entry,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1719 self.properties[propname].classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1720 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1721 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1722 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1723
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1724 # figure the journal entry for this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1725 add = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1726 remove = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1727
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1728 # handle removals
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1729 if node.has_key(propname):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1730 l = node[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1731 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1732 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1733 for id in l[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1734 if id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1735 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1736 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1737 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1738 self.db.addjournal(link_class, id, 'unlink',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1739 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1740 l.remove(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1741 remove.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1742
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1743 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1744 for id in value:
4031
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1745 # If this node is in the cache, then we do not need to go to
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1746 # the database. (We don't consider this an LRU hit, though.)
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1747 if self.cache.has_key((classname, nodeid)):
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1748 # Return 1, not True, to match the type of the result of
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1749 # the SQL operation below.
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1750 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1751 if not self.db.getclass(link_class).hasnode(id):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1752 raise IndexError, '%s has no node %s'%(link_class, id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1753 if id in l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1754 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1755 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1756 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1757 self.db.addjournal(link_class, id, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1758 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1759 l.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1760 add.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1761
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1762 # figure the journal entry
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1763 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1764 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1765 l.append(('+', add))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1766 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1767 l.append(('-', remove))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1768 multilink_changes[propname] = (add, remove)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1769 if l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1770 journalvalues[propname] = tuple(l)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1771
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1772 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1773 if value is not None and type(value) != type('') and type(value) != type(u''):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1774 raise TypeError, 'new property "%s" not a string'%propname
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1775 if prop.indexme:
3399
3a0d4e4a0f34 merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
1776 if value is None: value = ''
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1777 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1778 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1779
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1780 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1781 if not isinstance(value, password.Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1782 raise TypeError, 'new property "%s" not a Password'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1783 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1784
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1785 elif value is not None and isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1786 if not isinstance(value, date.Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1787 raise TypeError, 'new property "%s" not a Date'% propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1788 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1789
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1790 elif value is not None and isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1791 if not isinstance(value, date.Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1792 raise TypeError, 'new property "%s" not an '\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1793 'Interval'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1794 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1795
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1796 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1797 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1798 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1799 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1800 raise TypeError, 'new property "%s" not numeric'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1801
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1802 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1803 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1804 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1805 except ValueError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1806 raise TypeError, 'new property "%s" not boolean'%propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1807
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1808 # nothing to do?
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1809 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1810 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1811
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1812 # update the activity time
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1813 propvalues['activity'] = date.Date()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1814 propvalues['actor'] = self.db.getuid()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1815
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1816 # do the set
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1817 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1818
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1819 # remove the activity props now they're handled
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1820 del propvalues['activity']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1821 del propvalues['actor']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1822
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
1823 # journal the set
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1824 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1825 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1826
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1827 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1828
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1829 def retire(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1830 """Retire a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1831
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1832 The properties on the node remain available from the get() method,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1833 and the node's id is never reused.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1834
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1835 Retired nodes are not returned by the find(), list(), or lookup()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1836 methods, and other nodes may reuse the values of their key properties.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1837 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1838 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
1839 raise DatabaseError, _('Database open read-only')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1840
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1841 self.fireAuditors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1842
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1843 # use the arg for __retired__ to cope with any odd database type
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1844 # conversion (hello, sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1845 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1846 self.db.arg, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
1847 self.db.sql(sql, (nodeid, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1848 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1849 self.db.addjournal(self.classname, nodeid, ''"retired", None)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1850
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1851 self.fireReactors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
1852
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1853 def restore(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1854 """Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1855
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1856 Make node available for all operations like it was before retirement.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1857 """
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1858 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
1859 raise DatabaseError, _('Database open read-only')
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1860
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1861 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1862 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1863 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1864 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1865 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1866 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1867 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1868 else:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1869 raise KeyError, "Key property (%s) of retired node clashes with \
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1870 existing one (%s)" % (key, node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1871
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1872 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1873 # use the arg for __retired__ to cope with any odd database type
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1874 # conversion (hello, sqlite)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1875 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1876 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1877 self.db.sql(sql, (0, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1878 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1879 self.db.addjournal(self.classname, nodeid, ''"restored", None)
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1880
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
1881 self.fireReactors('restore', nodeid, None)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1882
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1883 def is_retired(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1884 """Return true if the node is rerired
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1885 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1886 sql = 'select __retired__ from _%s where id=%s'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1887 self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1888 self.db.sql(sql, (nodeid,))
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
1889 return int(self.db.sql_fetchone()[0]) > 0
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1890
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1891 def destroy(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1892 """Destroy a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1893
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1894 WARNING: this method should never be used except in extremely rare
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1895 situations where there could never be links to the node being
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1896 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1897
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1898 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1899
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1900 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
1901
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1902 WARNING: really, use retire() instead
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1903
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1904 Well, I think that's enough warnings. This method exists mostly to
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1905 support the session storage of the cgi interface.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1906
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1907 The node is completely removed from the hyperdb, including all journal
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1908 entries. It will no longer be available, and will generally break code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1909 if there are any references to the node.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1910 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1911 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
1912 raise DatabaseError, _('Database open read-only')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1913 self.db.destroynode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1914
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1915 def history(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1916 """Retrieve the journal of edits on a particular node.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1917
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1918 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1919 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1920
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1921 The returned list contains tuples of the form
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1922
1409
8dc60d87ab42 Fixed a backlog of bug reports, and worked on python 2.3 compatibility:
Richard Jones <richard@users.sourceforge.net>
parents: 1383
diff changeset
1923 (nodeid, date, tag, action, params)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1924
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1925 'date' is a Timestamp object specifying the time of the change and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1926 'tag' is the journaltag specified when the database was opened.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1927 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1928 if not self.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1929 raise ValueError, 'Journalling is disabled for this class'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1930 return self.db.getjournal(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1931
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1932 # Locating nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1933 def hasnode(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1934 """Determine if the given nodeid actually exists
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1935 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1936 return self.db.hasnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1937
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1938 def setkey(self, propname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1939 """Select a String property of this class to be the key property.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1940
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1941 'propname' must be the name of a String property of this class or
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1942 None, or a TypeError is raised. The values of the key property on
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1943 all existing nodes must be unique or a ValueError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1944 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1945 prop = self.getprops()[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1946 if not isinstance(prop, String):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1947 raise TypeError, 'key properties must be String'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1948 self.key = propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1949
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1950 def getkey(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1951 """Return the name of the key property for this class or None."""
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1952 return self.key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1953
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1954 def lookup(self, keyvalue):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1955 """Locate a particular node by its key property and return its id.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1956
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1957 If this class has no key property, a TypeError is raised. If the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1958 'keyvalue' matches one of the values for the key property among
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1959 the nodes in this class, the matching node's id is returned;
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1960 otherwise a KeyError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1961 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1962 if not self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1963 raise TypeError, 'No key property set for class %s'%self.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1964
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1965 # use the arg to handle any odd database type conversion (hello,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1966 # sqlite)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
1967 sql = "select id from _%s where _%s=%s and __retired__=%s"%(
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1968 self.classname, self.key, self.db.arg, self.db.arg)
3977
732a37da3a10 Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479])
Richard Jones <richard@users.sourceforge.net>
parents: 3969
diff changeset
1969 self.db.sql(sql, (str(keyvalue), 0))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1970
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1971 # see if there was a result that's not retired
1203
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1972 row = self.db.sql_fetchone()
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1973 if not row:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1974 raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key,
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1975 keyvalue, self.classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1976
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1977 # return the id
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1978 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1979 return str(row[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1980
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1981 def find(self, **propspec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1982 """Get the ids of nodes in this class which link to the given nodes.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1983
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1984 'propspec' consists of keyword args propname=nodeid or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1985 propname={nodeid:1, }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1986 'propname' must be the name of a property in this class, or a
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1987 KeyError is raised. That property must be a Link or
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
1988 Multilink property, or a TypeError is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1989
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1990 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1991 the nodeids will be returned. Examples::
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1992
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
1993 db.issue.find(messages='1')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1994 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1995 """
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1996 # shortcut
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1997 if not propspec:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1998 return []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
1999
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2000 # validate the args
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2001 props = self.getprops()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2002 propspec = propspec.items()
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2003 for propname, nodeids in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2004 # check the prop is OK
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2005 prop = props[propname]
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2006 if not isinstance(prop, Link) and not isinstance(prop, Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2007 raise TypeError, "'%s' not a Link/Multilink property"%propname
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2008
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2009 # first, links
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2010 a = self.db.arg
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2011 allvalues = ()
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2012 sql = []
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2013 where = []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2014 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2015 if not isinstance(props[prop], hyperdb.Link):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2016 continue
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2017 if type(values) is type({}) and len(values) == 1:
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2018 values = values.keys()[0]
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2019 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2020 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2021 where.append('_%s = %s'%(prop, a))
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
2022 elif values is None:
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
2023 where.append('_%s is NULL'%prop)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2024 else:
2494
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2025 values = values.keys()
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2026 s = ''
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2027 if None in values:
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2028 values.remove(None)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2029 s = '_%s is NULL or '%prop
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2030 allvalues += tuple(values)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2031 s += '_%s in (%s)'%(prop, ','.join([a]*len(values)))
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2032 where.append('(' + s +')')
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2033 if where:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2034 allvalues = (0, ) + allvalues
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2035 sql.append("""select id from _%s where __retired__=%s
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2036 and %s"""%(self.classname, a, ' and '.join(where)))
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2037
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2038 # now multilinks
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2039 for prop, values in propspec:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2040 if not isinstance(props[prop], hyperdb.Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2041 continue
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2042 if not values:
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2043 continue
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2044 allvalues += (0, )
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2045 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2046 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2047 s = a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2048 else:
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2049 allvalues += tuple(values.keys())
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2050 s = ','.join([a]*len(values))
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2051 tn = '%s_%s'%(self.classname, prop)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2052 sql.append("""select id from _%s, %s where __retired__=%s
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2053 and id = %s.nodeid and %s.linkid in (%s)"""%(self.classname,
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2054 tn, a, tn, tn, s))
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2055
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2056 if not sql:
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2057 return []
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2058 sql = ' union '.join(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
2059 self.db.sql(sql, allvalues)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2060 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2061 l = [str(x[0]) for x in self.db.sql_fetchall()]
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2062 return l
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2063
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2064 def stringFind(self, **requirements):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2065 """Locate a particular node by matching a set of its String
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2066 properties in a caseless search.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2067
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2068 If the property is not a String property, a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2069
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2070 The return is a list of the id of all nodes that match.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2071 """
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2072 where = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2073 args = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2074 for propname in requirements.keys():
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2075 prop = self.properties[propname]
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
2076 if not isinstance(prop, String):
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2077 raise TypeError, "'%s' not a String property"%propname
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2078 where.append(propname)
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2079 args.append(requirements[propname].lower())
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2080
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2081 # generate the where clause
1549
a53a7e197360 fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents: 1539
diff changeset
2082 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where])
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2083 sql = 'select id from _%s where %s and __retired__=%s'%(
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2084 self.classname, s, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2085 args.append(0)
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2086 self.db.sql(sql, tuple(args))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2087 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2088 l = [str(x[0]) for x in self.db.sql_fetchall()]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2089 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2090
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2091 def list(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2092 """ Return a list of the ids of the active nodes in this class.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2093 """
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2094 return self.getnodeids(retired=0)
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2095
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2096 def getnodeids(self, retired=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2097 """ Retrieve all the ids of the nodes for a particular Class.
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2098
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2099 Set retired=None to get all nodes. Otherwise it'll get all the
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2100 retired or non-retired nodes, depending on the flag.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2101 """
1707
3d627e34f18e sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1599
diff changeset
2102 # flip the sense of the 'retired' flag if we don't want all of them
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2103 if retired is not None:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2104 args = (0, )
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2105 if retired:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2106 compare = '>'
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2107 else:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2108 compare = '='
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2109 sql = 'select id from _%s where __retired__%s%s'%(self.classname,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2110 compare, self.db.arg)
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2111 else:
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2112 args = ()
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2113 sql = 'select id from _%s'%self.classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2114 self.db.sql(sql, args)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2115 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2116 ids = [str(x[0]) for x in self.db.cursor.fetchall()]
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2117 return ids
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2118
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2119 def _subselect(self, classname, multilink_table):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2120 """Create a subselect. This is factored out because some
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2121 databases (hmm only one, so far) doesn't support subselects
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2122 look for "I can't believe it's not a toy RDBMS" in the mysql
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2123 backend.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2124 """
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2125 return '_%s.id not in (select nodeid from %s)'%(classname,
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2126 multilink_table)
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2127
3685
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2128 # Some DBs order NULL values last. Set this variable in the backend
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2129 # for prepending an order by clause for each attribute that causes
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2130 # correct sort order for NULLs. Examples:
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2131 # order_by_null_values = '(%s is not NULL)'
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2132 # order_by_null_values = 'notnull(%s)'
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2133 # The format parameter is replaced with the attribute.
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2134 order_by_null_values = None
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2135
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2136 def filter(self, search_matches, filterspec, sort=[], group=[]):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2137 """Return a list of the ids of the active nodes in this class that
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2138 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2139 sort spec
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2140
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2141 "filterspec" is {propname: value(s)}
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2142
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2143 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-'
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2144 or None and prop is a prop name or None. Note that for
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2145 backward-compatibility reasons a single (dir, prop) tuple is
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2146 also allowed.
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2147
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2148 "search_matches" is a container type or None
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2149
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2150 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2151 value to match is a list:
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2152
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2153 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
2154 2. Other properties must match any of the elements in the list.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2155 """
2578
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2156 # we can't match anything if search_matches is empty
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 4039
diff changeset
2157 if not search_matches and search_matches is not None:
2578
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2158 return []
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2159
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2160 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2161 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2162
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2163 icn = self.classname
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2164
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2165 # vars to hold the components of the SQL statement
2586
3c67f4bfa225 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2578
diff changeset
2166 frum = [] # FROM clauses
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2167 loj = [] # LEFT OUTER JOIN clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2168 where = [] # WHERE clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2169 args = [] # *any* positional arguments
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2170 a = self.db.arg
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2171
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2172 # figure the WHERE clause from the filterspec
2256
0b198ed096af fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents: 2248
diff changeset
2173 mlfilt = 0 # are we joining with Multilink tables?
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2174 sortattr = self._sortattr (group = group, sort = sort)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2175 proptree = self._proptree(filterspec, sortattr)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2176 mlseen = 0
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2177 for pt in reversed(proptree.sortattr):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2178 p = pt
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2179 while p.parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2180 if isinstance (p.propclass, Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2181 mlseen = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2182 if mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2183 p.sort_ids_needed = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2184 p.tree_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2185 p = p.parent
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2186 if not mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2187 pt.attr_sort_done = pt.tree_sort_done = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2188 proptree.compute_sort_done()
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2189
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2190 ordercols = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2191 auxcols = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2192 mlsort = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2193 rhsnum = 0
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2194 for p in proptree:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2195 oc = None
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2196 cn = p.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2197 ln = p.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2198 pln = p.parent.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2199 pcn = p.parent.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2200 k = p.name
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2201 v = p.val
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2202 propclass = p.propclass
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2203 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2204 oc = ac = '_%s._%s'%(pln, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2205 if isinstance(propclass, Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2206 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2207 mlfilt = 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2208 tn = '%s_%s'%(pcn, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2209 if v in ('-1', ['-1']):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2210 # only match rows that have count(linkid)=0 in the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2211 # corresponding multilink table)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2212 where.append(self._subselect(pcn, tn))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2213 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2214 frum.append(tn)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2215 where.append('_%s.id=%s.nodeid'%(pln,tn))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2216 if p.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2217 frum.append('_%s as _%s' % (cn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2218 where.append('%s.linkid=_%s.id'%(tn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2219 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2220 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2221 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2222 where.append('%s.linkid in (%s)'%(tn, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2223 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2224 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2225 where.append('%s.linkid=%s'%(tn, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2226 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2227 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2228 assert not p.attr_sort_done and not p.sort_ids_needed
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2229 elif k == 'id':
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2230 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2231 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2232 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2233 where.append('_%s.%s in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2234 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2235 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2236 where.append('_%s.%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2237 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2238 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2239 oc = ac = '_%s.id'%pln
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2240 elif isinstance(propclass, String):
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2241 if p.sort_type < 2:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2242 if not isinstance(v, type([])):
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2243 v = [v]
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2244
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2245 # Quote the bits in the string that need it and then embed
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2246 # in a "substring" search. Note - need to quote the '%' so
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2247 # they make it through the python layer happily
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2248 v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v]
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2249
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2250 # now add to the where clause
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2251 where.append('('
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2252 +' and '.join(["_%s._%s LIKE '%s'"%(pln, k, s) for s in v])
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2253 +')')
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2254 # note: args are embedded in the query string now
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2255 if p.sort_type > 0:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2256 oc = ac = 'lower(_%s._%s)'%(pln, k)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2257 elif isinstance(propclass, Link):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2258 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2259 if p.children:
3694
595041d78104 Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3693
diff changeset
2260 if p.sort_type == 0:
595041d78104 Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3693
diff changeset
2261 frum.append('_%s as _%s' % (cn, ln))
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2262 where.append('_%s._%s=_%s.id'%(pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2263 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2264 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2265 d = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2266 for entry in v:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2267 if entry == '-1':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2268 entry = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2269 d[entry] = entry
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2270 l = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2271 if d.has_key(None) or not d:
3821
5ed4c9d30148 Handling of unset Link search in RDBMS backend
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
2272 if d.has_key(None): del d[None]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2273 l.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2274 if d:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2275 v = d.keys()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2276 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2277 l.append('(_%s._%s in (%s))'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2278 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2279 if l:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2280 where.append('(' + ' or '.join(l) +')')
3636
fa7becc62534 Of course for Links there is the same bug as I fixed before for multilinks.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3635
diff changeset
2281 else:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2282 if v in ('-1', None):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2283 v = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2284 where.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2285 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2286 where.append('_%s._%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2287 args.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2288 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2289 lp = p.cls.labelprop()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2290 oc = ac = '_%s._%s'%(pln, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2291 if lp != 'id':
3694
595041d78104 Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3693
diff changeset
2292 if p.tree_sort_done and p.sort_type > 0:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2293 loj.append(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2294 'LEFT OUTER JOIN _%s as _%s on _%s._%s=_%s.id'%(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2295 cn, ln, pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2296 oc = '_%s._%s'%(ln, lp)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2297 elif isinstance(propclass, Date) and p.sort_type < 2:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
2298 dc = self.db.to_sql_value(hyperdb.Date)
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2299 if isinstance(v, type([])):
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2300 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2301 where.append('_%s._%s in (%s)'%(pln, k, s))
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2302 args = args + [dc(date.Date(x)) for x in v]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2303 else:
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2304 try:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2305 # Try to filter on range of dates
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3554
diff changeset
2306 date_rng = propclass.range_from_raw(v, self.db)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2307 if date_rng.from_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2308 where.append('_%s._%s >= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2309 args.append(dc(date_rng.from_value))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2310 if date_rng.to_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2311 where.append('_%s._%s <= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2312 args.append(dc(date_rng.to_value))
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2313 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2314 # If range creation fails - ignore that search parameter
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2315 pass
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2316 elif isinstance(propclass, Interval):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2317 # filter/sort using the __<prop>_int__ column
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2318 if p.sort_type < 2:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2319 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2320 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2321 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2322 args = args + [date.Interval(x).as_seconds() for x in v]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2323 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2324 try:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2325 # Try to filter on range of intervals
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2326 date_rng = Range(v, date.Interval)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2327 if date_rng.from_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2328 where.append('_%s.__%s_int__ >= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2329 args.append(date_rng.from_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2330 if date_rng.to_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2331 where.append('_%s.__%s_int__ <= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2332 args.append(date_rng.to_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2333 except ValueError:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2334 # If range creation fails - ignore search parameter
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2335 pass
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2336 if p.sort_type > 0:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2337 oc = ac = '_%s.__%s_int__'%(pln,k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2338 elif p.sort_type < 2:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2339 if isinstance(v, type([])):
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2340 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2341 where.append('_%s._%s in (%s)'%(pln, k, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2342 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2343 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2344 where.append('_%s._%s=%s'%(pln, k, a))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2345 args.append(v)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2346 if oc:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2347 if p.sort_ids_needed:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2348 auxcols[ac] = p
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2349 if p.tree_sort_done and p.sort_direction:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2350 # Don't select top-level id twice
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2351 if p.name != 'id' or p.parent != proptree:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2352 ordercols.append(oc)
3685
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2353 desc = ['', ' desc'][p.sort_direction == '-']
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2354 # Some SQL dbs sort NULL values last -- we want them first.
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2355 if (self.order_by_null_values and p.name != 'id'):
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2356 nv = self.order_by_null_values % oc
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2357 ordercols.append(nv)
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2358 p.orderby.append(nv + desc)
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2359 p.orderby.append(oc + desc)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2360
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2361 props = self.getprops()
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2362
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2363 # don't match retired nodes
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2364 where.append('_%s.__retired__=0'%icn)
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2365
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2366 # add results of full text search
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2367 if search_matches is not None:
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 4039
diff changeset
2368 s = ','.join([a for x in search_matches])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2369 where.append('_%s.id in (%s)'%(icn, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2370 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2371
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2372 # construct the SQL
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2373 frum.append('_'+icn)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2374 frum = ','.join(frum)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2375 if where:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2376 where = ' where ' + (' and '.join(where))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2377 else:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2378 where = ''
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2379 if mlfilt:
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2380 # we're joining tables on the id, so we will get dupes if we
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2381 # don't distinct()
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2382 cols = ['distinct(_%s.id)'%icn]
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2383 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2384 cols = ['_%s.id'%icn]
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2385 if ordercols:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2386 cols = cols + ordercols
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2387 order = []
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2388 # keep correct sequence of order attributes.
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2389 for sa in proptree.sortattr:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2390 if not sa.attr_sort_done:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2391 continue
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2392 order.extend(sa.orderby)
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2393 if order:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2394 order = ' order by %s'%(','.join(order))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2395 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2396 order = ''
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2397 for o, p in auxcols.iteritems ():
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2398 cols.append (o)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2399 p.auxcol = len (cols) - 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2400
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2401 cols = ','.join(cols)
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2402 loj = ' '.join(loj)
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2403 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2404 args = tuple(args)
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2405 __traceback_info__ = (sql, args)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2406 self.db.sql(sql, args)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
2407 l = self.db.sql_fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2408
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2409 # Compute values needed for sorting in proptree.sort
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2410 for p in auxcols.itervalues():
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2411 p.sort_ids = p.sort_result = [row[p.auxcol] for row in l]
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2412 # return the IDs (the first column)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2413 # XXX numeric ids
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2414 l = [str(row[0]) for row in l]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2415 l = proptree.sort (l)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2416
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2417 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2418 self.db.stats['filtering'] += (time.time() - start_t)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2419 return l
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2420
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2421 def filter_sql(self, sql):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2422 """Return a list of the ids of the items in this class that match
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2423 the SQL provided. The SQL is a complete "select" statement.
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2424
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2425 The SQL select must include the item id as the first column.
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2426
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2427 This function DOES NOT filter out retired items, add on a where
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2428 clause "__retired__=0" if you don't want retired nodes.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2429 """
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2430 if __debug__:
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2431 start_t = time.time()
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2432
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2433 self.db.sql(sql)
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2434 l = self.db.sql_fetchall()
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2435
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2436 if __debug__:
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2437 self.db.stats['filtering'] += (time.time() - start_t)
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2438 return l
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2439
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2440 def count(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2441 """Get the number of nodes in this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2442
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2443 If the returned integer is 'numnodes', the ids of all the nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2444 in this class run from 1 to numnodes, and numnodes+1 will be the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2445 id of the next node to be created in this class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2446 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2447 return self.db.countnodes(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2448
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2449 # Manipulating properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2450 def getprops(self, protected=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2451 """Return a dictionary mapping property names to property objects.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2452 If the "protected" flag is true, we include protected properties -
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2453 those which may not be modified.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2454 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2455 d = self.properties.copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2456 if protected:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2457 d['id'] = String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2458 d['creation'] = hyperdb.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2459 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1175
diff changeset
2460 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2461 d['actor'] = hyperdb.Link('user')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2462 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2463
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2464 def addprop(self, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2465 """Add properties to this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2466
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2467 The keyword arguments in 'properties' must map names to property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2468 objects, or a TypeError is raised. None of the keys in 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2469 may collide with the names of existing properties, or a ValueError
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2470 is raised before any properties have been added.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2471 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2472 for key in properties.keys():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2473 if self.properties.has_key(key):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2474 raise ValueError, key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2475 self.properties.update(properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2476
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2477 def index(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2478 """Add (or refresh) the node to search indexes
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2479 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2480 # find all the String properties that have indexme
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2481 for prop, propclass in self.getprops().items():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2482 if isinstance(propclass, String) and propclass.indexme:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2483 self.db.indexer.add_text((self.classname, nodeid, prop),
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2484 str(self.get(nodeid, prop)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2485
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2486 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2487 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2488 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2489 def export_list(self, propnames, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2490 """ Export a node - generate a list of CSV-able data in the order
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2491 specified by propnames for the given node.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2492 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2493 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2494 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2495 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2496 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2497 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2498 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2499 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2500 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2501 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2502 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2503 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2504 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2505 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2506 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2507 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2508 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2509 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2510
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2511 def import_list(self, propnames, proplist):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2512 """ Import a node - all information including "id" is present and
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2513 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2514 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2515 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2516
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2517 Return the nodeid of the node imported.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2518 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2519 if self.db.journaltag is None:
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
2520 raise DatabaseError, _('Database open read-only')
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2521 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2522
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2523 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2524 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2525 retire = 0
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2526 if not "id" in propnames:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2527 newid = self.db.newid(self.classname)
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2528 else:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2529 newid = eval(proplist[propnames.index("id")])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2530 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2531 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2532 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2533
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2534 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2535 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2536
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2537 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2538 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2539 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2540 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2541 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2542 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2543 retire = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2544 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2545 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2546 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2547 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2548
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2549 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2550 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2551 # don't set Nones
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2552 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2553 elif isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2554 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2555 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2556 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2557 elif isinstance(prop, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2558 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2559 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2560 value = pwd
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2561 elif isinstance(prop, String):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2562 if isinstance(value, unicode):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2563 value = value.encode('utf8')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2564 if not isinstance(value, str):
2691
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2565 raise TypeError, \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2566 'new property "%(propname)s" not a string: %(value)r' \
4008af78746a Class.import_list():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2690
diff changeset
2567 % locals()
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2568 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2569 self.db.indexer.add_text((self.classname, newid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2570 value)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2571 d[propname] = value
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2572
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2573 # get a new id if necessary
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2574 if newid is None:
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2575 newid = self.db.newid(self.classname)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2576
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2577 # insert new node or update existing?
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2578 if not self.hasnode(newid):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2579 self.db.addnode(self.classname, newid, d) # insert
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2580 else:
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2581 self.db.setnode(self.classname, newid, d) # update
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2582
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2583 # retire?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2584 if retire:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2585 # use the arg for __retired__ to cope with any odd database type
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2586 # conversion (hello, sqlite)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2587 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2588 self.db.arg, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2589 self.db.sql(sql, (newid, newid))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2590 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2591
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2592 def export_journals(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2593 """Export a class's journal - generate a list of lists of
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2594 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2595
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2596 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2597
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2598 No heading here - the columns are fixed.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2599 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2600 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2601 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2602 for nodeid in self.getnodeids():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2603 for nodeid, date, user, action, params in self.history(nodeid):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2604 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2605 if action == 'set':
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2606 export_data = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2607 for propname, value in params.items():
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2608 if not properties.has_key(propname):
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2609 # property no longer in the schema
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2610 continue
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2611
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2612 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2613 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2614 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2615 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2616 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2617 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2618 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2619 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2620 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2621 value = str(value)
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2622 export_data[propname] = value
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2623 params = export_data
3933
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2624 elif action == 'create' and params:
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2625 # old tracker with data stored in the create!
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2626 params = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2627 l = [nodeid, date, user, action, params]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2628 r.append(map(repr, l))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2629 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2630
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2631 def import_journals(self, entries):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2632 """Import a class's journal.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2633
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2634 Uses setjournal() to set the journal for each item."""
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2635 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2636 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2637 for l in entries:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2638 l = map(eval, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2639 nodeid, jdate, user, action, params = l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2640 r = d.setdefault(nodeid, [])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2641 if action == 'set':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2642 for propname, value in params.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2643 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2644 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2645 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2646 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2647 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2648 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2649 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2650 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2651 pwd = password.Password()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2652 pwd.unpack(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2653 value = pwd
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2654 params[propname] = value
3933
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2655 elif action == 'create' and params:
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2656 # old tracker with data stored in the create!
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
2657 params = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2658 r.append((nodeid, date.Date(jdate), user, action, params))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2659
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2660 for nodeid, l in d.items():
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2661 self.db.setjournal(self.classname, nodeid, l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2662
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2586
diff changeset
2663 class FileClass(hyperdb.FileClass, Class):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2664 """This class defines a large chunk of data. To support this, it has a
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2665 mandatory String property "content" which is typically saved off
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2666 externally to the hyperdb.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2667
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2668 The default MIME type of this data is defined by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2669 "default_mime_type" class attribute, which may be overridden by each
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2670 node if the class defines a "type" String property.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2671 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2672 def __init__(self, db, classname, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2673 """The newly-created class automatically includes the "content"
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2674 and "type" properties.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2675 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2676 if not properties.has_key('content'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2677 properties['content'] = hyperdb.String(indexme='yes')
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2678 if not properties.has_key('type'):
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2679 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2680 Class.__init__(self, db, classname, **properties)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2681
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2682 def create(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2683 """ snaffle the file propvalue and store in a file
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2684 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2685 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2686 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2687 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2688
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2689 # now remove the content property so it's not stored in the db
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2690 content = propvalues['content']
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2691 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2692
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2693 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2694 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2695
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2696 # figure the mime type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2697 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2698
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2699 # and index!
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2700 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2701 self.db.indexer.add_text((self.classname, newid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2702 content, mime_type)
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2703
3979
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
2704 # store off the content as a file
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
2705 self.db.storefile(self.classname, newid, None, content)
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
2706
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2707 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2708 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
2709
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2710 return newid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2711
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2712 def get(self, nodeid, propname, default=_marker, cache=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2713 """ Trap the content propname and get it from the file
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2714
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2715 'cache' exists for backwards compatibility, and is not used.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2716 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2717 poss_msg = 'Possibly a access right configuration problem.'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2718 if propname == 'content':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2719 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2720 return self.db.getfile(self.classname, nodeid, None)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2721 except IOError, (strerror):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2722 # BUG: by catching this we donot see an error in the log.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2723 return 'ERROR reading file: %s%s\n%s\n%s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2724 self.classname, nodeid, poss_msg, strerror)
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2725 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2726 return Class.get(self, nodeid, propname, default)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2727 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2728 return Class.get(self, nodeid, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2729
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2730 def set(self, itemid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2731 """ Snarf the "content" propvalue and update it in a file
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2732 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2733 self.fireAuditors('set', itemid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2734 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2735
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2736 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2737 content = None
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2738 if propvalues.has_key('content'):
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2739 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2740 del propvalues['content']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2741
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2742 # do the database create
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2743 propvalues = self.set_inner(itemid, **propvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2744
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2745 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2746 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2747 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2748 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2749 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2750 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2751 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2752 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2753 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
2754
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2755 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2756 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2757 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2758
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2759 def index(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2760 """ Add (or refresh) the node to search indexes.
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2761
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2762 Use the content-type property for the content property.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2763 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2764 # find all the String properties that have indexme
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2765 for prop, propclass in self.getprops().items():
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2766 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2767 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2768 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2769 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2770 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2771 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2772 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2773 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2774 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2775 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2776 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
2777 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2778
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2779 # XXX deviation from spec - was called ItemClass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2780 class IssueClass(Class, roundupdb.IssueClass):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2781 # Overridden methods:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2782 def __init__(self, db, classname, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2783 """The newly-created class automatically includes the "messages",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2784 "files", "nosy", and "superseder" properties. If the 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2785 dictionary attempts to specify any of these properties or a
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2786 "creation", "creator", "activity" or "actor" property, a ValueError
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2787 is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2788 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2789 if not properties.has_key('title'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2790 properties['title'] = hyperdb.String(indexme='yes')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2791 if not properties.has_key('messages'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2792 properties['messages'] = hyperdb.Multilink("msg")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2793 if not properties.has_key('files'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2794 properties['files'] = hyperdb.Multilink("file")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2795 if not properties.has_key('nosy'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2796 # note: journalling is turned off as it really just wastes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2797 # space. this behaviour may be overridden in an instance
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2798 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2799 if not properties.has_key('superseder'):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2800 properties['superseder'] = hyperdb.Multilink(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2801 Class.__init__(self, db, classname, **properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2802
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
2803 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/