Mercurial > p > roundup > code

annotate roundup/backends/rdbms_common.py @ 5319:62de601bdf6f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix commits although a Reject exception is raised Fix the problem that changes are committed to the database (due to commits to otk handling) even when a Reject exception occurs. The fix implements separate database connections for otk/session handling and normal database operation.
author Ralf Schlatterbeck <rsc@runtux.com>
date Fri, 20 Apr 2018 18:46:28 +0200
parents 506c7ee9a385
children f7432fc3db4d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
1 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
4 # under the same terms as Python, so long as this copyright message and
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
5 # disclaimer are retained in their original form.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
6 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
11 #
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
17 #
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
18 """ Relational database (SQL) backend common code.
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
19
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
20 Basics:
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
21
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
22 - map roundup classes to relational tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
23 - automatically detect schema changes and modify the table schemas
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
24 appropriately (we store the "database version" of the schema in the
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
25 database itself as the only row of the "schema" table)
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
26 - multilinks (which represent a many-to-many relationship) are handled through
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
27 intermediate tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
28 - journals are stored adjunct to the per-class tables
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
29 - table names and columns have "_" prepended so the names can't clash with
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
30 restricted names (like "order")
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
31 - retirement is determined by the __retired__ column being > 0
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
32
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
33 Database-specific changes may generally be pushed out to the overridable
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
34 sql_* methods, since everything else should be fairly generic. There's
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
35 probably a bit of work to be done if a database is used that actually
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1222
diff changeset
36 honors column typing, since the initial databases don't (sqlite stores
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
37 everything as a string.)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
38
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
39 The schema of the hyperdb being mapped to the database is stored in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
40 database itself as a repr()'ed dictionary of information about each Class
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
41 that maps to a table. If that information differs from the hyperdb schema,
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
42 then we update it. We also store in the schema dict a version which
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
43 allows us to upgrade the database schema when necessary. See upgrade_db().
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
44
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
45 To force a unqiueness constraint on the key properties we put the item
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
46 id into the __retired__ column duing retirement (so it's 0 for "active"
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
47 items) and place a unqiueness constraint on key + __retired__. This is
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
48 particularly important for the users class where multiple users may
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
49 try to have the same username, with potentially many retired users with
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
50 the same name.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
51 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
52 __docformat__ = 'restructuredtext'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 # standard python modules
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
55 import sys, os, time, re, errno, weakref, copy, logging, datetime
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 # roundup modules
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
58 from roundup import hyperdb, date, password, roundupdb, security, support
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 from roundup.hyperdb import String, Password, Date, Interval, Link, \
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
60 Multilink, DatabaseError, Boolean, Number, Integer, Node
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
61 from roundup.backends import locking
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
62 from roundup.i18n import _
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
64
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 # support
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
66 from roundup.backends.blobfiles import FileStorage
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5086
diff changeset
67 from roundup.backends.indexer_common import get_indexer
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
68 from roundup.backends.sessions_rdbms import Sessions, OneTimeKeys
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
69 from roundup.date import Range
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
71 from roundup.backends.back_anydbm import compile_expression
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
72
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
73
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
74 # dummy value meaning "argument not passed"
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
75 _marker = []
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
76
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
77 def _num_cvt(num):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
78 num = str(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
79 try:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
80 return int(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
81 except:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
82 return float(num)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
83
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
84 def _bool_cvt(value):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
85 if value in ('TRUE', 'FALSE'):
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
86 return {'TRUE': 1, 'FALSE': 0}[value]
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
87 # assume it's a number returned from the db API
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
88 return int(value)
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
89
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
90 def date_to_hyperdb_value(d):
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
91 """ convert date d to a roundup date """
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
92 if isinstance (d, datetime.datetime):
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
93 return date.Date(d)
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
94 return date.Date (str(d).replace(' ', '.'))
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
95
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
96
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
97 def connection_dict(config, dbnamestr=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
98 """ Used by Postgresql and MySQL to detemine the keyword args for
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
99 opening the database connection."""
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
100 d = { }
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
101 if dbnamestr:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
102 d[dbnamestr] = config.RDBMS_NAME
3099
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
103 for name in ('host', 'port', 'password', 'user', 'read_default_group',
519b92df37dc handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
104 'read_default_file'):
2634
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
105 cvar = 'RDBMS_'+name.upper()
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
106 if config[cvar] is not None:
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
107 d[name] = config[cvar]
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
108 return d
f47ca4541770 Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents: 2597
diff changeset
109
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
110
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
111 class IdListOptimizer:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
112 """ To prevent flooding the SQL parser of the underlaying
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
113 db engine with "x IN (1, 2, 3, ..., <large number>)" collapses
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
114 these cases to "x BETWEEN 1 AND <large number>".
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
115 """
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
116
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
117 def __init__(self):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
118 self.ranges = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
119 self.singles = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
120
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
121 def append(self, nid):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
122 """ Invariant: nid are ordered ascending """
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
123 if self.ranges:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
124 last = self.ranges[-1]
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
125 if last[1] == nid-1:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
126 last[1] = nid
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
127 return
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
128 if self.singles:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
129 last = self.singles[-1]
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
130 if last == nid-1:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
131 self.singles.pop()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
132 self.ranges.append([last, nid])
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
133 return
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
134 self.singles.append(nid)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
135
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
136 def where(self, field, placeholder):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
137 ranges = self.ranges
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
138 singles = self.singles
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
139
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
140 if not singles and not ranges: return "(1=0)", []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
141
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
142 if ranges:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
143 between = '%s BETWEEN %s AND %s' % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
144 field, placeholder, placeholder)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
145 stmnt = [between] * len(ranges)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
146 else:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
147 stmnt = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
148 if singles:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
149 stmnt.append('%s in (%s)' % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
150 field, ','.join([placeholder]*len(singles))))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
151
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
152 return '(%s)' % ' OR '.join(stmnt), sum(ranges, []) + singles
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
153
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
154 def __str__(self):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
155 return "ranges: %r / singles: %r" % (self.ranges, self.singles)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
156
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
157
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
159 """ Wrapper around an SQL database that presents a hyperdb interface.
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
160
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
161 - some functionality is specific to the actual SQL database, hence
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
162 the sql_* methods that are NotImplemented
4074
e039f3cbbb96 Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents: 4062
diff changeset
163 - we keep a cache of the latest N row fetches (where N is configurable).
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
164 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 def __init__(self, config, journaltag=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
166 """ Open the database and load the schema from it.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
167 """
3897
e2c2d91932ad respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3872
diff changeset
168 FileStorage.__init__(self, config.UMASK)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
169 self.config, self.journaltag = config, journaltag
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
170 self.dir = config.DATABASE
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171 self.classes = {}
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5086
diff changeset
172 self.indexer = get_indexer(config, self)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 self.security = security.Security(self)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
175 # additional transaction support for external files and the like
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
176 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
177
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
178 # keep a cache of the N most recently retrieved rows of any kind
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
179 # (classname, nodeid) = row
4074
e039f3cbbb96 Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents: 4062
diff changeset
180 self.cache_size = config.RDBMS_CACHE_SIZE
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
181 self.clearCache()
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
182 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
183 'filtering': 0}
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
184
5041
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
185 # make sure the database directory exists
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
186 if not os.path.isdir(self.config.DATABASE):
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
187 os.makedirs(self.config.DATABASE)
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
188
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
189 # database lock
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
190 self.lockfile = None
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1304
diff changeset
191
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
192 # Uppercase to not collide with Class names
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
193 self.Session = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
194 self.Otk = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
195
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 # open a connection to the database, creating the "conn" attribute
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
197 self.open_connection()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
199 def clearCache(self):
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
200 self.cache = {}
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
201 self.cache_lru = []
4652
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4517
diff changeset
202 # upcall is necessary!
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4517
diff changeset
203 roundupdb.Database.clearCache(self)
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
204
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
205 def getSessionManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
206 if not self.Session:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
207 self.Session = Sessions(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
208 return self.Session
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
209
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
210 def getOTKManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
211 if not self.Otk:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
212 self.Otk = OneTimeKeys(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
213 return self.Otk
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
214
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2081
diff changeset
215 def open_connection(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
216 """ Open a connection to the database, creating it if necessary.
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
217
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
218 Must call self.load_dbschema()
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
219 """
5236
0a20e1965f4b Pylint checks bark about:
John Rouillard <rouilj@ieee.org>
parents: 5232
diff changeset
220 raise NotImplementedError
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
222 def sql(self, sql, args=None, cursor=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
223 """ Execute the sql with the optional args.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
224 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
225 self.log_debug('SQL %r %r'%(sql, args))
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
226 if not cursor:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
227 cursor = self.cursor
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
228 if args:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
229 cursor.execute(sql, args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230 else:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
231 cursor.execute(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
233 def sql_fetchone(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
234 """ Fetch a single row. If there's nothing to fetch, return None.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
235 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
236 return self.cursor.fetchone()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
237
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
238 def sql_fetchall(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
239 """ Fetch all rows. If there's nothing to fetch, return [].
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
240 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
241 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
243 def sql_fetchiter(self):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
244 """ Fetch all row as a generator
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
245 """
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
246 while True:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
247 row = self.cursor.fetchone()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
248 if not row: break
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
249 yield row
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
250
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
251 def search_stringquote(self, value):
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
252 """ Quote a search string to escape magic search characters
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
253 '%' and '_', also need to quote '\' (first)
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
254 Then put '%' around resulting string for LIKE (or ILIKE) operator
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
255 """
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
256 v = value.replace('\\', '\\\\')
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
257 v = v.replace('%', '\\%')
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
258 v = v.replace('_', '\\_')
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
259 return '%' + v + '%'
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
260
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
261
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
262 def init_dbschema(self):
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
263 self.database_schema = {
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
264 'version': self.current_db_version,
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
265 'tables': {}
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
266 }
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
267
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
268 def load_dbschema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
269 """ Load the schema definition that the database currently implements
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
270 """
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
271 self.cursor.execute('select schema from schema')
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
272 schema = self.cursor.fetchone()
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
273 if schema:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
274 self.database_schema = eval(schema[0])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
275 else:
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
276 self.database_schema = {}
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
277
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
278 def save_dbschema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
279 """ Save the schema definition that the database currently implements
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
280 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
281 s = repr(self.database_schema)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
282 self.sql('delete from schema')
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3694
diff changeset
283 self.sql('insert into schema values (%s)'%self.arg, (s,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
284
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
285 def post_init(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
286 """ Called once the schema initialisation has finished.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
287
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
288 We should now confirm that the schema defined by our "classes"
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
289 attribute actually matches the schema in the database.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
290 """
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
291
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
292 # upgrade the database for column type changes, new internal
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
293 # tables, etc.
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
294 save = self.upgrade_db()
3553
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
295
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
296 # handle changes in the schema
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
297 tables = self.database_schema['tables']
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
298 for classname, spec in self.classes.iteritems():
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
299 if classname in tables:
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
300 dbspec = tables[classname]
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
301 if self.update_class(spec, dbspec):
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
302 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
303 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 self.create_class(spec)
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
306 tables[classname] = spec.schema()
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
307 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
308
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
309 for classname, spec in list(tables.items()):
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
310 if classname not in self.classes:
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
311 self.drop_class(classname, tables[classname])
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
312 del tables[classname]
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
313 save = 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
315 # update the database version of the schema
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
316 if save:
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
317 self.save_dbschema()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
318
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
319 # reindex the db if necessary
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
320 if self.indexer.should_reindex():
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
321 self.reindex()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
322
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
323 # commit
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
324 self.sql_commit()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
326 # update this number when we need to make changes to the SQL structure
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
327 # of the backen database
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
328 current_db_version = 5
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
329 db_version_updated = False
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
330 def upgrade_db(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
331 """ Update the SQL database to reflect changes in the backend code.
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
332
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
333 Return boolean whether we need to save the schema.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
334 """
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
335 version = self.database_schema.get('version', 1)
3993
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
336 if version > self.current_db_version:
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
337 raise DatabaseError('attempting to run rev %d DATABASE with rev '
1eba65a4b0aa Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents: 3979
diff changeset
338 '%d CODE!'%(version, self.current_db_version))
2081
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
339 if version == self.current_db_version:
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
340 # nothing to do
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
341 return 0
fb4bf55b94d7 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
342
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
343 if version < 2:
4062
6ea417bafd9c Fix typo.
Stefan Seefeld <stefan@seefeld.name>
parents: 4061
diff changeset
344 self.log_info('upgrade to version 2')
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
345 # change the schema structure
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
346 self.database_schema = {'tables': self.database_schema}
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
347
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
348 # version 1 didn't have the actor column (note that in
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
349 # MySQL this will also transition the tables to typed columns)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
350 self.add_new_columns_v2()
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
351
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
352 # version 1 doesn't have the OTK, session and indexing in the
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
353 # database
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
354 self.create_version_2_tables()
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
355
2429
e28a5d5d95a7 better upgrading
Richard Jones <richard@users.sourceforge.net>
parents: 2424
diff changeset
356 if version < 3:
4062
6ea417bafd9c Fix typo.
Stefan Seefeld <stefan@seefeld.name>
parents: 4061
diff changeset
357 self.log_info('upgrade to version 3')
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
358 self.fix_version_2_tables()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
359
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
360 if version < 4:
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
361 self.log_info('upgrade to version 4')
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
362 self.fix_version_3_tables()
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
363
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
364 if version < 5:
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
365 self.log_info('upgrade to version 5')
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
366 self.fix_version_4_tables()
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
367
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
368 self.database_schema['version'] = self.current_db_version
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
369 self.db_version_updated = True
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
370 return 1
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
371
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
372 def fix_version_3_tables(self):
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
373 # drop the shorter VARCHAR OTK column and add a new TEXT one
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
374 for name in ('otk', 'session'):
2755
e3cd28cec23d Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents: 2733
diff changeset
375 self.sql('DELETE FROM %ss'%name)
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
376 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
377 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name))
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2720
diff changeset
378
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
379 def fix_version_2_tables(self):
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
380 # Default (used by sqlite): NOOP
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
381 pass
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
382
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
383 def fix_version_4_tables(self):
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
384 # note this is an explicit call now
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
385 c = self.cursor
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
386 for cn, klass in self.classes.iteritems():
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
387 c.execute('select id from _%s where __retired__<>0'%(cn,))
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
388 for (id,) in c.fetchall():
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
389 c.execute('update _%s set __retired__=%s where id=%s'%(cn,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
390 self.arg, self.arg), (id, id))
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
391
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
392 if klass.key:
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
393 self.add_class_key_required_unique_constraint(cn, klass.key)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
394
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
395 def _convert_journal_tables(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
396 """Get current journal table contents, drop the table and re-create"""
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
397 c = self.cursor
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
398 cols = ','.join('nodeid date tag action params'.split())
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
399 for klass in self.classes.itervalues():
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
400 # slurp and drop
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
401 sql = 'select %s from %s__journal order by date'%(cols,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
402 klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
403 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
404 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
405 self.drop_journal_table_indexes(klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
406 c.execute('drop table %s__journal'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
407
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
408 # re-create and re-populate
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
409 self.create_journal_table(klass)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
410 a = self.arg
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
411 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
412 klass.classname, cols, a, a, a, a, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
413 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
414 # no data conversion needed
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
415 self.cursor.execute(sql, row)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
416
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
417 def _convert_string_properties(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
418 """Get current Class tables that contain String properties, and
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
419 convert the VARCHAR columns to TEXT"""
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
420 c = self.cursor
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
421 for klass in self.classes.itervalues():
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
422 # slurp and drop
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
423 cols, mls = self.determine_columns(list(klass.properties.iteritems()))
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
424 scols = ','.join([i[0] for i in cols])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
425 sql = 'select id,%s from _%s'%(scols, klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
426 c.execute(sql)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
427 contents = c.fetchall()
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
428 self.drop_class_table_indexes(klass.classname, klass.getkey())
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
429 c.execute('drop table _%s'%klass.classname)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
430
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
431 # re-create and re-populate
2424
74474ec41050 argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents: 2416
diff changeset
432 self.create_class_table(klass, create_sequence=0)
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
433 a = ','.join([self.arg for i in range(len(cols)+1)])
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
434 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname,
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
435 scols, a)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
436 for row in contents:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
437 l = []
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
438 for entry in row:
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
439 # mysql will already be a string - psql needs "help"
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
440 if entry is not None and not isinstance(entry, type('')):
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
441 entry = str(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
442 l.append(entry)
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
443 self.cursor.execute(sql, l)
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
444
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
445 def refresh_database(self):
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
446 self.post_init()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
447
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
448
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
449 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
450 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
451 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
452 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
453 classes = list(self.classes.itervalues())
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
454 for klass in classes:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
455 if show_progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
456 for nodeid in support.Progress('Reindex %s'%klass.classname,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
457 klass.list()):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
458 klass.index(nodeid)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
459 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
460 for nodeid in klass.list():
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
461 klass.index(nodeid)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
462 self.indexer.save_index()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
463
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
464 # Used here in the generic backend to determine if the database
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
465 # supports 'DOUBLE PRECISION' for floating point numbers.
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
466 implements_double_precision = True
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
467
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
468 hyperdb_to_sql_datatypes = {
2413
7d0bb6601809 fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents: 2379
diff changeset
469 hyperdb.String : 'TEXT',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
470 hyperdb.Date : 'TIMESTAMP',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
471 hyperdb.Link : 'INTEGER',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
472 hyperdb.Interval : 'VARCHAR(255)',
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
473 hyperdb.Password : 'VARCHAR(255)',
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
474 hyperdb.Boolean : 'BOOLEAN',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
475 hyperdb.Number : 'REAL',
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
476 hyperdb.Integer : 'INTEGER',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
477 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
478
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
479 def hyperdb_to_sql_datatype(self, propclass, prop = None):
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
480
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
481 datatype = self.hyperdb_to_sql_datatypes.get(propclass)
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
482 if self.implements_double_precision and prop and \
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
483 isinstance(prop, Number) and prop.use_double:
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
484 datatype = 'DOUBLE PRECISION'
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
485 if datatype:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
486 return datatype
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
487
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
488 for k, v in self.hyperdb_to_sql_datatypes.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
489 if issubclass(propclass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
490 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
491
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
492 raise ValueError('%r is not a hyperdb property class' % propclass)
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
493
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
494 def determine_columns(self, properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
495 """ Figure the column names and multilink properties from the spec
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
496
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
497 "properties" is a list of (name, prop) where prop may be an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
498 instance of a hyperdb "type" _or_ a string repr of that type.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
499 """
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
500 cols = [
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
501 ('_actor', self.hyperdb_to_sql_datatype(hyperdb.Link)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
502 ('_activity', self.hyperdb_to_sql_datatype(hyperdb.Date)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
503 ('_creator', self.hyperdb_to_sql_datatype(hyperdb.Link)),
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
504 ('_creation', self.hyperdb_to_sql_datatype(hyperdb.Date)),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
505 ]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
506 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
507 # add the multilinks separately
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
508 for col, prop in properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
509 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
510 mls.append(col)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
511 continue
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
512
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
513 if isinstance(prop, type('')):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
514 raise ValueError("string property spec!")
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
515 #and prop.find('Multilink') != -1:
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
516 #mls.append(col)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
517
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
518 datatype = self.hyperdb_to_sql_datatype(prop.__class__, prop)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
519 cols.append(('_'+col, datatype))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
520
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
521 # Intervals stored as two columns
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
522 if isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
523 cols.append(('__'+col+'_int__', 'BIGINT'))
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
524
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
525 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
526 return cols, mls
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
527
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
528 def update_class(self, spec, old_spec, force=0):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
529 """ Determine the differences between the current spec and the
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
530 database version of the spec, and update where necessary.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
531
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
532 If 'force' is true, update the database anyway.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
533 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
534 new_spec = spec.schema()
1515
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
535 new_spec[1].sort()
a516bbb9896b fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents: 1500
diff changeset
536 old_spec[1].sort()
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1838
diff changeset
537 if not force and new_spec == old_spec:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
538 # no changes
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
539 return 0
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
540
4474
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
541 if not self.config.RDBMS_ALLOW_ALTER:
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
542 raise DatabaseError(_('ALTER operation disallowed: %r -> %r.'%(old_spec, new_spec)))
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
543
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
544 logger = logging.getLogger('roundup.hyperdb.backend')
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
545 logger.info('update_class %s'%spec.classname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
546
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
547 logger.debug('old_spec %r'%(old_spec,))
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
548 logger.debug('new_spec %r'%(new_spec,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
549
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
550 # detect key prop change for potential index change
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
551 keyprop_changes = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
552 if new_spec[0] != old_spec[0]:
3554
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
553 if old_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
554 keyprop_changes['remove'] = old_spec[0]
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
555 if new_spec[0]:
5e70726a86dd fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents: 3553
diff changeset
556 keyprop_changes['add'] = new_spec[0]
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
557
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
558 # detect multilinks that have been removed, and drop their table
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
559 old_has = {}
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
560 for name, prop in old_spec[1]:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
561 old_has[name] = 1
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
562 if name in spec.properties:
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
563 continue
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
564
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
565 if prop.find('Multilink to') != -1:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
566 # first drop indexes.
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
567 self.drop_multilink_table_indexes(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
568
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
569 # now the multilink table itself
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
570 sql = 'drop table %s_%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
571 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
572 # if this is the key prop, drop the index first
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
573 if old_spec[0] == prop:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
574 self.drop_class_table_key_index(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
575 del keyprop_changes['remove']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
576
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
577 # drop the column
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
578 sql = 'alter table _%s drop column _%s'%(spec.classname, name)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
579
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
580 self.sql(sql)
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
581
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
582 # if we didn't remove the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
583 # changed, we still need to remove the old index
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
584 if 'remove' in keyprop_changes:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
585 self.drop_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
586 keyprop_changes['remove'])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
587
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
588 # add new columns
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
589 for propname, prop in new_spec[1]:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
590 if propname in old_has:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
591 continue
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
592 prop = spec.properties[propname]
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
593 if isinstance(prop, Multilink):
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
594 self.create_multilink_table(spec, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
595 else:
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
596 # add the column
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5112
diff changeset
597 coltype = self.hyperdb_to_sql_datatype(prop.__class__, prop)
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
598 sql = 'alter table _%s add column _%s %s'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
599 spec.classname, propname, coltype)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
600 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
601
2727
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
602 # extra Interval column
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
603 if isinstance(prop, Interval):
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
604 sql = 'alter table _%s add column __%s_int__ BIGINT'%(
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
605 spec.classname, propname)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
606 self.sql(sql)
93e2e5b55a3c new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
607
2196
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
608 # if the new column is a key prop, we need an index!
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
609 if new_spec[0] == propname:
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
610 self.create_class_table_key_index(spec.classname, propname)
85954067e496 mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents: 2185
diff changeset
611 del keyprop_changes['add']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
612
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
613 # if we didn't add the key prop just then, but the key prop has
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
614 # changed, we still need to add the new index
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
615 if 'add' in keyprop_changes:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
616 self.create_class_table_key_index(spec.classname,
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
617 keyprop_changes['add'])
1479
405e91b5be46 fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
618
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
619 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
620
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
621 def determine_all_columns(self, spec):
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
622 """Figure out the columns from the spec and also add internal columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
623
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
624 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
625 cols, mls = self.determine_columns(list(spec.properties.iteritems()))
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
626
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
627 # add on our special columns
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
628 cols.append(('id', 'INTEGER PRIMARY KEY'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
629 cols.append(('__retired__', 'INTEGER DEFAULT 0'))
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
630 return cols, mls
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
631
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
632 def create_class_table(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
633 """Create the class table for the given Class "spec". Creates the
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
634 indexes too."""
3016
224c7c0b9708 First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2985
diff changeset
635 cols, mls = self.determine_all_columns(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
636
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
637 # create the base table
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
638 scols = ','.join(['%s %s'%x for x in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
639 sql = 'create table _%s (%s)'%(spec.classname, scols)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
640 self.sql(sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
641
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
642 self.create_class_table_indexes(spec)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
643
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
644 return cols, mls
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
645
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
646 def create_class_table_indexes(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
647 """ create the class table for the given spec
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
648 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
649 # create __retired__ index
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
650 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%(
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
651 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
652 self.sql(index_sql2)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
653
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
654 # create index for key property
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
655 if spec.key:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
656 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%(
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
657 spec.classname, spec.key,
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
658 spec.classname, spec.key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
659 self.sql(index_sql3)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
660
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
661 # and the unique index for key / retired(id)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
662 self.add_class_key_required_unique_constraint(spec.classname,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
663 spec.key)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
664
2228
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
665 # TODO: create indexes on (selected?) Link property columns, as
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
666 # they're more likely to be used for lookup
1d1362c54c94 Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 2217
diff changeset
667
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
668 def add_class_key_required_unique_constraint(self, cn, key):
3969
905faf52a51f fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents: 3967
diff changeset
669 sql = '''create unique index _%s_key_retired_idx
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
670 on _%s(__retired__, _%s)'''%(cn, cn, key)
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
671 try:
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
672 self.sql(sql)
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
673 except StandardError:
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
674 # XXX catch e.g.:
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
675 # _sqlite.DatabaseError: index _status_key_retired_idx already exists
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4652
diff changeset
676 pass
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
677
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
678 def drop_class_table_indexes(self, cn, key):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
679 # drop the old table indexes first
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
680 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
681 if key:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
682 l.append('_%s_%s_idx'%(cn, key))
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
683
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
684 table_name = '_%s'%cn
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
685 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
686 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
687 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
688 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
689 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
690
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
691 def create_class_table_key_index(self, cn, key):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
692 """ create the class table for the given spec
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
693 """
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
694 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
695 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
696
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
697 def drop_class_table_key_index(self, cn, key):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
698 table_name = '_%s'%cn
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
699 index_name = '_%s_%s_idx'%(cn, key)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
700 if self.sql_index_exists(table_name, index_name):
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
701 sql = 'drop index '+index_name
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
702 self.sql(sql)
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
703
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
704 # and now the retired unique index too
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
705 index_name = '_%s_key_retired_idx'%cn
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
706 if self.sql_index_exists(table_name, index_name):
3967
b1e81ad3fa6a don't recompute the index name
Richard Jones <richard@users.sourceforge.net>
parents: 3963
diff changeset
707 sql = 'drop index '+index_name
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
708 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
709
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
710 def create_journal_table(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
711 """ create the journal table for a class given the spec and
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
712 already-determined cols
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
713 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
714 # journal table
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
715 cols = ','.join(['%s varchar'%x
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
716 for x in 'nodeid date tag action params'.split()])
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
717 sql = """create table %s__journal (
3224
1fd11a9803bb use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3155
diff changeset
718 nodeid integer, date %s, tag varchar(255),
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
719 action varchar(255), params text)""" % (spec.classname,
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
720 self.hyperdb_to_sql_datatype(hyperdb.Date))
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
721 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
722 self.create_journal_table_indexes(spec)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
723
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
724 def create_journal_table_indexes(self, spec):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
725 # index on nodeid
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
726 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%(
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
727 spec.classname, spec.classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
728 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
729
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
730 def drop_journal_table_indexes(self, classname):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
731 index_name = '%s_journ_idx'%classname
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
732 if not self.sql_index_exists('%s__journal'%classname, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
733 return
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
734 index_sql = 'drop index '+index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
735 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
736
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
737 def create_multilink_table(self, spec, ml):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
738 """ Create a multilink table for the "ml" property of the class
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
739 given by the spec
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
740 """
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
741 # create the table
2828
3a2f138fcb28 use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2755
diff changeset
742 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%(
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
743 spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
744 self.sql(sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
745 self.create_multilink_table_indexes(spec, ml)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
746
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
747 def create_multilink_table_indexes(self, spec, ml):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
748 # create index on linkid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
749 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
750 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
751 self.sql(index_sql)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
752
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
753 # create index on nodeid
1836
94e430ad4fdb make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1800
diff changeset
754 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%(
2100
62ed6505cbec MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
755 spec.classname, ml, spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
756 self.sql(index_sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
757
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
758 def drop_multilink_table_indexes(self, classname, ml):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
759 l = [
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
760 '%s_%s_l_idx'%(classname, ml),
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
761 '%s_%s_n_idx'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
762 ]
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
763 table_name = '%s_%s'%(classname, ml)
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
764 for index_name in l:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
765 if not self.sql_index_exists(table_name, index_name):
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
766 continue
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
767 index_sql = 'drop index %s'%index_name
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
768 self.sql(index_sql)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
769
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
770 def create_class(self, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
771 """ Create a database table according to the given spec.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
772 """
4474
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
773
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
774 if not self.config.RDBMS_ALLOW_CREATE:
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
775 raise DatabaseError(_('CREATE operation disallowed: "%s".'%spec.classname))
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
776
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
777 cols, mls = self.create_class_table(spec)
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
778 self.create_journal_table(spec)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
779
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
780 # now create the multilink tables
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
781 for ml in mls:
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
782 self.create_multilink_table(spec, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
783
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
784 def drop_class(self, cn, spec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
785 """ Drop the given table from the database.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
786
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
787 Drop the journal and multilink tables too.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
788 """
4474
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
789
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
790 if not self.config.RDBMS_ALLOW_DROP:
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
791 raise DatabaseError(_('DROP operation disallowed: "%s".'%cn))
9b4cf6c96ee2 Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents: 4473
diff changeset
792
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
793 properties = spec[1]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
794 # figure the multilinks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
795 mls = []
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3906
diff changeset
796 for propname, prop in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
797 if isinstance(prop, Multilink):
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
798 mls.append(propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
799
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
800 # drop class table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
801 self.drop_class_table_indexes(cn, spec[0])
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
802
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
803 self.drop_class_table(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
804
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
805 # drop journal table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
806 self.drop_journal_table_indexes(cn)
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
807 sql = 'drop table %s__journal'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
808 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
809
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
810 for ml in mls:
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
811 # drop multilink table and indexes
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
812 self.drop_multilink_table_indexes(cn, ml)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
813 sql = 'drop table %s_%s'%(spec.classname, ml)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
814 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
815
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
816 def drop_class_table(self, cn):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
817 sql = 'drop table _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
818 self.sql(sql)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
819
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
820 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
821 # Classes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
822 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
823 def __getattr__(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
824 """ A convenient way of calling self.getclass(classname).
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
825 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
826 if classname in self.classes:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
827 return self.classes[classname]
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
828 raise AttributeError(classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
829
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
830 def addclass(self, cl):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
831 """ Add a Class to the hyperdatabase.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
832 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
833 cn = cl.classname
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
834 if cn in self.classes:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
835 raise ValueError(cn)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
836 self.classes[cn] = cl
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
837
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
838 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
839 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
840 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
841 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
842 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
843 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
844 description="User is allowed to access "+cn)
4517
f8e85cf5f0fe how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents: 4490
diff changeset
845 self.security.addPermission(name="Retire", klass=cn,
f8e85cf5f0fe how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents: 4490
diff changeset
846 description="User is allowed to retire "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
847
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
848 def getclasses(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
849 """ Return a list of the names of all existing classes.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
850 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
851 return sorted(self.classes)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
852
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
853 def getclass(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
854 """Get the Class object representing a particular class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
855
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
856 If 'classname' is not a valid class name, a KeyError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
857 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
858 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
859 return self.classes[classname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
860 except KeyError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
861 raise KeyError('There is no class called "%s"'%classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
862
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
863 def clear(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
864 """Delete all database contents.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
865
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
866 Note: I don't commit here, which is different behaviour to the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
867 "nuke from orbit" behaviour in the dbs.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
868 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
869 logging.getLogger('roundup.hyperdb.backend').info('clear')
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
870 for cn in self.classes:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
871 sql = 'delete from _%s'%cn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
872 self.sql(sql)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
873
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
874 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
875 # Nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
876 #
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
877
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
878 hyperdb_to_sql_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
879 hyperdb.String : str,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
880 # fractional seconds by default
3612
afda59d5d546 prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
881 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'),
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
882 hyperdb.Link : int,
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
883 hyperdb.Interval : str,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
884 hyperdb.Password : str,
2166
cd42c3c7173a MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents: 2102
diff changeset
885 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE',
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
886 hyperdb.Number : lambda x: x,
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
887 hyperdb.Integer : lambda x: x,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
888 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
889 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
890
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
891 def to_sql_value(self, propklass):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
892
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
893 fn = self.hyperdb_to_sql_value.get(propklass)
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
894 if fn:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
895 return fn
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
896
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
897 for k, v in self.hyperdb_to_sql_value.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
898 if issubclass(propklass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
899 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
900
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
901 raise ValueError('%r is not a hyperdb property class' % propklass)
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
902
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
903 def _cache_del(self, key):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
904 del self.cache[key]
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
905 self.cache_lru.remove(key)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
906
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
907 def _cache_refresh(self, key):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
908 self.cache_lru.remove(key)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
909 self.cache_lru.insert(0, key)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
910
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
911 def _cache_save(self, key, node):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
912 self.cache[key] = node
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
913 # update the LRU
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
914 self.cache_lru.insert(0, key)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
915 if len(self.cache_lru) > self.cache_size:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
916 del self.cache[self.cache_lru.pop()]
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
917
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
918 def addnode(self, classname, nodeid, node):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
919 """ Add the specified node to its class's db.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
920 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
921 self.log_debug('addnode %s%s %r'%(classname,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
922 nodeid, node))
1528
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
923
96cd422532ef bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
924 # determine the column definitions and multilink tables
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
925 cl = self.classes[classname]
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
926 cols, mls = self.determine_columns(list(cl.properties.iteritems()))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
927
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
928 # we'll be supplied these props if we're doing an import
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
929 values = node.copy()
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
930 if 'creator' not in values:
1189
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
931 # add in the "calculated" properties (dupe so we don't affect
23b8d1e87fe3 import fixes
Richard Jones <richard@users.sourceforge.net>
parents: 1187
diff changeset
932 # calling code's node assumptions)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
933 values['creation'] = values['activity'] = date.Date()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
934 values['actor'] = values['creator'] = self.getuid()
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
935
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
936 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
937 props = cl.getprops(protected=1)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
938 del props['id']
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
939
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
940 # default the non-multilink columns
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
941 for col, prop in props.iteritems():
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
942 if col not in values:
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
943 if isinstance(prop, Multilink):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
944 values[col] = []
1496
e6ac4e074acb relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents: 1492
diff changeset
945 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
946 values[col] = None
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
947
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
948 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
949 key = (classname, nodeid)
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
950 if key in self.cache:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
951 self._cache_del(key)
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
952
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
953 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
954 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
955 for col,dt in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
956 # this is somewhat dodgy....
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
957 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
958 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
959 value = values[col[2:-6]]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
960 # this is an Interval special "int" column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
961 if value is not None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
962 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
963 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
964 vals.append(value)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
965 continue
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
966
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
967 prop = props[col[1:]]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
968 value = values[col[1:]]
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
969 if value is not None:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
970 value = self.to_sql_value(prop.__class__)(value)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
971 vals.append(value)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
972 vals.append(nodeid)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
973 vals = tuple(vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
974
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
975 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
976 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
977 cols = ','.join([col for col,dt in cols]) + ',id'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
978
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
979 # perform the inserts
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
980 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
981 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
982
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
983 # insert the multilink rows
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
984 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
985 t = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
986 for entry in node[col]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
987 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
988 self.arg, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
989 self.sql(sql, (entry, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
990
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
991 def setnode(self, classname, nodeid, values, multilink_changes={}):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
992 """ Change the specified node.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
993 """
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
994 self.log_debug('setnode %s%s %r'
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
995 % (classname, nodeid, values))
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
996
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
997 # clear this node out of the cache if it's in there
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
998 key = (classname, nodeid)
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
999 if key in self.cache:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1000 self._cache_del(key)
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1001
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1002 cl = self.classes[classname]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1003 props = cl.getprops()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1004
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1005 cols = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1006 mls = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1007 # add the multilinks separately
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1008 for col in values:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1009 prop = props[col]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1010 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1011 mls.append(col)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1012 elif isinstance(prop, Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1013 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1014 cols.append(col)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1015 # extra leading '_' added by code below
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1016 cols.append('_' +col + '_int__')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1017 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1018 cols.append(col)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1019 cols.sort()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1020
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1021 # figure the values to insert
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1022 vals = []
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1023 for col in cols:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1024 if col.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1025 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1026 # Intervals store the seconds value too
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1027 col = col[1:-6]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1028 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1029 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1030 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1031 vals.append(None)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1032 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1033 vals.append(value.as_seconds())
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1034 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1035 prop = props[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1036 value = values[col]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1037 if value is None:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1038 e = None
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1039 else:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1040 e = self.to_sql_value(prop.__class__)(value)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1041 vals.append(e)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1042
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1043 vals.append(int(nodeid))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1044 vals = tuple(vals)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1045
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1046 # if there's any updates to regular columns, do them
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1047 if cols:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1048 # make sure the ordering is correct for column name -> column value
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1049 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols])
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1050 cols = ','.join(cols)
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1051
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1052 # perform the update
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1053 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1054 self.sql(sql, vals)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1055
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1056 # we're probably coming from an import, not a change
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1057 if not multilink_changes:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1058 for name in mls:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1059 prop = props[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1060 value = values[name]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1061
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1062 t = '%s_%s'%(classname, name)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1063
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1064 # clear out previous values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1065 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1066 self.sql('delete from %s where nodeid=%s'%(t, self.arg),
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1067 (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1068
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1069 # insert the values for this node
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1070 for entry in values[name]:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1071 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1072 self.arg, self.arg)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1073 # XXX numeric ids
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1074 self.sql(sql, (entry, nodeid))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1075
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1076 # we have multilink changes to apply
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1077 for col, (add, remove) in multilink_changes.iteritems():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1078 tn = '%s_%s'%(classname, col)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1079 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1080 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1081 self.arg, self.arg)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1082 for addid in add:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1083 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1084 self.sql(sql, (int(nodeid), int(addid)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1085 if remove:
4039
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
1086 s = ','.join([self.arg]*len(remove))
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
1087 sql = 'delete from %s where nodeid=%s and linkid in (%s)'%(tn,
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
1088 self.arg, s)
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
1089 # XXX numeric ids
f44ec45cedb4 Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents: 4038
diff changeset
1090 self.sql(sql, [int(nodeid)] + remove)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1091
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1092 sql_to_hyperdb_value = {
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1093 hyperdb.String : str,
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
1094 hyperdb.Date : date_to_hyperdb_value,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1095 # hyperdb.Link : int, # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1096 hyperdb.Link : str,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1097 hyperdb.Interval : date.Interval,
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1098 hyperdb.Password : lambda x: password.Password(encrypted=x),
2472
f41539b3c486 fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents: 2456
diff changeset
1099 hyperdb.Boolean : _bool_cvt,
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1100 hyperdb.Number : _num_cvt,
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1101 hyperdb.Integer : int,
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1102 hyperdb.Multilink : lambda x: x, # used in journal marshalling
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1103 }
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1104
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1105 def to_hyperdb_value(self, propklass):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1106
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1107 fn = self.sql_to_hyperdb_value.get(propklass)
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1108 if fn:
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1109 return fn
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1110
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1111 for k, v in self.sql_to_hyperdb_value.iteritems():
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1112 if issubclass(propklass, k):
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1113 return v
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1114
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1115 raise ValueError('%r is not a hyperdb property class' % propklass)
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1116
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1117 def _materialize_multilink(self, classname, nodeid, node, propname):
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1118 """ evaluation of single Multilink (lazy eval may have skipped this)
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1119 """
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1120 if propname not in node:
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1121 sql = 'select linkid from %s_%s where nodeid=%s'%(classname,
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1122 propname, self.arg)
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1123 self.sql(sql, (nodeid,))
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1124 # extract the first column from the result
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1125 # XXX numeric ids
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1126 items = [int(x[0]) for x in self.cursor.fetchall()]
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1127 items.sort ()
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1128 node[propname] = [str(x) for x in items]
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1129
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1130 def _materialize_multilinks(self, classname, nodeid, node, props=None):
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1131 """ get all Multilinks of a node (lazy eval may have skipped this)
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1132 """
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1133 cl = self.classes[classname]
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1134 props = props or [pn for (pn, p) in cl.properties.iteritems()
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1135 if isinstance(p, Multilink)]
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1136 for propname in props:
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1137 if propname not in node:
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1138 self._materialize_multilink(classname, nodeid, node, propname)
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1139
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1140 def getnode(self, classname, nodeid, fetch_multilinks=True):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1141 """ Get a node from the database.
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1142 For optimisation optionally we don't fetch multilinks
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1143 (lazy Multilinks).
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1144 But for internal database operations we need them.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1145 """
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1146 # see if we have this node cached
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1147 key = (classname, nodeid)
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1148 if key in self.cache:
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1149 # push us back to the top of the LRU
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1150 self._cache_refresh(key)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1151 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1152 self.stats['cache_hits'] += 1
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1153 # return the cached information
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1154 if fetch_multilinks:
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1155 self._materialize_multilinks(classname, nodeid, self.cache[key])
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1156 return self.cache[key]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1157
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1158 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1159 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1160 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1161
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1162 # figure the columns we're fetching
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1163 cl = self.classes[classname]
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1164 cols, mls = self.determine_columns(list(cl.properties.iteritems()))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1165 scols = ','.join([col for col,dt in cols])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1166
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1167 # perform the basic property fetch
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1168 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1169 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1170
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1171 values = self.sql_fetchone()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1172 if values is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1173 raise IndexError('no such %s node %s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1174
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1175 # make up the node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1176 node = {}
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1177 props = cl.getprops(protected=1)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1178 for col in range(len(cols)):
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1179 name = cols[col][0][1:]
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1180 if name.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1181 # XXX eugh, this test suxxors
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1182 # ignore the special Interval-as-seconds column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
1183 continue
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1184 value = values[col]
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1185 if value is not None:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1186 value = self.to_hyperdb_value(props[name].__class__)(value)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1187 node[name] = value
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1188
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1189 if fetch_multilinks and mls:
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1190 self._materialize_multilinks(classname, nodeid, node, mls)
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1191
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1192 # save off in the cache
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1193 key = (classname, nodeid)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1194 self._cache_save(key, node)
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1195
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1196 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1197 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
1198
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1199 return node
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1200
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1201 def destroynode(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1202 """Remove a node from the database. Called exclusively by the
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1203 destroy() method on Class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1204 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1205 logging.getLogger('roundup.hyperdb.backend').info('destroynode %s%s'%(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
1206 classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1207
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1208 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1209 if not self.hasnode(classname, nodeid):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1210 raise IndexError('%s has no node %s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1211
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1212 # see if we have this node cached
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1213 if (classname, nodeid) in self.cache:
1173
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1214 del self.cache[(classname, nodeid)]
58f1a2c174ed simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents: 1172
diff changeset
1215
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1216 # see if there's any obvious commit actions that we should get rid of
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1217 for entry in self.transactions[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1218 if entry[1][:2] == (classname, nodeid):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1219 self.transactions.remove(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1220
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1221 # now do the SQL
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1222 sql = 'delete from _%s where id=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1223 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1224
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1225 # remove from multilnks
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1226 cl = self.getclass(classname)
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1227 x, mls = self.determine_columns(list(cl.properties.iteritems()))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1228 for col in mls:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1229 # get the link ids
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1230 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
1231 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1232
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1233 # remove journal entries
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1234 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1235 self.sql(sql, (nodeid,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1236
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1237 # cleanup any blob filestorage when we commit
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1238 self.transactions.append((FileStorage.destroy, (self, classname, nodeid)))
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1239
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1240 def hasnode(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1241 """ Determine if the database has a given node.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1242 """
5086
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1243 # nodeid (aka link) is sql type INTEGER. max positive value
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1244 # for INTEGER is 2^31-1 for Postgres and MySQL. The max
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1245 # positive value for SqLite is 2^63 -1, so arguably this check
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1246 # needs to adapt for the type of the RDBMS. For right now,
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1247 # choose lowest common denominator.
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1248 if int(nodeid) >= 2**31:
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1249 # value out of range return false
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1250 return 0
2b551b6b0830 issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
1251
4031
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1252 # If this node is in the cache, then we do not need to go to
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1253 # the database. (We don't consider this an LRU hit, though.)
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1254 if (classname, nodeid) in self.cache:
4031
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1255 # Return 1, not True, to match the type of the result of
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1256 # the SQL operation below.
6e2dec4f9836 Fix issue2550501
Stefan Seefeld <stefan@seefeld.name>
parents: 3993
diff changeset
1257 return 1
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1258 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1259 self.sql(sql, (nodeid,))
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1260 return int(self.cursor.fetchone()[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1261
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1262 def countnodes(self, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1263 """ Count the number of nodes that exist for a particular Class.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1264 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1265 sql = 'select count(*) from _%s'%classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1266 self.sql(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1267 return self.cursor.fetchone()[0]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1268
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1269 def addjournal(self, classname, nodeid, action, params, creator=None,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1270 creation=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1271 """ Journal the Action
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1272 'action' may be:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1273
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1274 'set' -- 'params' is a dictionary of property values
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1275 'create' -- 'params' is an empty dictionary as of
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1276 Wed Nov 06 11:38:43 2002 +0000
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1277 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1278 'retired' or 'restored' -- 'params' is None
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1279
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1280 'creator' -- the user performing the action, which defaults to
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1281 the current user.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1282 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1283 # handle supply of the special journalling parameters (usually
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1284 # supplied on importing an existing database)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1285 if creator:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1286 journaltag = creator
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1287 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1288 journaltag = self.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1289 if creation:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1290 journaldate = creation
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1291 else:
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1292 journaldate = date.Date()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1293
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1294 # create the journal entry
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1295 cols = 'nodeid,date,tag,action,params'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1296
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1297 self.log_debug('addjournal %s%s %r %s %s %r'%(classname,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1298 nodeid, journaldate, journaltag, action, params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1299
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1300 # make the journalled data marshallable
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1301 if isinstance(params, type({})):
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1302 self._journal_marshal(params, classname)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1303
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1304 params = repr(params)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1305
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1306 dc = self.to_sql_value(hyperdb.Date)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1307 journaldate = dc(journaldate)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1308
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1309 self.save_journal(classname, cols, nodeid, journaldate,
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1310 journaltag, action, params)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1311
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1312 def setjournal(self, classname, nodeid, journal):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1313 """Set the journal to the "journal" list."""
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1314 # clear out any existing entries
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1315 self.sql('delete from %s__journal where nodeid=%s'%(classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1316 self.arg), (nodeid,))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1317
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1318 # create the journal entry
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1319 cols = 'nodeid,date,tag,action,params'
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1320
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1321 dc = self.to_sql_value(hyperdb.Date)
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1322 for nodeid, journaldate, journaltag, action, params in journal:
4059
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1323 self.log_debug('addjournal %s%s %r %s %s %r'%(
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1324 classname, nodeid, journaldate, journaltag, action,
ef0b4396888a Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1325 params))
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1326
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1327 # make the journalled data marshallable
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1328 if isinstance(params, type({})):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1329 self._journal_marshal(params, classname)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1330 params = repr(params)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1331
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1332 self.save_journal(classname, cols, nodeid, dc(journaldate),
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1333 journaltag, action, params)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
1334
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1335 def _journal_marshal(self, params, classname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1336 """Convert the journal params values into safely repr'able and
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1337 eval'able values."""
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1338 properties = self.getclass(classname).getprops()
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1339 for param, value in params.iteritems():
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1340 if not value:
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1341 continue
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1342 property = properties[param]
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1343 cvt = self.to_sql_value(property.__class__)
2275
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1344 if isinstance(property, Password):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1345 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1346 elif isinstance(property, Date):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1347 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1348 elif isinstance(property, Interval):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1349 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1350 elif isinstance(property, Boolean):
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1351 params[param] = cvt(value)
3197e37346de merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2256
diff changeset
1352
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1353 def getjournal(self, classname, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1354 """ get the journal for id
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1355 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1356 # make sure the node exists
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1357 if not self.hasnode(classname, nodeid):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1358 raise IndexError('%s has no node %s'%(classname, nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1359
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1360 cols = ','.join('nodeid date tag action params'.split())
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1361 journal = self.load_journal(classname, cols, nodeid)
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1362
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1363 # now unmarshal the data
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1364 dc = self.to_hyperdb_value(hyperdb.Date)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1365 res = []
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1366 properties = self.getclass(classname).getprops()
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1367 for nodeid, date_stamp, user, action, params in journal:
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1368 params = eval(params)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1369 if isinstance(params, type({})):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1370 for param, value in params.iteritems():
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1371 if not value:
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1372 continue
2731
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1373 property = properties.get(param, None)
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1374 if property is None:
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1375 # deleted property
7c472ed6babf handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents: 2727
diff changeset
1376 continue
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1377 cvt = self.to_hyperdb_value(property.__class__)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1378 if isinstance(property, Password):
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1379 params[param] = password.JournalPassword(value)
2248
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1380 elif isinstance(property, Date):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1381 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1382 elif isinstance(property, Interval):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1383 params[param] = cvt(value)
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1384 elif isinstance(property, Boolean):
cd7e6d6288c6 fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents: 2244
diff changeset
1385 params[param] = cvt(value)
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1386 # XXX numeric ids
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1387 res.append((str(nodeid), dc(date_stamp), user, action, params))
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1388 return res
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1389
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1390 def save_journal(self, classname, cols, nodeid, journaldate,
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1391 journaltag, action, params):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1392 """ Save the journal entry to the database
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1393 """
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1394 entry = (nodeid, journaldate, journaltag, action, params)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1395
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1396 # do the insert
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1397 a = self.arg
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1398 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1399 classname, cols, a, a, a, a, a)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1400 self.sql(sql, entry)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1401
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
1402 def load_journal(self, classname, cols, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1403 """ Load the journal from the database
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1404 """
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1405 # now get the journal entries
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1406 sql = 'select %s from %s__journal where nodeid=%s order by date'%(
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1407 cols, classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1408 self.sql(sql, (nodeid,))
2244
ac4f295499a4 fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents: 2240
diff changeset
1409 return self.cursor.fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1410
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1411 def pack(self, pack_before):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1412 """ Delete all journal entries except "create" before 'pack_before'.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1413 """
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
1414 date_stamp = self.to_sql_value(Date)(pack_before)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1415
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1416 # do the delete
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1417 for classname in self.classes:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1418 sql = "delete from %s__journal where date<%s and "\
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1419 "action<>'create'"%(classname, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
1420 self.sql(sql, (date_stamp,))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1421
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1422 def sql_commit(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1423 """ Actually commit to the database.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1424 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1425 logging.getLogger('roundup.hyperdb.backend').info('commit')
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3685
diff changeset
1426
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1427 self.conn.commit()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1428
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1429 # open a new cursor for subsequent work
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1430 self.cursor = self.conn.cursor()
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2362
diff changeset
1431
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1432 def commit(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1433 """ Commit the current transactions.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1434
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1435 Save all data changed since the database was opened or since the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1436 last commit() or rollback().
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1437 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1438 # commit the database
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1439 self.sql_commit()
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1440
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1441 # session and otk are committed with the db but not the other
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1442 # way round
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1443 if self.Session:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1444 self.Session.commit()
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1445 if self.Otk:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1446 self.Otk.commit()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1447
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1448 # now, do all the other transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1449 for method, args in self.transactions:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1450 method(*args)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1451
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1452 # save the indexer
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1453 self.indexer.save_index()
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
1454
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1455 # clear out the transactions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1456 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1457
4448
2784c239e6c8 clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1458 # clear the cache: Don't carry over cached values from one
2784c239e6c8 clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1459 # transaction to the next (there may be other changes from other
2784c239e6c8 clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1460 # transactions)
2784c239e6c8 clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1461 self.clearCache()
2784c239e6c8 clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1462
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1463 def sql_rollback(self):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1464 self.conn.rollback()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1465
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1466 def rollback(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1467 """ Reverse all actions from the current transaction.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1468
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1469 Undo all the changes made since the database was opened or the last
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1470 commit() or rollback() was performed.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1471 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1472 logging.getLogger('roundup.hyperdb.backend').info('rollback')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1473
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1474 self.sql_rollback()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1475
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1476 # roll back "other" transaction stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1477 for method, args in self.transactions:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1478 # delete temporary files
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1479 if method == self.doStoreFile:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1480 self.rollbackStoreFile(*args)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1481 self.transactions = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1482
1492
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1483 # clear the cache
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1484 self.clearCache()
2fc7d4a8c9e7 fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1485
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1486 def sql_close(self):
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1487 logging.getLogger('roundup.hyperdb.backend').info('close')
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1488 self.conn.close()
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1489
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1490 def close(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1491 """ Close off the connection.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1492 """
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
1493 self.indexer.close()
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
1494 self.sql_close()
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1495 if self.Session:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1496 self.Session.close()
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1497 self.Session = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1498 if self.Otk:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1499 self.Otk.close()
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
1500 self.Otk = None
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1501
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1502 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1503 # The base Class class
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1504 #
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1505 class Class(hyperdb.Class):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1506 """ The handle to a particular class of nodes in a hyperdatabase.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1507
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1508 All methods except __repr__ and getnode must be implemented by a
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1509 concrete backend Class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1510 """
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
1511 # For many databases the LIKE operator ignores case.
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
1512 # Postgres and Oracle have an ILIKE operator to support this.
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
1513 # We define the default here, can be changed in derivative class
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
1514 case_insensitive_like = 'LIKE'
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1515
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1516 def schema(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1517 """ A dumpable version of the schema that we can store in the
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1518 database
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1519 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1520 return (self.key, [(x, repr(y)) for x,y in self.properties.iteritems()])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1521
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1522 def enableJournalling(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1523 """Turn journalling on for this class
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1524 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1525 self.do_journal = 1
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1526
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1527 def disableJournalling(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1528 """Turn journalling off for this class
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1529 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1530 self.do_journal = 0
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1531
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1532 # Editing nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1533 def create(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1534 """ Create a new node of this class and return its id.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1535
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1536 The keyword arguments in 'propvalues' map property names to values.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1537
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1538 The values of arguments must be acceptable for the types of their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1539 corresponding properties or a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1540
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1541 If this class has a key property, it must be present and its value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1542 must not collide with other key strings or a ValueError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1543
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1544 Any other properties on this class that are missing from the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1545 'propvalues' dictionary are set to None.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1546
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1547 If an id in a link or multilink property does not refer to a valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1548 node, an IndexError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1549 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1550 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1551 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1552 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1553 return newid
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1554
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
1555 def create_inner(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1556 """ Called by create, in-between the audit and react calls.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1557 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1558 if 'id' in propvalues:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1559 raise KeyError('"id" is reserved')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1560
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1561 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1562 raise DatabaseError(_('Database open read-only'))
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1563
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1564 if ('creator' in propvalues or 'actor' in propvalues or
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1565 'creation' in propvalues or 'activity' in propvalues):
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1566 raise KeyError('"creator", "actor", "creation" and '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1567 '"activity" are reserved')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1568
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1569 # new node's id
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1570 newid = self.db.newid(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1571
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1572 # validate propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1573 num_re = re.compile('^\d+$')
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1574 for key, value in propvalues.iteritems():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1575 if key == self.key:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1576 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1577 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1578 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1579 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1580 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1581 raise ValueError('node with key "%s" exists'%value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1582
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1583 # try to handle this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1584 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1585 prop = self.properties[key]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1586 except KeyError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1587 raise KeyError('"%s" has no property "%s"'%(self.classname,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1588 key))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1589
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1590 if value is not None and isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1591 if type(value) != type(''):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1592 raise ValueError('link value must be String')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1593 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1594 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1595 if not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1596 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1597 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1598 except (TypeError, KeyError):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1599 raise IndexError('new property "%s": %s not a %s'%(
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1600 key, value, link_class))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1601 elif not self.db.getclass(link_class).hasnode(value):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1602 raise IndexError('%s has no node %s'%(link_class,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1603 value))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1604
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1605 # save off the value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1606 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1607
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1608 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1609 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1610 self.db.addjournal(link_class, value, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1611 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1612
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1613 elif isinstance(prop, Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1614 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1615 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1616 if not hasattr(value, '__iter__'):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1617 raise TypeError('new property "%s" not an iterable of ids'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1618 # clean up and validate the list of links
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1619 link_class = self.properties[key].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1620 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1621 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1622 if type(entry) != type(''):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1623 raise ValueError('"%s" multilink value (%r) '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1624 'must contain Strings'%(key, value))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1625 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1626 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1627 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1628 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1629 except (TypeError, KeyError):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1630 raise IndexError('new property "%s": %s not a %s'%(
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1631 key, entry, self.properties[key].classname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1632 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1633 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1634 propvalues[key] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1635
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1636 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1637 for nodeid in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1638 if not self.db.getclass(link_class).hasnode(nodeid):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1639 raise IndexError('%s has no node %s'%(link_class,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1640 nodeid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1641 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1642 if self.do_journal and self.properties[key].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1643 self.db.addjournal(link_class, nodeid, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1644 (self.classname, newid, key))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1645
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1646 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1647 if type(value) != type('') and type(value) != type(u''):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1648 raise TypeError('new property "%s" not a string'%key)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1649 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1650 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1651 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1652
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1653 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1654 if not isinstance(value, password.Password):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1655 raise TypeError('new property "%s" not a Password'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1656
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1657 elif isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1658 if value is not None and not isinstance(value, date.Date):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1659 raise TypeError('new property "%s" not a Date'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1660
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1661 elif isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1662 if value is not None and not isinstance(value, date.Interval):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1663 raise TypeError('new property "%s" not an Interval'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1664
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1665 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1666 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1667 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1668 except ValueError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1669 raise TypeError('new property "%s" not numeric'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1670
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1671 elif value is not None and isinstance(prop, Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1672 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1673 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1674 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1675 raise TypeError('new property "%s" not integer'%key)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1676
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1677 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1678 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1679 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1680 except ValueError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1681 raise TypeError('new property "%s" not boolean'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1682
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1683 # make sure there's data where there needs to be
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1684 for key, prop in self.properties.iteritems():
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1685 if key in propvalues:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1686 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1687 if key == self.key:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1688 raise ValueError('key property "%s" is required'%key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1689 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1690 propvalues[key] = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1691 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1692 propvalues[key] = None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1693
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1694 # done
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1695 self.db.addnode(self.classname, newid, propvalues)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1696 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1697 self.db.addjournal(self.classname, newid, ''"create", {})
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1698
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1699 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
1700 return str(newid)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1701
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1702 def get(self, nodeid, propname, default=_marker, cache=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1703 """Get the value of a property on an existing node of this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1704
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1705 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1706 IndexError is raised. 'propname' must be the name of a property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1707 of this class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1708
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1709 'cache' exists for backwards compatibility, and is not used.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1710 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1711 if propname == 'id':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1712 return nodeid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1713
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1714 # get the node's dict
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1715 d = self.db.getnode(self.classname, nodeid, fetch_multilinks=False)
4476
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1716 # handle common case -- that property is in dict -- first
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1717 # if None and one of creator/creation actor/activity return None
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1718 if propname in d:
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1719 r = d [propname]
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1720 # return copy of our list
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1721 if isinstance (r, list):
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1722 return r[:]
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1723 if r is not None:
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1724 return r
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1725 elif propname in ('creation', 'activity', 'creator', 'actor'):
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1726 return r
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1727
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1728 # propname not in d:
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1729 if propname == 'creation' or propname == 'activity':
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1730 return date.Date()
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1731 if propname == 'creator' or propname == 'actor':
143f52d48e60 - small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4474
diff changeset
1732 return self.db.getuid()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1733
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1734 # get the property (raises KeyError if invalid)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1735 prop = self.properties[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1736
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1737 # lazy evaluation of Multilink
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1738 if propname not in d and isinstance(prop, Multilink):
4490
559d9a2a0191 Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4483
diff changeset
1739 self.db._materialize_multilink(self.classname, nodeid, d, propname)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1740
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1741 # handle there being no value in the table for the property
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1742 if propname not in d or d[propname] is None:
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
1743 if default is _marker:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1744 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1745 return []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1746 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1747 return None
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1748 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1749 return default
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1750
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1751 # don't pass our list to other code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1752 if isinstance(prop, Multilink):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1753 return d[propname][:]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1754
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1755 return d[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1756
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1757 def set(self, nodeid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1758 """Modify a property on an existing node of this class.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1759
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1760 'nodeid' must be the id of an existing node of this class or an
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1761 IndexError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1762
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1763 Each key in 'propvalues' must be the name of a property of this
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1764 class or a KeyError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1765
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1766 All values in 'propvalues' must be acceptable types for their
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1767 corresponding properties or a TypeError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1768
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1769 If the value of the key property is set, it must not collide with
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1770 other key strings or a ValueError is raised.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1771
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1772 If the value of a Link or Multilink property contains an invalid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1773 node id, a ValueError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1774 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1775 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1776 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1777 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1778 self.fireReactors('set', nodeid, oldvalues)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
1779 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1780
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1781 def set_inner(self, nodeid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1782 """ Called by set, in-between the audit and react calls.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
1783 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1784 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1785 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1786
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1787 if ('creator' in propvalues or 'actor' in propvalues or
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1788 'creation' in propvalues or 'activity' in propvalues):
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1789 raise KeyError('"creator", "actor", "creation" and '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1790 '"activity" are reserved')
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1791
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1792 if 'id' in propvalues:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1793 raise KeyError('"id" is reserved')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1794
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1795 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1796 raise DatabaseError(_('Database open read-only'))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1797
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1798 node = self.db.getnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1799 if self.is_retired(nodeid):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1800 raise IndexError('Requested item is retired')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1801 num_re = re.compile('^\d+$')
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1802
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1803 # make a copy of the values dictionary - we'll modify the contents
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1804 propvalues = propvalues.copy()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1805
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1806 # if the journal value is to be different, store it in here
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1807 journalvalues = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1808
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1809 # remember the add/remove stuff for multilinks, making it easier
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1810 # for the Database layer to do its stuff
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1811 multilink_changes = {}
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1812
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1813 # omit quiet properties from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1814 quiet_props = []
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1815
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1816 for propname, value in list(propvalues.items()):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1817 # check to make sure we're not duplicating an existing key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1818 if propname == self.key and node[propname] != value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1819 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1820 self.lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1821 except KeyError:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1822 pass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1823 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1824 raise ValueError('node with key "%s" exists'%value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1825
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1826 # this will raise the KeyError if the property isn't valid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1827 # ... we don't use getprops() here because we only care about
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1828 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1829 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1830 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
1831 except KeyError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1832 raise KeyError('"%s" has no property named "%s"'%(
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1833 self.classname, propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1834
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1835 # if the value's the same as the existing value, no sense in
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1836 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1837 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1838 if value == current:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1839 del propvalues[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1840 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1295
diff changeset
1841 journalvalues[propname] = current
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1842
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1843 # do stuff based on the prop type
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1844 if isinstance(prop, Link):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1845 link_class = prop.classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1846 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1847 if value is not None and not isinstance(value, type('')):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1848 raise ValueError('property "%s" link value be a string'%(
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1849 propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1850 if isinstance(value, type('')) and not num_re.match(value):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1851 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1852 value = self.db.classes[link_class].lookup(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1853 except (TypeError, KeyError):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1854 raise IndexError('new property "%s": %s not a %s'%(
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1855 propname, value, prop.classname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1856
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1857 if (value is not None and
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1858 not self.db.getclass(link_class).hasnode(value)):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1859 raise IndexError('%s has no node %s'%(link_class,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1860 value))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1861
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1862 if self.do_journal and prop.do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1863 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1864 if node[propname] is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1865 self.db.addjournal(link_class, node[propname],
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1866 ''"unlink", (self.classname, nodeid, propname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1867
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1868 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1869 if value is not None:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
1870 self.db.addjournal(link_class, value, ''"link",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1871 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1872
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1873 elif isinstance(prop, Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1874 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1875 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3858
diff changeset
1876 if not hasattr(value, '__iter__'):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1877 raise TypeError('new property "%s" not an iterable of'
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1878 ' ids'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1879 link_class = self.properties[propname].classname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1880 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1881 for entry in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1882 # if it isn't a number, it's a key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1883 if type(entry) != type(''):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1884 raise ValueError('new property "%s" link value '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1885 'must be a string'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1886 if not num_re.match(entry):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1887 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1888 entry = self.db.classes[link_class].lookup(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1889 except (TypeError, KeyError):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1890 raise IndexError('new property "%s": %s not a %s'%(
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1891 propname, entry,
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1892 self.properties[propname].classname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1893 l.append(entry)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1894 value = l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1895 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1896
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1897 # figure the journal entry for this property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1898 add = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1899 remove = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1900
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1901 # handle removals
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1902 if propname in node:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1903 l = node[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1904 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1905 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1906 for id in l[:]:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1907 if id in value:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1908 continue
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1909 # register the unlink with the old linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1910 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1911 self.db.addjournal(link_class, id, 'unlink',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1912 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1913 l.remove(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1914 remove.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1915
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1916 # handle additions
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1917 for id in value:
4095
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1918 if id in l:
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1919 continue
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1920 # We can safely check this condition after
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1921 # checking that this is an addition to the
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1922 # multilink since the condition was checked for
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1923 # existing entries at the point they were added to
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1924 # the multilink. Since the hasnode call will
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1925 # result in a SQL query, it is more efficient to
01eb89b07c13 Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents: 4085
diff changeset
1926 # avoid the check if possible.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1927 if not self.db.getclass(link_class).hasnode(id):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1928 raise IndexError('%s has no node %s'%(link_class,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1929 id))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1930 # register the link with the newly linked node
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1931 if self.do_journal and self.properties[propname].do_journal:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1932 self.db.addjournal(link_class, id, 'link',
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1933 (self.classname, nodeid, propname))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1934 l.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1935 add.append(id)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1936
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1937 # figure the journal entry
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1938 l = []
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1939 if add:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1940 l.append(('+', add))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1941 if remove:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1942 l.append(('-', remove))
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1943 multilink_changes[propname] = (add, remove)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1944 if l:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1945 journalvalues[propname] = tuple(l)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1946
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1947 elif isinstance(prop, String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1365
diff changeset
1948 if value is not None and type(value) != type('') and type(value) != type(u''):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1949 raise TypeError('new property "%s" not a string'%propname)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1950 if prop.indexme:
3399
3a0d4e4a0f34 merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
1951 if value is None: value = ''
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1952 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
1953 value)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1954
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1955 elif isinstance(prop, Password):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1956 if not isinstance(value, password.Password):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1957 raise TypeError('new property "%s" not a Password'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1958 propvalues[propname] = value
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1959 journalvalues[propname] = \
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1960 current and password.JournalPassword(current)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1961
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1962 elif value is not None and isinstance(prop, Date):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1963 if not isinstance(value, date.Date):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1964 raise TypeError('new property "%s" not a Date'% propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1965 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1966
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1967 elif value is not None and isinstance(prop, Interval):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1968 if not isinstance(value, date.Interval):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1969 raise TypeError('new property "%s" not an '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1970 'Interval'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1971 propvalues[propname] = value
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1972
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1973 elif value is not None and isinstance(prop, Number):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1974 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1975 float(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1976 except ValueError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1977 raise TypeError('new property "%s" not numeric'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1978
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1979 elif value is not None and isinstance(prop, Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1980 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1981 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1982 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1983 raise TypeError('new property "%s" not integer'%propname)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1984
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1985 elif value is not None and isinstance(prop, Boolean):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1986 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1987 int(value)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1988 except ValueError:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
1989 raise TypeError('new property "%s" not boolean'%propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1990
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1991 # record quiet properties to omit from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1992 if prop.quiet:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1993 quiet_props.append(propname)
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
1994
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1995 # nothing to do?
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1996 if not propvalues:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1997 return propvalues
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1998
3082
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
1999 # update the activity time
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
2000 propvalues['activity'] = date.Date()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
2001 propvalues['actor'] = self.db.getuid()
29bd9f06160e actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3057
diff changeset
2002
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2003 # do the set
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
2004 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2005
3328
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2006 # remove the activity props now they're handled
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2007 del propvalues['activity']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2008 del propvalues['actor']
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2009
475c8560ef9b merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3310
diff changeset
2010 # journal the set
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2011 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
2012 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2013
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
2014 # remove quiet properties from output
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
2015 for propname in quiet_props:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
2016 if propname in propvalues:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
2017 del propvalues[propname]
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
2018
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2019 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2020
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2021 def retire(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2022 """Retire a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2023
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2024 The properties on the node remain available from the get() method,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2025 and the node's id is never reused.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2026
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2027 Retired nodes are not returned by the find(), list(), or lookup()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2028 methods, and other nodes may reuse the values of their key properties.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2029 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2030 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2031 raise DatabaseError(_('Database open read-only'))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2032
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
2033 self.fireAuditors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
2034
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2035 # use the arg for __retired__ to cope with any odd database type
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2036 # conversion (hello, sqlite)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2037 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2038 self.db.arg, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2039 self.db.sql(sql, (nodeid, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2040 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
2041 self.db.addjournal(self.classname, nodeid, ''"retired", None)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2042
1345
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
2043 self.fireReactors('retire', nodeid, None)
618aa9c37d65 fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents: 1333
diff changeset
2044
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2045 def restore(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2046 """Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2047
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2048 Make node available for all operations like it was before retirement.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2049 """
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2050 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2051 raise DatabaseError(_('Database open read-only'))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2052
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2053 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2054 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2055 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2056 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2057 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2058 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2059 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2060 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2061 raise KeyError("Key property (%s) of retired node clashes "
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2062 "with existing one (%s)" % (key, node[key]))
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
2063
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2064 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2065 # use the arg for __retired__ to cope with any odd database type
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2066 # conversion (hello, sqlite)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2067 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2068 self.db.arg, self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2069 self.db.sql(sql, (0, nodeid))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2070 if self.do_journal:
2546
1b07c9740bba mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2539
diff changeset
2071 self.db.addjournal(self.classname, nodeid, ''"restored", None)
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2072
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1515
diff changeset
2073 self.fireReactors('restore', nodeid, None)
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2074
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2075 def is_retired(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2076 """Return true if the node is rerired
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2077 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2078 sql = 'select __retired__ from _%s where id=%s'%(self.classname,
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2079 self.db.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2080 self.db.sql(sql, (nodeid,))
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2081 return int(self.db.sql_fetchone()[0]) > 0
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2082
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2083 def destroy(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2084 """Destroy a node.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2085
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2086 WARNING: this method should never be used except in extremely rare
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2087 situations where there could never be links to the node being
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2088 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2089
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2090 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2091
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2092 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1988
diff changeset
2093
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2094 WARNING: really, use retire() instead
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2095
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2096 Well, I think that's enough warnings. This method exists mostly to
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2097 support the session storage of the cgi interface.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2098
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2099 The node is completely removed from the hyperdb, including all journal
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2100 entries. It will no longer be available, and will generally break code
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2101 if there are any references to the node.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2102 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2103 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2104 raise DatabaseError(_('Database open read-only'))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2105 self.db.destroynode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2106
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2107 # Locating nodes:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2108 def hasnode(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2109 """Determine if the given nodeid actually exists
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2110 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2111 return self.db.hasnode(self.classname, nodeid)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2112
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2113 def setkey(self, propname):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2114 """Select a String property of this class to be the key property.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2115
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2116 'propname' must be the name of a String property of this class or
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2117 None, or a TypeError is raised. The values of the key property on
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2118 all existing nodes must be unique or a ValueError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2119 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2120 prop = self.getprops()[propname]
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2121 if not isinstance(prop, String):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2122 raise TypeError('key properties must be String')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2123 self.key = propname
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2124
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2125 def getkey(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2126 """Return the name of the key property for this class or None."""
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2127 return self.key
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2128
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2129 def lookup(self, keyvalue):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2130 """Locate a particular node by its key property and return its id.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2131
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2132 If this class has no key property, a TypeError is raised. If the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2133 'keyvalue' matches one of the values for the key property among
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2134 the nodes in this class, the matching node's id is returned;
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2135 otherwise a KeyError is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2136 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2137 if not self.key:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2138 raise TypeError('No key property set for class %s'%self.classname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2139
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2140 # use the arg to handle any odd database type conversion (hello,
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2141 # sqlite)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2142 sql = "select id from _%s where _%s=%s and __retired__=%s"%(
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2143 self.classname, self.key, self.db.arg, self.db.arg)
3977
732a37da3a10 Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479])
Richard Jones <richard@users.sourceforge.net>
parents: 3969
diff changeset
2144 self.db.sql(sql, (str(keyvalue), 0))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2145
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1173
diff changeset
2146 # see if there was a result that's not retired
1203
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
2147 row = self.db.sql_fetchone()
735adcbfc665 fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
2148 if not row:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2149 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key,
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2150 keyvalue, self.classname))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2151
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2152 # return the id
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2153 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2154 return str(row[0])
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2155
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2156 def find(self, **propspec):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2157 """Get the ids of nodes in this class which link to the given nodes.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2158
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2159 'propspec' consists of keyword args propname=nodeid or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2160 propname={nodeid:1, }
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2161 'propname' must be the name of a property in this class, or a
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2162 KeyError is raised. That property must be a Link or
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2163 Multilink property, or a TypeError is raised.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2164
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
2165 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
2166 the nodeids will be returned. Examples::
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
2167
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3224
diff changeset
2168 db.issue.find(messages='1')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2169 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2170 """
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2171 # shortcut
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2172 if not propspec:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2173 return []
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2174
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2175 # validate the args
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2176 props = self.getprops()
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2177 for propname, nodeids in propspec.iteritems():
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2178 # check the prop is OK
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2179 prop = props[propname]
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2180 if not isinstance(prop, Link) and not isinstance(prop, Multilink):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2181 raise TypeError("'%s' not a Link/Multilink property"%propname)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2182
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2183 # first, links
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2184 a = self.db.arg
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2185 allvalues = ()
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2186 sql = []
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2187 where = []
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2188 for prop, values in propspec.iteritems():
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2189 if not isinstance(props[prop], hyperdb.Link):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2190 continue
1912
2b0ab61db194 fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2191 if type(values) is type({}) and len(values) == 1:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2192 values = list(values)[0]
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2193 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2194 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2195 where.append('_%s = %s'%(prop, a))
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
2196 elif values is None:
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1557
diff changeset
2197 where.append('_%s is NULL'%prop)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2198 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2199 values = list(values)
2494
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2200 s = ''
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2201 if None in values:
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2202 values.remove(None)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2203 s = '_%s is NULL or '%prop
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2204 allvalues += tuple(values)
ea7fb2f416db fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents: 2472
diff changeset
2205 s += '_%s in (%s)'%(prop, ','.join([a]*len(values)))
2530
a182c536b72d fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2516
diff changeset
2206 where.append('(' + s +')')
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2207 if where:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2208 allvalues = (0, ) + allvalues
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2209 sql.append("""select id from _%s where __retired__=%s
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2210 and %s"""%(self.classname, a, ' and '.join(where)))
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2211
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2212 # now multilinks
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2213 for prop, values in propspec.iteritems():
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2214 if not isinstance(props[prop], hyperdb.Multilink):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2215 continue
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2216 if not values:
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2217 continue
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2218 allvalues += (0, )
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2219 if type(values) is type(''):
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2220 allvalues += (values,)
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2221 s = a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2222 else:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2223 allvalues += tuple(values)
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1212
diff changeset
2224 s = ','.join([a]*len(values))
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2225 tn = '%s_%s'%(self.classname, prop)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2226 sql.append("""select id from _%s, %s where __retired__=%s
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2227 and id = %s.nodeid and %s.linkid in (%s)"""%(self.classname,
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2228 tn, a, tn, tn, s))
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2229
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2230 if not sql:
1988
5660b89f8903 more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents: 1986
diff changeset
2231 return []
2733
ef396596a24e more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents: 2731
diff changeset
2232 sql = ' union '.join(sql)
1183
08a13a84ed43 Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents: 1181
diff changeset
2233 self.db.sql(sql, allvalues)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2234 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2235 l = [str(x[0]) for x in self.db.sql_fetchall()]
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2236 return l
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2237
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2238 def stringFind(self, **requirements):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2239 """Locate a particular node by matching a set of its String
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2240 properties in a caseless search.
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2241
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2242 If the property is not a String property, a TypeError is raised.
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2243
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2244 The return is a list of the id of all nodes that match.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2245 """
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2246 where = []
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2247 args = []
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2248 for propname in requirements:
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2249 prop = self.properties[propname]
1951
767ff2a03eee more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents: 1926
diff changeset
2250 if not isinstance(prop, String):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2251 raise TypeError("'%s' not a String property"%propname)
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2252 where.append(propname)
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2253 args.append(requirements[propname].lower())
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2254
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2255 # generate the where clause
1549
a53a7e197360 fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents: 1539
diff changeset
2256 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where])
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2257 sql = 'select id from _%s where %s and __retired__=%s'%(
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2258 self.classname, s, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2259 args.append(0)
1195
e0032f4ab334 added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents: 1189
diff changeset
2260 self.db.sql(sql, tuple(args))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2261 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2262 l = [str(x[0]) for x in self.db.sql_fetchall()]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2263 return l
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2264
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2265 def list(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2266 """ Return a list of the ids of the active nodes in this class.
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2267 """
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2268 return self.getnodeids(retired=0)
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2269
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2270 def getnodeids(self, retired=None):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2271 """ Retrieve all the ids of the nodes for a particular Class.
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2272
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2273 Set retired=None to get all nodes. Otherwise it'll get all the
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2274 retired or non-retired nodes, depending on the flag.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2275 """
1707
3d627e34f18e sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1599
diff changeset
2276 # flip the sense of the 'retired' flag if we don't want all of them
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1479
diff changeset
2277 if retired is not None:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2278 args = (0, )
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2279 if retired:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2280 compare = '>'
1719
eeb167fb8faf *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1707
diff changeset
2281 else:
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2282 compare = '='
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2283 sql = 'select id from _%s where __retired__%s%s'%(self.classname,
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2284 compare, self.db.arg)
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2285 else:
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2286 args = ()
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2287 sql = 'select id from _%s'%self.classname
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2288 self.db.sql(sql, args)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2289 # XXX numeric ids
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2290 ids = [str(x[0]) for x in self.db.cursor.fetchall()]
1539
800b5896e14a fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents: 1528
diff changeset
2291 return ids
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2292
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2293 def _subselect(self, classname, multilink_table):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2294 """Create a subselect. This is factored out because some
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2295 databases (hmm only one, so far) doesn't support subselects
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2296 look for "I can't believe it's not a toy RDBMS" in the mysql
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2297 backend.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2298 """
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2299 return '_%s.id not in (select nodeid from %s)'%(classname,
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2300 multilink_table)
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2301
3685
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2302 # Some DBs order NULL values last. Set this variable in the backend
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2303 # for prepending an order by clause for each attribute that causes
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2304 # correct sort order for NULLs. Examples:
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2305 # order_by_null_values = '(%s is not NULL)'
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2306 # order_by_null_values = 'notnull(%s)'
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2307 # The format parameter is replaced with the attribute.
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2308 order_by_null_values = None
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2309
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2310 def supports_subselects(self):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2311 '''Assuming DBs can do subselects, overwrite if they cannot.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2312 '''
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2313 return True
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2314
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2315 def _filter_multilink_expression_fallback(
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2316 self, classname, multilink_table, expr):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2317 '''This is a fallback for database that do not support
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2318 subselects.'''
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2319
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2320 is_valid = expr.evaluate
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2321
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2322 last_id, kws = None, []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2323
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2324 ids = IdListOptimizer()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2325 append = ids.append
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2326
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2327 # This join and the evaluation in program space
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2328 # can be expensive for larger databases!
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2329 # TODO: Find a faster way to collect the data needed
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2330 # to evalute the expression.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2331 # Moving the expression evaluation into the database
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2332 # would be nice but this tricky: Think about the cases
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2333 # where the multilink table does not have join values
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2334 # needed in evaluation.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2335
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2336 stmnt = "SELECT c.id, m.linkid FROM _%s c " \
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2337 "LEFT OUTER JOIN %s m " \
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2338 "ON c.id = m.nodeid ORDER BY c.id" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2339 classname, multilink_table)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2340 self.db.sql(stmnt)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2341
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2342 # collect all multilink items for a class item
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2343 for nid, kw in self.db.sql_fetchiter():
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2344 if nid != last_id:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2345 if last_id is None:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2346 last_id = nid
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2347 else:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2348 # we have all multilink items -> evaluate!
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2349 if is_valid(kws): append(last_id)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2350 last_id, kws = nid, []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2351 if kw is not None:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2352 kws.append(kw)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2353
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2354 if last_id is not None and is_valid(kws):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2355 append(last_id)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2356
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2357 # we have ids of the classname table
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2358 return ids.where("_%s.id" % classname, self.db.arg)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2359
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2360 def _filter_multilink_expression(self, classname, multilink_table, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2361 """ Filters out elements of the classname table that do not
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2362 match the given expression.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2363 Returns tuple of 'WHERE' introns for the overall filter.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2364 """
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2365 try:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2366 opcodes = [int(x) for x in v]
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2367 if min(opcodes) >= -1: raise ValueError()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2368
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2369 expr = compile_expression(opcodes)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2370
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2371 if not self.supports_subselects():
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2372 # We heavily rely on subselects. If there is
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2373 # no decent support fall back to slower variant.
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2374 return self._filter_multilink_expression_fallback(
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2375 classname, multilink_table, expr)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2376
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2377 atom = \
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2378 "%s IN(SELECT linkid FROM %s WHERE nodeid=a.id)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2379 self.db.arg,
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2380 multilink_table)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2381
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2382 intron = \
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2383 "_%(classname)s.id in (SELECT id " \
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2384 "FROM _%(classname)s AS a WHERE %(condition)s) " % {
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2385 'classname' : classname,
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2386 'condition' : expr.generate(lambda n: atom) }
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2387
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2388 values = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2389 def collect_values(n): values.append(n.x)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2390 expr.visit(collect_values)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2391
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2392 return intron, values
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2393 except:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2394 # original behavior
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2395 where = "%s.linkid in (%s)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2396 multilink_table, ','.join([self.db.arg] * len(v)))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2397 return where, v, True # True to indicate original
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2398
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2399 def _filter_sql (self, search_matches, filterspec, srt=[], grp=[], retr=0,
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2400 retired=False):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2401 """ Compute the proptree and the SQL/ARGS for a filter.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2402 For argument description see filter below.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2403 We return a 3-tuple, the proptree, the sql and the sql-args
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2404 or None if no SQL is necessary.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2405 The flag retr serves to retrieve *all* non-Multilink properties
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2406 (for filling the cache during a filter_iter)
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2407 """
2578
7f25486ff85e fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents: 2546
diff changeset
2408 # we can't match anything if search_matches is empty
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 4039
diff changeset
2409 if not search_matches and search_matches is not None:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2410 return None
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2411
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2412 icn = self.classname
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2413
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2414 # vars to hold the components of the SQL statement
2586
3c67f4bfa225 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2578
diff changeset
2415 frum = [] # FROM clauses
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2416 loj = [] # LEFT OUTER JOIN clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2417 where = [] # WHERE clauses
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2418 args = [] # *any* positional arguments
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2419 a = self.db.arg
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2420
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2421 # figure the WHERE clause from the filterspec
2256
0b198ed096af fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents: 2248
diff changeset
2422 mlfilt = 0 # are we joining with Multilink tables?
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2423 sortattr = self._sortattr (group = grp, sort = srt)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2424 proptree = self._proptree(filterspec, sortattr, retr)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2425 mlseen = 0
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2426 for pt in reversed(proptree.sortattr):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2427 p = pt
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2428 while p.parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2429 if isinstance (p.propclass, Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2430 mlseen = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2431 if mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2432 p.sort_ids_needed = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2433 p.tree_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2434 p = p.parent
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2435 if not mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2436 pt.attr_sort_done = pt.tree_sort_done = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2437 proptree.compute_sort_done()
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2438
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2439 cols = ['_%s.id'%icn]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2440 mlsort = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2441 rhsnum = 0
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
2442 for p in proptree:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2443 rc = ac = oc = None
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2444 cn = p.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2445 ln = p.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2446 pln = p.parent.uniqname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2447 pcn = p.parent.classname
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2448 k = p.name
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2449 v = p.val
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2450 propclass = p.propclass
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2451 if p.parent == proptree and p.name == 'id' \
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2452 and 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2453 p.sql_idx = 0
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2454 if 'sort' in p.need_for or 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2455 rc = oc = ac = '_%s._%s'%(pln, k)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2456 if isinstance(propclass, Multilink):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2457 if 'search' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2458 mlfilt = 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2459 tn = '%s_%s'%(pcn, k)
4061
52431f2ad76b Correct handling of a Multilink filterspec with an empty list of linked-to items.
Stefan Seefeld <stefan@seefeld.name>
parents: 4060
diff changeset
2460 if v in ('-1', ['-1'], []):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2461 # only match rows that have count(linkid)=0 in the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2462 # corresponding multilink table)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2463 where.append(self._subselect(pcn, tn))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2464 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2465 frum.append(tn)
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2466 gen_join = True
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2467
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2468 if p.has_values and isinstance(v, type([])):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2469 result = self._filter_multilink_expression(pln, tn, v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2470 # XXX: We dont need an id join if we used the filter
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2471 gen_join = len(result) == 3
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2472
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2473 if gen_join:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2474 where.append('_%s.id=%s.nodeid'%(pln,tn))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2475
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2476 if p.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2477 frum.append('_%s as _%s' % (cn, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2478 where.append('%s.linkid=_%s.id'%(tn, ln))
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2479
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2480 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2481 if isinstance(v, type([])):
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2482 where.append(result[0])
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4462
diff changeset
2483 args += result[1]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2484 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2485 where.append('%s.linkid=%s'%(tn, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2486 args.append(v)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2487 if 'sort' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2488 assert not p.attr_sort_done and not p.sort_ids_needed
1365
4884fb0860f9 fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents: 1351
diff changeset
2489 elif k == 'id':
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2490 if 'search' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2491 if isinstance(v, type([])):
4462
0f4a8fcb9a1d fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents: 4448
diff changeset
2492 # If there are no permitted values, then the
0f4a8fcb9a1d fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents: 4448
diff changeset
2493 # where clause will always be false, and we
0f4a8fcb9a1d fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents: 4448
diff changeset
2494 # can optimize the query away.
0f4a8fcb9a1d fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents: 4448
diff changeset
2495 if not v:
4803
5e679e364f9a Bug-fix: need to return None for empty query
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4787
diff changeset
2496 return None
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2497 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2498 where.append('_%s.%s in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2499 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2500 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2501 where.append('_%s.%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2502 args.append(v)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2503 if 'sort' in p.need_for or 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2504 rc = oc = ac = '_%s.id'%pln
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2505 elif isinstance(propclass, String):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2506 if 'search' in p.need_for:
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2507 if not isinstance(v, type([])):
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2508 v = [v]
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2509
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2510 # Quote special search characters '%' and '_' for
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2511 # correct matching with LIKE/ILIKE
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2512 # Note that we now pass the elements of v as query
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2513 # arguments and don't interpolate the quoted string
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2514 # into the sql statement. Should be safer.
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2515 v = [self.db.search_stringquote(s) for s in v]
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2516
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2517 # now add to the where clause
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2518 where.append('('
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2519 +' and '.join(["_%s._%s %s %s ESCAPE %s"%(
4787
4a017661e414 Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents: 4774
diff changeset
2520 pln,
4a017661e414 Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents: 4774
diff changeset
2521 k,
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2522 self.case_insensitive_like,
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2523 a,
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2524 a) for s in v])
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2525 +')')
4849
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2526 for vv in v:
e68920390aad Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4803
diff changeset
2527 args.extend((vv, '\\'))
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2528 if 'sort' in p.need_for:
3683
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
2529 oc = ac = 'lower(_%s._%s)'%(pln, k)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2530 elif isinstance(propclass, Link):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2531 if 'search' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2532 if p.children:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2533 if 'sort' not in p.need_for:
3694
595041d78104 Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3693
diff changeset
2534 frum.append('_%s as _%s' % (cn, ln))
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2535 where.append('_%s._%s=_%s.id'%(pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2536 if p.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2537 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2538 d = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2539 for entry in v:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2540 if entry == '-1':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2541 entry = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2542 d[entry] = entry
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2543 l = []
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2544 if None in d or not d:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2545 if None in d: del d[None]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2546 l.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2547 if d:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2548 v = list(d)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2549 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2550 l.append('(_%s._%s in (%s))'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2551 args = args + v
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2552 if l:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2553 where.append('(' + ' or '.join(l) +')')
3636
fa7becc62534 Of course for Links there is the same bug as I fixed before for multilinks.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3635
diff changeset
2554 else:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2555 if v in ('-1', None):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2556 v = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2557 where.append('_%s._%s is NULL'%(pln, k))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2558 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2559 where.append('_%s._%s=%s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2560 args.append(v)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2561 if 'sort' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2562 lp = p.cls.labelprop()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2563 oc = ac = '_%s._%s'%(pln, k)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2564 if lp != 'id':
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2565 if p.tree_sort_done:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2566 loj.append(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2567 'LEFT OUTER JOIN _%s as _%s on _%s._%s=_%s.id'%(
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2568 cn, ln, pln, k, ln))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2569 oc = '_%s._%s'%(ln, lp)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2570 if 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2571 rc = '_%s._%s'%(pln, k)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2572 elif isinstance(propclass, Date) and 'search' in p.need_for:
4060
2a68d7494bbc Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents: 4059
diff changeset
2573 dc = self.db.to_sql_value(hyperdb.Date)
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2574 if isinstance(v, type([])):
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2575 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2576 where.append('_%s._%s in (%s)'%(pln, k, s))
3633
a292054b4393 The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3612
diff changeset
2577 args = args + [dc(date.Date(x)) for x in v]
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2578 else:
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2579 try:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2580 # Try to filter on range of dates
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3554
diff changeset
2581 date_rng = propclass.range_from_raw(v, self.db)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2582 if date_rng.from_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2583 where.append('_%s._%s >= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2584 args.append(dc(date_rng.from_value))
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2585 if date_rng.to_value:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2586 where.append('_%s._%s <= %s'%(pln, k, a))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2587 args.append(dc(date_rng.to_value))
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2588 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
2589 # If range creation fails - ignore that search parameter
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2590 pass
1351
d1bfb479e527 fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents: 1345
diff changeset
2591 elif isinstance(propclass, Interval):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2592 # filter/sort using the __<prop>_int__ column
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2593 if 'search' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2594 if isinstance(v, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2595 s = ','.join([a for x in v])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2596 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2597 args = args + [date.Interval(x).as_seconds() for x in v]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2598 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2599 try:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2600 # Try to filter on range of intervals
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2601 date_rng = Range(v, date.Interval)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2602 if date_rng.from_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2603 where.append('_%s.__%s_int__ >= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2604 args.append(date_rng.from_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2605 if date_rng.to_value:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2606 where.append('_%s.__%s_int__ <= %s'%(pln, k, a))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2607 args.append(date_rng.to_value.as_seconds())
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2608 except ValueError:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2609 # If range creation fails - ignore search parameter
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2610 pass
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2611 if 'sort' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2612 oc = ac = '_%s.__%s_int__'%(pln,k)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2613 if 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2614 rc = '_%s._%s'%(pln,k)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2615 elif isinstance(propclass, Boolean) and 'search' in p.need_for:
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2616 if type(v) == type(""):
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2617 v = v.split(',')
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2618 if type(v) != type([]):
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2619 v = [v]
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2620 bv = []
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2621 for val in v:
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2622 if type(val) is type(''):
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2623 bv.append(propclass.from_raw (val))
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2624 else:
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2625 bv.append(bool(val))
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2626 if len(bv) == 1:
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2627 where.append('_%s._%s=%s'%(pln, k, a))
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2628 args = args + bv
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2629 else:
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2630 s = ','.join([a for x in v])
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2631 where.append('_%s._%s in (%s)'%(pln, k, s))
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4359
diff changeset
2632 args = args + bv
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2633 elif 'search' in p.need_for:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2634 if isinstance(v, type([])):
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2635 s = ','.join([a for x in v])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2636 where.append('_%s._%s in (%s)'%(pln, k, s))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2637 args = args + v
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2638 else:
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2639 where.append('_%s._%s=%s'%(pln, k, a))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2640 args.append(v)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2641 if oc:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2642 if p.sort_ids_needed:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2643 if rc == ac:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2644 p.sql_idx = len(cols)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2645 p.auxcol = len(cols)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2646 cols.append(ac)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2647 if p.tree_sort_done and p.sort_direction:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2648 # Don't select top-level id or multilink twice
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2649 if (not p.sort_ids_needed or ac != oc) and (p.name != 'id'
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2650 or p.parent != proptree):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2651 if rc == oc:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2652 p.sql_idx = len(cols)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2653 cols.append(oc)
3685
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2654 desc = ['', ' desc'][p.sort_direction == '-']
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2655 # Some SQL dbs sort NULL values last -- we want them first.
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2656 if (self.order_by_null_values and p.name != 'id'):
4d9adb8bc3b1 Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3683
diff changeset
2657 nv = self.order_by_null_values % oc
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2658 cols.append(nv)
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2659 p.orderby.append(nv + desc)
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2660 p.orderby.append(oc + desc)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2661 if 'retrieve' in p.need_for and p.sql_idx is None:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2662 assert(rc)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2663 p.sql_idx = len(cols)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2664 cols.append (rc)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2665
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2666 props = self.getprops()
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2667
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2668 # don't match retired nodes
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2669 if retired is not None:
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2670 op = '='
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2671 if retired:
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2672 op = '!='
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2673 where.append('_%s.__retired__%s0'%(icn, op))
1740
5ca448ff8052 don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents: 1719
diff changeset
2674
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2675 # add results of full text search
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2676 if search_matches is not None:
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 4039
diff changeset
2677 s = ','.join([a for x in search_matches])
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2678 where.append('_%s.id in (%s)'%(icn, s))
4113
f4641359b892 Fix issue2550505
Stefan Seefeld <stefan@seefeld.name>
parents: 4095
diff changeset
2679 args = args + [x for x in search_matches]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2680
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2681 # construct the SQL
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3633
diff changeset
2682 frum.append('_'+icn)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2683 frum = ','.join(frum)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2684 if where:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2685 where = ' where ' + (' and '.join(where))
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2686 else:
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2687 where = ''
2240
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2688 if mlfilt:
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2689 # we're joining tables on the id, so we will get dupes if we
ad4b717e12b9 Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
2690 # don't distinct()
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2691 cols[0] = 'distinct(_%s.id)'%icn
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2692
3693
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2693 order = []
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2694 # keep correct sequence of order attributes.
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2695 for sa in proptree.sortattr:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2696 if not sa.attr_sort_done:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2697 continue
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2698 order.extend(sa.orderby)
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2699 if order:
a775afeeee8b Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3688
diff changeset
2700 order = ' order by %s'%(','.join(order))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2701 else:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2702 order = ''
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2703
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2704 cols = ','.join(cols)
2379
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2705 loj = ' '.join(loj)
a2025bdd1491 fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
2706 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2707 args = tuple(args)
2319
7cf7e3bd1b31 more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2318
diff changeset
2708 __traceback_info__ = (sql, args)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2709 return proptree, sql, args
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2710
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2711 def filter(self, search_matches, filterspec, sort=[], group=[],
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2712 retired=False):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2713 """Return a list of the ids of the active nodes in this class that
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2714 match the 'filter' spec, sorted by the group spec and then the
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2715 sort spec
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2716
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2717 "filterspec" is {propname: value(s)}
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2718
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2719 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-'
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2720 or None and prop is a prop name or None. Note that for
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2721 backward-compatibility reasons a single (dir, prop) tuple is
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2722 also allowed.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2723
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2724 "search_matches" is a container type or None
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2725
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2726 The filter must match all properties specificed. If the property
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2727 value to match is a list:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2728
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2729 1. String properties must match all elements in the list, and
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2730 2. Other properties must match any of the elements in the list.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2731 """
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2732 if __debug__:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2733 start_t = time.time()
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2734
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2735 sq = self._filter_sql (search_matches, filterspec, sort, group,
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2736 retired=retired)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2737 # nothing to match?
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2738 if sq is None:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2739 return []
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2740 proptree, sql, args = sq
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2741
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
2742 self.db.sql(sql, args)
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
2743 l = self.db.sql_fetchall()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2744
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2745 # Compute values needed for sorting in proptree.sort
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2746 for p in proptree:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2747 if hasattr(p, 'auxcol'):
4886
a116de39e38c Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4849
diff changeset
2748 p.sort_ids = [row[p.auxcol] for row in l]
a116de39e38c Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4849
diff changeset
2749 p.sort_result = p._sort_repr (p.propclass.sort_repr, p.sort_ids)
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1168
diff changeset
2750 # return the IDs (the first column)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
2751 # XXX numeric ids
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2752 l = [str(row[0]) for row in l]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2753 l = proptree.sort (l)
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2754
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2755 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2234
diff changeset
2756 self.db.stats['filtering'] += (time.time() - start_t)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3636
diff changeset
2757 return l
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
2758
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2759 def filter_iter(self, search_matches, filterspec, sort=[], group=[],
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2760 retired=False):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2761 """Iterator similar to filter above with same args.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2762 Limitation: We don't sort on multilinks.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2763 This uses an optimisation: We put all nodes that are in the
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2764 current row into the node cache. Then we return the node id.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2765 That way a fetch of a node won't create another sql-fetch (with
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2766 a join) from the database because the nodes are already in the
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2767 cache. We're using our own temporary cursor.
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2768 """
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2769 sq = self._filter_sql(search_matches, filterspec, sort, group, retr=1,
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
2770 retired=retired)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2771 # nothing to match?
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2772 if sq is None:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2773 return
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2774 proptree, sql, args = sq
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2775 cursor = self.db.conn.cursor()
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2776 self.db.sql(sql, args, cursor)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2777 classes = {}
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2778 for p in proptree:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2779 if 'retrieve' in p.need_for:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2780 cn = p.parent.classname
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2781 ptid = p.parent.id # not the nodeid!
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2782 key = (cn, ptid)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2783 if key not in classes:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2784 classes[key] = {}
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2785 name = p.name
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2786 assert (name)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2787 classes[key][name] = p
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
2788 p.to_hyperdb = self.db.to_hyperdb_value(p.propclass.__class__)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2789 while True:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2790 row = cursor.fetchone()
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2791 if not row: break
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2792 # populate cache with current items
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2793 for (classname, ptid), pt in classes.iteritems():
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2794 nodeid = str(row[pt['id'].sql_idx])
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2795 key = (classname, nodeid)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2796 if key in self.db.cache:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2797 self.db._cache_refresh(key)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2798 continue
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2799 node = {}
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2800 for propname, p in pt.iteritems():
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2801 value = row[p.sql_idx]
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2802 if value is not None:
4473
fccf7e09af0c - optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
2803 value = p.to_hyperdb(value)
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2804 node[propname] = value
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2805 self.db._cache_save(key, node)
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2806 yield str(row[0])
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2807
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2808 def filter_sql(self, sql):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2809 """Return a list of the ids of the items in this class that match
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2810 the SQL provided. The SQL is a complete "select" statement.
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2811
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2812 The SQL select must include the item id as the first column.
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2813
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2814 This function DOES NOT filter out retired items, add on a where
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2815 clause "__retired__=0" if you don't want retired nodes.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2816 """
3688
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2817 if __debug__:
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2818 start_t = time.time()
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2819
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2820 self.db.sql(sql)
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2821 l = self.db.sql_fetchall()
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2822
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2823 if __debug__:
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2824 self.db.stats['filtering'] += (time.time() - start_t)
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2825 return l
722ab52d47fc added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents: 3687
diff changeset
2826
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2827 def count(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2828 """Get the number of nodes in this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2829
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2830 If the returned integer is 'numnodes', the ids of all the nodes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2831 in this class run from 1 to numnodes, and numnodes+1 will be the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2832 id of the next node to be created in this class.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2833 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2834 return self.db.countnodes(self.classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2835
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2836 # Manipulating properties:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2837 def getprops(self, protected=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2838 """Return a dictionary mapping property names to property objects.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2839 If the "protected" flag is true, we include protected properties -
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2840 those which may not be modified.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2841 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2842 d = self.properties.copy()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2843 if protected:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2844 d['id'] = String()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2845 d['creation'] = hyperdb.Date()
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2846 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1175
diff changeset
2847 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
2848 d['actor'] = hyperdb.Link('user')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2849 return d
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2850
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2851 def addprop(self, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2852 """Add properties to this class.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2853
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2854 The keyword arguments in 'properties' must map names to property
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2855 objects, or a TypeError is raised. None of the keys in 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2856 may collide with the names of existing properties, or a ValueError
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2857 is raised before any properties have been added.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2858 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2859 for key in properties:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2860 if key in self.properties:
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2861 raise ValueError(key)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2862 self.properties.update(properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2863
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2864 def index(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2865 """Add (or refresh) the node to search indexes
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2866 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2867 # find all the String properties that have indexme
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2868 for prop, propclass in self.getprops().iteritems():
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2869 if isinstance(propclass, String) and propclass.indexme:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2870 self.db.indexer.add_text((self.classname, nodeid, prop),
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2871 str(self.get(nodeid, prop)))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2872
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2873 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2874 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2875 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2876 def export_list(self, propnames, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2877 """ Export a node - generate a list of CSV-able data in the order
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2878 specified by propnames for the given node.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2879 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2880 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2881 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2882 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2883 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2884 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2885 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2886 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2887 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2888 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2889 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2890 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2891 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2892 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2893 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2894 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2895 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2896 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2897
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2898 def import_list(self, propnames, proplist):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2899 """ Import a node - all information including "id" is present and
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2900 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2901 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2902 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2903
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2904 Return the nodeid of the node imported.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2905 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2906 if self.db.journaltag is None:
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2907 raise DatabaseError(_('Database open read-only'))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2908 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2909
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2910 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2911 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2912 retire = 0
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2913 if not "id" in propnames:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2914 newid = self.db.newid(self.classname)
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2915 else:
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2916 newid = eval(proplist[propnames.index("id")])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2917 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2918 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2919 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2920
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2921 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2922 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2923
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2924 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2925 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2926 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2927 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2928 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2929 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2930 retire = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2931 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2932 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2933 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2934 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2935
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2936 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2937 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2938 # don't set Nones
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2939 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2940 elif isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2941 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2942 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2943 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2944 elif isinstance(prop, hyperdb.Password):
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4476
diff changeset
2945 value = password.Password(encrypted=value)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2946 elif isinstance(prop, String):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2947 if isinstance(value, unicode):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2948 value = value.encode('utf8')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2949 if not isinstance(value, str):
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2950 raise TypeError('new property "%(propname)s" not a '
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2951 'string: %(value)r'%locals())
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2952 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2953 self.db.indexer.add_text((self.classname, newid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2887
diff changeset
2954 value)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3525
diff changeset
2955 d[propname] = value
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2956
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2957 # get a new id if necessary
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2958 if newid is None:
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2959 newid = self.db.newid(self.classname)
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2960
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2961 # insert new node or update existing?
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2962 if not self.hasnode(newid):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2196
diff changeset
2963 self.db.addnode(self.classname, newid, d) # insert
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2964 else:
2516
125311efe783 fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2514
diff changeset
2965 self.db.setnode(self.classname, newid, d) # update
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2966
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2967 # retire?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2968 if retire:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2969 # use the arg for __retired__ to cope with any odd database type
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2970 # conversion (hello, sqlite)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2971 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2972 self.db.arg, self.db.arg)
3963
3230f9c88086 Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents: 3933
diff changeset
2973 self.db.sql(sql, (newid, newid))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2974 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2975
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2976 def export_journals(self):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2977 """Export a class's journal - generate a list of lists of
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2978 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2979
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2980 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2981
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2982 No heading here - the columns are fixed.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
2983 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2984 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2985 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2986 for nodeid in self.getnodeids():
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
2987 for nodeid, date, user, action, params in self.history(nodeid,
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
2988 enforceperm=False, skipquiet=False):
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2989 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2990 if action == 'set':
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2991 export_data = {}
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2992 for propname, value in params.iteritems():
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
2993 if propname not in properties:
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2994 # property no longer in the schema
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2995 continue
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
2996
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2997 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2998 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
2999 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3000 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3001 elif isinstance(prop, Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3002 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3003 elif isinstance(prop, Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3004 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3005 elif isinstance(prop, Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3006 value = str(value)
3365
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
3007 export_data[propname] = value
e2d65f6c8d83 handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents: 3328
diff changeset
3008 params = export_data
3933
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
3009 elif action == 'create' and params:
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
3010 # old tracker with data stored in the create!
9095a4da67f9 Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
3011 params = {}
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3012 l = [nodeid, date, user, action, params]
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3013 r.append(list(map(repr, l)))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3014 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2166
diff changeset
3015
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2586
diff changeset
3016 class FileClass(hyperdb.FileClass, Class):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3017 """This class defines a large chunk of data. To support this, it has a
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3018 mandatory String property "content" which is typically saved off
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3019 externally to the hyperdb.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3020
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3021 The default MIME type of this data is defined by the
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3022 "default_mime_type" class attribute, which may be overridden by each
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3023 node if the class defines a "type" String property.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3024 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3025 def __init__(self, db, classname, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3026 """The newly-created class automatically includes the "content"
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3027 and "type" properties.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3028 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3029 if 'content' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3030 properties['content'] = hyperdb.String(indexme='yes')
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3031 if 'type' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3032 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3033 Class.__init__(self, db, classname, **properties)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3034
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3035 def create(self, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3036 """ snaffle the file propvalue and store in a file
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3037 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3038 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3039 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3040 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3041
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3042 # now remove the content property so it's not stored in the db
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3043 content = propvalues['content']
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3044 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3045
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3046 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3047 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3048
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3049 # figure the mime type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3050 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3051
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3052 # and index!
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3053 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3054 self.db.indexer.add_text((self.classname, newid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3055 content, mime_type)
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3056
3979
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
3057 # store off the content as a file
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
3058 self.db.storefile(self.classname, newid, None, content)
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3977
diff changeset
3059
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3060 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3061 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1417
diff changeset
3062
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3063 return newid
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3064
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3065 def get(self, nodeid, propname, default=_marker, cache=1):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3066 """ Trap the content propname and get it from the file
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
3067
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
3068 'cache' exists for backwards compatibility, and is not used.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3069 """
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3070 poss_msg = 'Possibly a access right configuration problem.'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3071 if propname == 'content':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3072 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3073 return self.db.getfile(self.classname, nodeid, None)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5236
diff changeset
3074 except IOError as strerror:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3075 # BUG: by catching this we donot see an error in the log.
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3076 return 'ERROR reading file: %s%s\n%s\n%s'%(
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3077 self.classname, nodeid, poss_msg, strerror)
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
3078 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
3079 return Class.get(self, nodeid, propname, default)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3080 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
3081 return Class.get(self, nodeid, propname)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3082
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3083 def set(self, itemid, **propvalues):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3084 """ Snarf the "content" propvalue and update it in a file
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3085 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3086 self.fireAuditors('set', itemid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3087 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3088
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3089 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3090 content = None
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3091 if 'content' in propvalues:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3092 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3093 del propvalues['content']
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3094
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3095 # do the database create
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3096 propvalues = self.set_inner(itemid, **propvalues)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3097
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3098 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3099 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3100 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3101 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3102 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3103 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3104 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3105 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
3106 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2634
diff changeset
3107
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3108 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3109 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
3110 return propvalues
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3111
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
3112 def index(self, nodeid):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3113 """ Add (or refresh) the node to search indexes.
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3114
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3115 Use the content-type property for the content property.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3116 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3117 # find all the String properties that have indexme
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3118 for prop, propclass in self.getprops().iteritems():
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3119 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3120 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3121 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3122 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3123 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3124 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3125 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3126 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3127 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3128 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3129 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
3130 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
3131
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3132 # XXX deviation from spec - was called ItemClass
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3133 class IssueClass(Class, roundupdb.IssueClass):
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3134 # Overridden methods:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3135 def __init__(self, db, classname, **properties):
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3136 """The newly-created class automatically includes the "messages",
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3137 "files", "nosy", and "superseder" properties. If the 'properties'
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3138 dictionary attempts to specify any of these properties or a
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
3139 "creation", "creator", "activity" or "actor" property, a ValueError
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
3140 is raised.
3834
8068eb6c704e Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents: 3821
diff changeset
3141 """
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3142 if 'title' not in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3143 properties['title'] = hyperdb.String(indexme='yes')
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3144 if 'messages' not in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3145 properties['messages'] = hyperdb.Multilink("msg")
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3146 if 'files' not in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3147 properties['files'] = hyperdb.Multilink("file")
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3148 if 'nosy' not in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3149 # note: journalling is turned off as it really just wastes
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3150 # space. this behaviour may be overridden in an instance
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3151 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
4359
b9abbdd15259 another module modernised
Richard Jones <richard@users.sourceforge.net>
parents: 4113
diff changeset
3152 if 'superseder' not in properties:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3153 properties['superseder'] = hyperdb.Multilink(classname)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3154 Class.__init__(self, db, classname, **properties)
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3155
2692
f1c9873496f0 fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2691
diff changeset
3156 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/