Mercurial > p > roundup > code
annotate roundup/backends/rdbms_common.py @ 5112:8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
quiet=True/False option for all property types. When quiet=True
changes to the property will not be displayed in the::
confirmation banner (shown in green) when a change is made
property change section of change note (nosy emails)
web history display for an item.
Note that this may confuse users if used on a property that is
meant to be changed by a user. It is most useful on administrative
properties that are changed by an auditor as part of a user
generated change. Original patch by Daniel Diniz (ajaksu2)
discussed also at:
http://psf.upfronthosting.co.za/roundup/meta/issue249
Support for setting quiet when calling the class specifiers:
E.G. prop=String(quiet=True) rather than::
prop=String()
prop.quiet=True
support for anydb backend, added tests, doc updates, support for
ignoring quiet setting using showall=True in call to history()
function in templates by John Rouillard.
In addition to documenting quiet, I also documented required and
default_value additions to the hyperdb property classes. Only place I
could find is design.txt.
Note tests for history in web interface are not done. It was manually
checked but there are no automated tests. The template for setup is in
db_test_base.py::testQuietJournal but it has no asserts. I need
access to template.py::_HTMLItem::history() and I don't know how to do
that. test_templates.py isn't helping me any at all and I want to get
this patch in because it handles nicely an issue I have in the design
of my own tracker. The issue is:
The properties of an issue are displayed in framesets/subframes. The
user can roll up the frameset leaving only the title bar. When the
user saves the changes, the current state of the framesets
(collapsed/uncollapsed) is saved to a property in the user's
object. However there is no reason the user should see that this is
updated since it's an administrative detail.
Similarly, you could count the number of times an issue is reopened or
reassigned. Updates to properties that are an indirect result of a
user's change should not be displayed to the user as they can be
confusing and distracting.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 30 Jun 2016 20:38:23 -0400 |
| parents | e74c3611b138 |
| children | e1e40674a0bc |
| rev | line source |
|---|---|
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
1 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
6 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
11 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
17 # |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
18 """ Relational database (SQL) backend common code. |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
19 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
20 Basics: |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
21 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
22 - map roundup classes to relational tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
23 - automatically detect schema changes and modify the table schemas |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
24 appropriately (we store the "database version" of the schema in the |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
25 database itself as the only row of the "schema" table) |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
26 - multilinks (which represent a many-to-many relationship) are handled through |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
27 intermediate tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
28 - journals are stored adjunct to the per-class tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
29 - table names and columns have "_" prepended so the names can't clash with |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
30 restricted names (like "order") |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
31 - retirement is determined by the __retired__ column being > 0 |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
32 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
33 Database-specific changes may generally be pushed out to the overridable |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
34 sql_* methods, since everything else should be fairly generic. There's |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
35 probably a bit of work to be done if a database is used that actually |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
36 honors column typing, since the initial databases don't (sqlite stores |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
37 everything as a string.) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
38 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
39 The schema of the hyperdb being mapped to the database is stored in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
40 database itself as a repr()'ed dictionary of information about each Class |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
41 that maps to a table. If that information differs from the hyperdb schema, |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
42 then we update it. We also store in the schema dict a version which |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
43 allows us to upgrade the database schema when necessary. See upgrade_db(). |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
44 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
45 To force a unqiueness constraint on the key properties we put the item |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
46 id into the __retired__ column duing retirement (so it's 0 for "active" |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
47 items) and place a unqiueness constraint on key + __retired__. This is |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
48 particularly important for the users class where multiple users may |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
49 try to have the same username, with potentially many retired users with |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
50 the same name. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
51 """ |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
52 __docformat__ = 'restructuredtext' |
| 1165 | 53 |
| 54 # standard python modules | |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
55 import sys, os, time, re, errno, weakref, copy, logging, datetime |
| 1165 | 56 |
| 57 # roundup modules | |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
58 from roundup import hyperdb, date, password, roundupdb, security, support |
| 1165 | 59 from roundup.hyperdb import String, Password, Date, Interval, Link, \ |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
60 Multilink, DatabaseError, Boolean, Number, Integer, Node |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
61 from roundup.backends import locking |
|
3925
603ec9630b08
i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3924
diff
changeset
|
62 from roundup.i18n import _ |
| 1165 | 63 |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
64 |
| 1165 | 65 # support |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
66 from roundup.backends.blobfiles import FileStorage |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5086
diff
changeset
|
67 from roundup.backends.indexer_common import get_indexer |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
68 from roundup.backends.sessions_rdbms import Sessions, OneTimeKeys |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
69 from roundup.date import Range |
| 1165 | 70 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
71 from roundup.backends.back_anydbm import compile_expression |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
72 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
73 |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
74 # dummy value meaning "argument not passed" |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
75 _marker = [] |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
76 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
77 def _num_cvt(num): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
78 num = str(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
79 try: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
80 return int(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
81 except: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
82 return float(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
83 |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
84 def _bool_cvt(value): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
85 if value in ('TRUE', 'FALSE'): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
86 return {'TRUE': 1, 'FALSE': 0}[value] |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
87 # assume it's a number returned from the db API |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
88 return int(value) |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
89 |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
90 def date_to_hyperdb_value(d): |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
91 """ convert date d to a roundup date """ |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
92 if isinstance (d, datetime.datetime): |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
93 return date.Date(d) |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
94 return date.Date (str(d).replace(' ', '.')) |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
95 |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
96 |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
97 def connection_dict(config, dbnamestr=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
98 """ Used by Postgresql and MySQL to detemine the keyword args for |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
99 opening the database connection.""" |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
100 d = { } |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
101 if dbnamestr: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
102 d[dbnamestr] = config.RDBMS_NAME |
|
3099
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
103 for name in ('host', 'port', 'password', 'user', 'read_default_group', |
|
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
104 'read_default_file'): |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
105 cvar = 'RDBMS_'+name.upper() |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
106 if config[cvar] is not None: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
107 d[name] = config[cvar] |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
108 return d |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
109 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
110 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
111 class IdListOptimizer: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
112 """ To prevent flooding the SQL parser of the underlaying |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
113 db engine with "x IN (1, 2, 3, ..., <large number>)" collapses |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
114 these cases to "x BETWEEN 1 AND <large number>". |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
115 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
116 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
117 def __init__(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
118 self.ranges = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
119 self.singles = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
120 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
121 def append(self, nid): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
122 """ Invariant: nid are ordered ascending """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
123 if self.ranges: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
124 last = self.ranges[-1] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
125 if last[1] == nid-1: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
126 last[1] = nid |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
127 return |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
128 if self.singles: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
129 last = self.singles[-1] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
130 if last == nid-1: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
131 self.singles.pop() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
132 self.ranges.append([last, nid]) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
133 return |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
134 self.singles.append(nid) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
135 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
136 def where(self, field, placeholder): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
137 ranges = self.ranges |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
138 singles = self.singles |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
139 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
140 if not singles and not ranges: return "(1=0)", [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
141 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
142 if ranges: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
143 between = '%s BETWEEN %s AND %s' % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
144 field, placeholder, placeholder) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
145 stmnt = [between] * len(ranges) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
146 else: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
147 stmnt = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
148 if singles: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
149 stmnt.append('%s in (%s)' % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
150 field, ','.join([placeholder]*len(singles)))) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
151 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
152 return '(%s)' % ' OR '.join(stmnt), sum(ranges, []) + singles |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
153 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
154 def __str__(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
155 return "ranges: %r / singles: %r" % (self.ranges, self.singles) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
156 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
157 |
| 1165 | 158 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
159 """ Wrapper around an SQL database that presents a hyperdb interface. |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
160 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
161 - some functionality is specific to the actual SQL database, hence |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
162 the sql_* methods that are NotImplemented |
|
4074
e039f3cbbb96
Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents:
4062
diff
changeset
|
163 - we keep a cache of the latest N row fetches (where N is configurable). |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
164 """ |
| 1165 | 165 def __init__(self, config, journaltag=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
166 """ Open the database and load the schema from it. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
167 """ |
|
3897
e2c2d91932ad
respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3872
diff
changeset
|
168 FileStorage.__init__(self, config.UMASK) |
| 1165 | 169 self.config, self.journaltag = config, journaltag |
| 170 self.dir = config.DATABASE | |
| 171 self.classes = {} | |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5086
diff
changeset
|
172 self.indexer = get_indexer(config, self) |
| 1165 | 173 self.security = security.Security(self) |
| 174 | |
| 175 # additional transaction support for external files and the like | |
| 176 self.transactions = [] | |
| 177 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
178 # keep a cache of the N most recently retrieved rows of any kind |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
179 # (classname, nodeid) = row |
|
4074
e039f3cbbb96
Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents:
4062
diff
changeset
|
180 self.cache_size = config.RDBMS_CACHE_SIZE |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
181 self.clearCache() |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
182 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
183 'filtering': 0} |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
184 |
|
5041
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
185 # make sure the database directory exists |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
186 if not os.path.isdir(self.config.DATABASE): |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
187 os.makedirs(self.config.DATABASE) |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
188 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
189 # database lock |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
190 self.lockfile = None |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
191 |
| 1165 | 192 # open a connection to the database, creating the "conn" attribute |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
193 self.open_connection() |
| 1165 | 194 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
195 def clearCache(self): |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
196 self.cache = {} |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
197 self.cache_lru = [] |
|
4652
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4517
diff
changeset
|
198 # upcall is necessary! |
|
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4517
diff
changeset
|
199 roundupdb.Database.clearCache(self) |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
200 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
201 def getSessionManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
202 return Sessions(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
203 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
204 def getOTKManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
205 return OneTimeKeys(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
206 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
207 def open_connection(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
208 """ Open a connection to the database, creating it if necessary. |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
209 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
210 Must call self.load_dbschema() |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
211 """ |
| 1165 | 212 raise NotImplemented |
| 213 | |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
214 def sql(self, sql, args=None, cursor=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
215 """ Execute the sql with the optional args. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
216 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
217 self.log_debug('SQL %r %r'%(sql, args)) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
218 if not cursor: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
219 cursor = self.cursor |
| 1165 | 220 if args: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
221 cursor.execute(sql, args) |
| 1165 | 222 else: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
223 cursor.execute(sql) |
| 1165 | 224 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
225 def sql_fetchone(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
226 """ Fetch a single row. If there's nothing to fetch, return None. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
227 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
228 return self.cursor.fetchone() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
229 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
230 def sql_fetchall(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
231 """ Fetch all rows. If there's nothing to fetch, return []. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
232 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
233 return self.cursor.fetchall() |
| 1165 | 234 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
235 def sql_fetchiter(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
236 """ Fetch all row as a generator |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
237 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
238 while True: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
239 row = self.cursor.fetchone() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
240 if not row: break |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
241 yield row |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
242 |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
243 def search_stringquote(self, value): |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
244 """ Quote a search string to escape magic search characters |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
245 '%' and '_', also need to quote '\' (first) |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
246 Then put '%' around resulting string for LIKE (or ILIKE) operator |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
247 """ |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
248 v = value.replace('\\', '\\\\') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
249 v = v.replace('%', '\\%') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
250 v = v.replace('_', '\\_') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
251 return '%' + v + '%' |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
252 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
253 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
254 def init_dbschema(self): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
255 self.database_schema = { |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
256 'version': self.current_db_version, |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
257 'tables': {} |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
258 } |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
259 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
260 def load_dbschema(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
261 """ Load the schema definition that the database currently implements |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
262 """ |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
263 self.cursor.execute('select schema from schema') |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
264 schema = self.cursor.fetchone() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
265 if schema: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
266 self.database_schema = eval(schema[0]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
267 else: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
268 self.database_schema = {} |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
269 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
270 def save_dbschema(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
271 """ Save the schema definition that the database currently implements |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
272 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
273 s = repr(self.database_schema) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
274 self.sql('delete from schema') |
|
3718
0d561b24ceff
support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents:
3694
diff
changeset
|
275 self.sql('insert into schema values (%s)'%self.arg, (s,)) |
| 1165 | 276 |
| 277 def post_init(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
278 """ Called once the schema initialisation has finished. |
| 1165 | 279 |
| 280 We should now confirm that the schema defined by our "classes" | |
| 281 attribute actually matches the schema in the database. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
282 """ |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
283 |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
284 # upgrade the database for column type changes, new internal |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
285 # tables, etc. |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
286 save = self.upgrade_db() |
| 3553 | 287 |
| 288 # handle changes in the schema | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
289 tables = self.database_schema['tables'] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
290 for classname, spec in self.classes.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
291 if classname in tables: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
292 dbspec = tables[classname] |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
293 if self.update_class(spec, dbspec): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
294 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
295 save = 1 |
| 1165 | 296 else: |
| 297 self.create_class(spec) | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
298 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
299 save = 1 |
| 1165 | 300 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
301 for classname, spec in list(tables.items()): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
302 if classname not in self.classes: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
303 self.drop_class(classname, tables[classname]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
304 del tables[classname] |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
305 save = 1 |
| 1165 | 306 |
| 307 # update the database version of the schema | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
308 if save: |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
309 self.save_dbschema() |
| 1165 | 310 |
| 311 # reindex the db if necessary | |
| 312 if self.indexer.should_reindex(): | |
| 313 self.reindex() | |
| 314 | |
| 315 # commit | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
316 self.sql_commit() |
| 1165 | 317 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
318 # update this number when we need to make changes to the SQL structure |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
319 # of the backen database |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
320 current_db_version = 5 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
321 db_version_updated = False |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
322 def upgrade_db(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
323 """ Update the SQL database to reflect changes in the backend code. |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
324 |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
325 Return boolean whether we need to save the schema. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
326 """ |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
327 version = self.database_schema.get('version', 1) |
|
3993
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
328 if version > self.current_db_version: |
|
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
329 raise DatabaseError('attempting to run rev %d DATABASE with rev ' |
|
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
330 '%d CODE!'%(version, self.current_db_version)) |
|
2081
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
331 if version == self.current_db_version: |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
332 # nothing to do |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
333 return 0 |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
334 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
335 if version < 2: |
| 4062 | 336 self.log_info('upgrade to version 2') |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
337 # change the schema structure |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
338 self.database_schema = {'tables': self.database_schema} |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
339 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
340 # version 1 didn't have the actor column (note that in |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
341 # MySQL this will also transition the tables to typed columns) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
342 self.add_new_columns_v2() |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
343 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
344 # version 1 doesn't have the OTK, session and indexing in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
345 # database |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
346 self.create_version_2_tables() |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
347 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
348 if version < 3: |
| 4062 | 349 self.log_info('upgrade to version 3') |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
350 self.fix_version_2_tables() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
351 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
352 if version < 4: |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
353 self.log_info('upgrade to version 4') |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
354 self.fix_version_3_tables() |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
355 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
356 if version < 5: |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
357 self.log_info('upgrade to version 5') |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
358 self.fix_version_4_tables() |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
359 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
360 self.database_schema['version'] = self.current_db_version |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
361 self.db_version_updated = True |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
362 return 1 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
363 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
364 def fix_version_3_tables(self): |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
365 # drop the shorter VARCHAR OTK column and add a new TEXT one |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
366 for name in ('otk', 'session'): |
|
2755
e3cd28cec23d
Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents:
2733
diff
changeset
|
367 self.sql('DELETE FROM %ss'%name) |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
368 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
369 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
370 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
371 def fix_version_2_tables(self): |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
372 # Default (used by sqlite): NOOP |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
373 pass |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
374 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
375 def fix_version_4_tables(self): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
376 # note this is an explicit call now |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
377 c = self.cursor |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
378 for cn, klass in self.classes.iteritems(): |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
379 c.execute('select id from _%s where __retired__<>0'%(cn,)) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
380 for (id,) in c.fetchall(): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
381 c.execute('update _%s set __retired__=%s where id=%s'%(cn, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
382 self.arg, self.arg), (id, id)) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
383 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
384 if klass.key: |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
385 self.add_class_key_required_unique_constraint(cn, klass.key) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
386 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
387 def _convert_journal_tables(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
388 """Get current journal table contents, drop the table and re-create""" |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
389 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
390 cols = ','.join('nodeid date tag action params'.split()) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
391 for klass in self.classes.itervalues(): |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
392 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
393 sql = 'select %s from %s__journal order by date'%(cols, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
394 klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
395 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
396 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
397 self.drop_journal_table_indexes(klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
398 c.execute('drop table %s__journal'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
399 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
400 # re-create and re-populate |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
401 self.create_journal_table(klass) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
402 a = self.arg |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
403 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
404 klass.classname, cols, a, a, a, a, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
405 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
406 # no data conversion needed |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
407 self.cursor.execute(sql, row) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
408 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
409 def _convert_string_properties(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
410 """Get current Class tables that contain String properties, and |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
411 convert the VARCHAR columns to TEXT""" |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
412 c = self.cursor |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
413 for klass in self.classes.itervalues(): |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
414 # slurp and drop |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
415 cols, mls = self.determine_columns(list(klass.properties.iteritems())) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
416 scols = ','.join([i[0] for i in cols]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
417 sql = 'select id,%s from _%s'%(scols, klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
418 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
419 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
420 self.drop_class_table_indexes(klass.classname, klass.getkey()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
421 c.execute('drop table _%s'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
422 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
423 # re-create and re-populate |
|
2424
74474ec41050
argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents:
2416
diff
changeset
|
424 self.create_class_table(klass, create_sequence=0) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
425 a = ','.join([self.arg for i in range(len(cols)+1)]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
426 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
427 scols, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
428 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
429 l = [] |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
430 for entry in row: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
431 # mysql will already be a string - psql needs "help" |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
432 if entry is not None and not isinstance(entry, type('')): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
433 entry = str(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
434 l.append(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
435 self.cursor.execute(sql, l) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
436 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
437 def refresh_database(self): |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
438 self.post_init() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
439 |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
440 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
441 def reindex(self, classname=None, show_progress=False): |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
442 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
443 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
444 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
445 classes = list(self.classes.itervalues()) |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
446 for klass in classes: |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
447 if show_progress: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
448 for nodeid in support.Progress('Reindex %s'%klass.classname, |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
449 klass.list()): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
450 klass.index(nodeid) |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
451 else: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
452 for nodeid in klass.list(): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
453 klass.index(nodeid) |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
454 self.indexer.save_index() |
| 1165 | 455 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
456 hyperdb_to_sql_datatypes = { |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
457 hyperdb.String : 'TEXT', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
458 hyperdb.Date : 'TIMESTAMP', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
459 hyperdb.Link : 'INTEGER', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
460 hyperdb.Interval : 'VARCHAR(255)', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
461 hyperdb.Password : 'VARCHAR(255)', |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
462 hyperdb.Boolean : 'BOOLEAN', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
463 hyperdb.Number : 'REAL', |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
464 hyperdb.Integer : 'INTEGER', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
465 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
466 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
467 def hyperdb_to_sql_datatype(self, propclass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
468 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
469 datatype = self.hyperdb_to_sql_datatypes.get(propclass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
470 if datatype: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
471 return datatype |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
472 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
473 for k, v in self.hyperdb_to_sql_datatypes.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
474 if issubclass(propclass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
475 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
476 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
477 raise ValueError('%r is not a hyperdb property class' % propclass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
478 |
| 1165 | 479 def determine_columns(self, properties): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
480 """ Figure the column names and multilink properties from the spec |
| 1165 | 481 |
| 482 "properties" is a list of (name, prop) where prop may be an | |
| 483 instance of a hyperdb "type" _or_ a string repr of that type. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
484 """ |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
485 cols = [ |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
486 ('_actor', self.hyperdb_to_sql_datatype(hyperdb.Link)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
487 ('_activity', self.hyperdb_to_sql_datatype(hyperdb.Date)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
488 ('_creator', self.hyperdb_to_sql_datatype(hyperdb.Link)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
489 ('_creation', self.hyperdb_to_sql_datatype(hyperdb.Date)), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
490 ] |
| 1165 | 491 mls = [] |
| 492 # add the multilinks separately | |
| 493 for col, prop in properties: | |
| 494 if isinstance(prop, Multilink): | |
| 495 mls.append(col) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
496 continue |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
497 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
498 if isinstance(prop, type('')): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
499 raise ValueError("string property spec!") |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
500 #and prop.find('Multilink') != -1: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
501 #mls.append(col) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
502 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
503 datatype = self.hyperdb_to_sql_datatype(prop.__class__) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
504 cols.append(('_'+col, datatype)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
505 |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
506 # Intervals stored as two columns |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
507 if isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
508 cols.append(('__'+col+'_int__', 'BIGINT')) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
509 |
| 1165 | 510 cols.sort() |
| 511 return cols, mls | |
| 512 | |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
513 def update_class(self, spec, old_spec, force=0): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
514 """ Determine the differences between the current spec and the |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
515 database version of the spec, and update where necessary. |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
516 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
517 If 'force' is true, update the database anyway. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
518 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
519 new_spec = spec.schema() |
|
1515
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
520 new_spec[1].sort() |
|
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
521 old_spec[1].sort() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
522 if not force and new_spec == old_spec: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
523 # no changes |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
524 return 0 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
525 |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
526 if not self.config.RDBMS_ALLOW_ALTER: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
527 raise DatabaseError(_('ALTER operation disallowed: %r -> %r.'%(old_spec, new_spec))) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
528 |
|
4420
9655a1b65974
- more logger fixes -- use correct hierarchical logger names...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4410
diff
changeset
|
529 logger = logging.getLogger('roundup.hyperdb') |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
530 logger.info('update_class %s'%spec.classname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
531 |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
532 logger.debug('old_spec %r'%(old_spec,)) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
533 logger.debug('new_spec %r'%(new_spec,)) |
| 1165 | 534 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
535 # detect key prop change for potential index change |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
536 keyprop_changes = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
537 if new_spec[0] != old_spec[0]: |
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
538 if old_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
539 keyprop_changes['remove'] = old_spec[0] |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
540 if new_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
541 keyprop_changes['add'] = new_spec[0] |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
542 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
543 # detect multilinks that have been removed, and drop their table |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
544 old_has = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
545 for name, prop in old_spec[1]: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
546 old_has[name] = 1 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
547 if name in spec.properties: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
548 continue |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
549 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
550 if prop.find('Multilink to') != -1: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
551 # first drop indexes. |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
552 self.drop_multilink_table_indexes(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
553 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
554 # now the multilink table itself |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
555 sql = 'drop table %s_%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
556 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
557 # if this is the key prop, drop the index first |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
558 if old_spec[0] == prop: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
559 self.drop_class_table_key_index(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
560 del keyprop_changes['remove'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
561 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
562 # drop the column |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
563 sql = 'alter table _%s drop column _%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
564 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
565 self.sql(sql) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
566 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
567 # if we didn't remove the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
568 # changed, we still need to remove the old index |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
569 if 'remove' in keyprop_changes: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
570 self.drop_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
571 keyprop_changes['remove']) |
| 1165 | 572 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
573 # add new columns |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
574 for propname, prop in new_spec[1]: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
575 if propname in old_has: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
576 continue |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
577 prop = spec.properties[propname] |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
578 if isinstance(prop, Multilink): |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
579 self.create_multilink_table(spec, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
580 else: |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
581 # add the column |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
582 coltype = self.hyperdb_to_sql_datatype(prop.__class__) |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
583 sql = 'alter table _%s add column _%s %s'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
584 spec.classname, propname, coltype) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
585 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
586 |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
587 # extra Interval column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
588 if isinstance(prop, Interval): |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
589 sql = 'alter table _%s add column __%s_int__ BIGINT'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
590 spec.classname, propname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
591 self.sql(sql) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
592 |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
593 # if the new column is a key prop, we need an index! |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
594 if new_spec[0] == propname: |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
595 self.create_class_table_key_index(spec.classname, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
596 del keyprop_changes['add'] |
| 1165 | 597 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
598 # if we didn't add the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
599 # changed, we still need to add the new index |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
600 if 'add' in keyprop_changes: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
601 self.create_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
602 keyprop_changes['add']) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
603 |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
604 return 1 |
| 1165 | 605 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
606 def determine_all_columns(self, spec): |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
607 """Figure out the columns from the spec and also add internal columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
608 |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
609 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
610 cols, mls = self.determine_columns(list(spec.properties.iteritems())) |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
611 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
612 # add on our special columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
613 cols.append(('id', 'INTEGER PRIMARY KEY')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
614 cols.append(('__retired__', 'INTEGER DEFAULT 0')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
615 return cols, mls |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
616 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
617 def create_class_table(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
618 """Create the class table for the given Class "spec". Creates the |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
619 indexes too.""" |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
620 cols, mls = self.determine_all_columns(spec) |
| 1165 | 621 |
| 622 # create the base table | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
623 scols = ','.join(['%s %s'%x for x in cols]) |
| 1165 | 624 sql = 'create table _%s (%s)'%(spec.classname, scols) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
625 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
626 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
627 self.create_class_table_indexes(spec) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
628 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
629 return cols, mls |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
630 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
631 def create_class_table_indexes(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
632 """ create the class table for the given spec |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
633 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
634 # create __retired__ index |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
635 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%( |
|
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
636 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
637 self.sql(index_sql2) |
| 1165 | 638 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
639 # create index for key property |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
640 if spec.key: |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
641 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%( |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
642 spec.classname, spec.key, |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
643 spec.classname, spec.key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
644 self.sql(index_sql3) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
645 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
646 # and the unique index for key / retired(id) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
647 self.add_class_key_required_unique_constraint(spec.classname, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
648 spec.key) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
649 |
|
2228
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
650 # TODO: create indexes on (selected?) Link property columns, as |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
651 # they're more likely to be used for lookup |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
652 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
653 def add_class_key_required_unique_constraint(self, cn, key): |
|
3969
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3967
diff
changeset
|
654 sql = '''create unique index _%s_key_retired_idx |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
655 on _%s(__retired__, _%s)'''%(cn, cn, key) |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
656 try: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
657 self.sql(sql) |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
658 except StandardError: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
659 # XXX catch e.g.: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
660 # _sqlite.DatabaseError: index _status_key_retired_idx already exists |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
661 pass |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
662 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
663 def drop_class_table_indexes(self, cn, key): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
664 # drop the old table indexes first |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
665 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
666 if key: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
667 l.append('_%s_%s_idx'%(cn, key)) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
668 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
669 table_name = '_%s'%cn |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
670 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
671 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
672 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
673 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
674 self.sql(index_sql) |
| 1165 | 675 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
676 def create_class_table_key_index(self, cn, key): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
677 """ create the class table for the given spec |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
678 """ |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
679 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
680 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
681 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
682 def drop_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
683 table_name = '_%s'%cn |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
684 index_name = '_%s_%s_idx'%(cn, key) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
685 if self.sql_index_exists(table_name, index_name): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
686 sql = 'drop index '+index_name |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
687 self.sql(sql) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
688 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
689 # and now the retired unique index too |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
690 index_name = '_%s_key_retired_idx'%cn |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
691 if self.sql_index_exists(table_name, index_name): |
|
3967
b1e81ad3fa6a
don't recompute the index name
Richard Jones <richard@users.sourceforge.net>
parents:
3963
diff
changeset
|
692 sql = 'drop index '+index_name |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
693 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
694 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
695 def create_journal_table(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
696 """ create the journal table for a class given the spec and |
| 1165 | 697 already-determined cols |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
698 """ |
| 1165 | 699 # journal table |
| 700 cols = ','.join(['%s varchar'%x | |
| 701 for x in 'nodeid date tag action params'.split()]) | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
702 sql = """create table %s__journal ( |
|
3224
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
703 nodeid integer, date %s, tag varchar(255), |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
704 action varchar(255), params text)""" % (spec.classname, |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
705 self.hyperdb_to_sql_datatype(hyperdb.Date)) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
706 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
707 self.create_journal_table_indexes(spec) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
708 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
709 def create_journal_table_indexes(self, spec): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
710 # index on nodeid |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
711 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%( |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
712 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
713 self.sql(sql) |
| 1165 | 714 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
715 def drop_journal_table_indexes(self, classname): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
716 index_name = '%s_journ_idx'%classname |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
717 if not self.sql_index_exists('%s__journal'%classname, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
718 return |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
719 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
720 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
721 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
722 def create_multilink_table(self, spec, ml): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
723 """ Create a multilink table for the "ml" property of the class |
| 1165 | 724 given by the spec |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
725 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
726 # create the table |
|
2828
3a2f138fcb28
use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2755
diff
changeset
|
727 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%( |
| 1165 | 728 spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
729 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
730 self.create_multilink_table_indexes(spec, ml) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
731 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
732 def create_multilink_table_indexes(self, spec, ml): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
733 # create index on linkid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
734 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
735 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
736 self.sql(index_sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
737 |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
738 # create index on nodeid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
739 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
740 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
741 self.sql(index_sql) |
| 1165 | 742 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
743 def drop_multilink_table_indexes(self, classname, ml): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
744 l = [ |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
745 '%s_%s_l_idx'%(classname, ml), |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
746 '%s_%s_n_idx'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
747 ] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
748 table_name = '%s_%s'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
749 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
750 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
751 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
752 index_sql = 'drop index %s'%index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
753 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
754 |
| 1165 | 755 def create_class(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
756 """ Create a database table according to the given spec. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
757 """ |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
758 |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
759 if not self.config.RDBMS_ALLOW_CREATE: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
760 raise DatabaseError(_('CREATE operation disallowed: "%s".'%spec.classname)) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
761 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
762 cols, mls = self.create_class_table(spec) |
|
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
763 self.create_journal_table(spec) |
| 1165 | 764 |
| 765 # now create the multilink tables | |
| 766 for ml in mls: | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
767 self.create_multilink_table(spec, ml) |
| 1165 | 768 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
769 def drop_class(self, cn, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
770 """ Drop the given table from the database. |
| 1165 | 771 |
| 772 Drop the journal and multilink tables too. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
773 """ |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
774 |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
775 if not self.config.RDBMS_ALLOW_DROP: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
776 raise DatabaseError(_('DROP operation disallowed: "%s".'%cn)) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
777 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
778 properties = spec[1] |
| 1165 | 779 # figure the multilinks |
| 780 mls = [] | |
|
3924
21d3d7eeea8c
assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3906
diff
changeset
|
781 for propname, prop in properties: |
| 1165 | 782 if isinstance(prop, Multilink): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
783 mls.append(propname) |
| 1165 | 784 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
785 # drop class table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
786 self.drop_class_table_indexes(cn, spec[0]) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
787 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
788 self.drop_class_table(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
789 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
790 # drop journal table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
791 self.drop_journal_table_indexes(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
792 sql = 'drop table %s__journal'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
793 self.sql(sql) |
| 1165 | 794 |
| 795 for ml in mls: | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
796 # drop multilink table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
797 self.drop_multilink_table_indexes(cn, ml) |
| 1165 | 798 sql = 'drop table %s_%s'%(spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
799 self.sql(sql) |
| 1165 | 800 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
801 def drop_class_table(self, cn): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
802 sql = 'drop table _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
803 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
804 |
| 1165 | 805 # |
| 806 # Classes | |
| 807 # | |
| 808 def __getattr__(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
809 """ A convenient way of calling self.getclass(classname). |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
810 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
811 if classname in self.classes: |
| 1165 | 812 return self.classes[classname] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
813 raise AttributeError(classname) |
| 1165 | 814 |
| 815 def addclass(self, cl): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
816 """ Add a Class to the hyperdatabase. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
817 """ |
| 1165 | 818 cn = cl.classname |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
819 if cn in self.classes: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
820 raise ValueError(cn) |
| 1165 | 821 self.classes[cn] = cl |
| 822 | |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
823 # add default Edit and View permissions |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
824 self.security.addPermission(name="Create", klass=cn, |
|
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
825 description="User is allowed to create "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
826 self.security.addPermission(name="Edit", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
827 description="User is allowed to edit "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
828 self.security.addPermission(name="View", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
829 description="User is allowed to access "+cn) |
|
4517
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4490
diff
changeset
|
830 self.security.addPermission(name="Retire", klass=cn, |
|
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4490
diff
changeset
|
831 description="User is allowed to retire "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
832 |
| 1165 | 833 def getclasses(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
834 """ Return a list of the names of all existing classes. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
835 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
836 return sorted(self.classes) |
| 1165 | 837 |
| 838 def getclass(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
839 """Get the Class object representing a particular class. |
| 1165 | 840 |
| 841 If 'classname' is not a valid class name, a KeyError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
842 """ |
| 1165 | 843 try: |
| 844 return self.classes[classname] | |
| 845 except KeyError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
846 raise KeyError('There is no class called "%s"'%classname) |
| 1165 | 847 |
| 848 def clear(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
849 """Delete all database contents. |
| 1165 | 850 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
851 Note: I don't commit here, which is different behaviour to the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
852 "nuke from orbit" behaviour in the dbs. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
853 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
854 logging.getLogger('roundup.hyperdb').info('clear') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
855 for cn in self.classes: |
| 1165 | 856 sql = 'delete from _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
857 self.sql(sql) |
| 1165 | 858 |
| 859 # | |
| 860 # Nodes | |
| 861 # | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
862 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
863 hyperdb_to_sql_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
864 hyperdb.String : str, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
865 # fractional seconds by default |
|
3612
afda59d5d546
prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents:
3601
diff
changeset
|
866 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
867 hyperdb.Link : int, |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
868 hyperdb.Interval : str, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
869 hyperdb.Password : str, |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
870 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
871 hyperdb.Number : lambda x: x, |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
872 hyperdb.Integer : lambda x: x, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
873 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
874 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
875 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
876 def to_sql_value(self, propklass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
877 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
878 fn = self.hyperdb_to_sql_value.get(propklass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
879 if fn: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
880 return fn |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
881 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
882 for k, v in self.hyperdb_to_sql_value.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
883 if issubclass(propklass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
884 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
885 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
886 raise ValueError('%r is not a hyperdb property class' % propklass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
887 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
888 def _cache_del(self, key): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
889 del self.cache[key] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
890 self.cache_lru.remove(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
891 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
892 def _cache_refresh(self, key): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
893 self.cache_lru.remove(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
894 self.cache_lru.insert(0, key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
895 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
896 def _cache_save(self, key, node): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
897 self.cache[key] = node |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
898 # update the LRU |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
899 self.cache_lru.insert(0, key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
900 if len(self.cache_lru) > self.cache_size: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
901 del self.cache[self.cache_lru.pop()] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
902 |
| 1165 | 903 def addnode(self, classname, nodeid, node): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
904 """ Add the specified node to its class's db. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
905 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
906 self.log_debug('addnode %s%s %r'%(classname, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
907 nodeid, node)) |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
908 |
|
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
909 # determine the column definitions and multilink tables |
| 1165 | 910 cl = self.classes[classname] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
911 cols, mls = self.determine_columns(list(cl.properties.iteritems())) |
| 1165 | 912 |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
913 # we'll be supplied these props if we're doing an import |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
914 values = node.copy() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
915 if 'creator' not in values: |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
916 # add in the "calculated" properties (dupe so we don't affect |
|
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
917 # calling code's node assumptions) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
918 values['creation'] = values['activity'] = date.Date() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
919 values['actor'] = values['creator'] = self.getuid() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
920 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
921 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
922 props = cl.getprops(protected=1) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
923 del props['id'] |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
924 |
| 1165 | 925 # default the non-multilink columns |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
926 for col, prop in props.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
927 if col not in values: |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
928 if isinstance(prop, Multilink): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
929 values[col] = [] |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
930 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
931 values[col] = None |
| 1165 | 932 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
933 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
934 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
935 if key in self.cache: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
936 self._cache_del(key) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
937 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
938 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
939 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
940 for col,dt in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
941 # this is somewhat dodgy.... |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
942 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
943 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
944 value = values[col[2:-6]] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
945 # this is an Interval special "int" column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
946 if value is not None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
947 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
948 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
949 vals.append(value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
950 continue |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
951 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
952 prop = props[col[1:]] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
953 value = values[col[1:]] |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
954 if value is not None: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
955 value = self.to_sql_value(prop.__class__)(value) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
956 vals.append(value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
957 vals.append(nodeid) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
958 vals = tuple(vals) |
| 1165 | 959 |
| 960 # make sure the ordering is correct for column name -> column value | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
961 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
962 cols = ','.join([col for col,dt in cols]) + ',id' |
| 1165 | 963 |
| 964 # perform the inserts | |
| 965 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
966 self.sql(sql, vals) |
| 1165 | 967 |
| 968 # insert the multilink rows | |
| 969 for col in mls: | |
| 970 t = '%s_%s'%(classname, col) | |
| 971 for entry in node[col]: | |
| 972 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, | |
| 973 self.arg, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
974 self.sql(sql, (entry, nodeid)) |
| 1165 | 975 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
976 def setnode(self, classname, nodeid, values, multilink_changes={}): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
977 """ Change the specified node. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
978 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
979 self.log_debug('setnode %s%s %r' |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
980 % (classname, nodeid, values)) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
981 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
982 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
983 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
984 if key in self.cache: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
985 self._cache_del(key) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
986 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
987 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
988 props = cl.getprops() |
| 1165 | 989 |
| 990 cols = [] | |
| 991 mls = [] | |
| 992 # add the multilinks separately | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
993 for col in values: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
994 prop = props[col] |
| 1165 | 995 if isinstance(prop, Multilink): |
| 996 mls.append(col) | |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
997 elif isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
998 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
999 cols.append(col) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1000 # extra leading '_' added by code below |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1001 cols.append('_' +col + '_int__') |
| 1165 | 1002 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1003 cols.append(col) |
| 1165 | 1004 cols.sort() |
| 1005 | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1006 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1007 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1008 for col in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1009 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1010 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1011 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1012 col = col[1:-6] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1013 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1014 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1015 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1016 vals.append(None) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1017 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1018 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1019 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1020 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1021 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1022 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1023 e = None |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1024 else: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1025 e = self.to_sql_value(prop.__class__)(value) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1026 vals.append(e) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1027 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1028 vals.append(int(nodeid)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1029 vals = tuple(vals) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1030 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1031 # if there's any updates to regular columns, do them |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1032 if cols: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1033 # make sure the ordering is correct for column name -> column value |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1034 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols]) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1035 cols = ','.join(cols) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1036 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1037 # perform the update |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1038 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1039 self.sql(sql, vals) |
| 1165 | 1040 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1041 # we're probably coming from an import, not a change |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1042 if not multilink_changes: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1043 for name in mls: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1044 prop = props[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1045 value = values[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1046 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1047 t = '%s_%s'%(classname, name) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1048 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1049 # clear out previous values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1050 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1051 self.sql('delete from %s where nodeid=%s'%(t, self.arg), |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1052 (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1053 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1054 # insert the values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1055 for entry in values[name]: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1056 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1057 self.arg, self.arg) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1058 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1059 self.sql(sql, (entry, nodeid)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1060 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1061 # we have multilink changes to apply |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1062 for col, (add, remove) in multilink_changes.iteritems(): |
| 1165 | 1063 tn = '%s_%s'%(classname, col) |
| 1064 if add: | |
| 1065 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn, | |
| 1066 self.arg, self.arg) | |
| 1067 for addid in add: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1068 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1069 self.sql(sql, (int(nodeid), int(addid))) |
| 1165 | 1070 if remove: |
|
4039
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1071 s = ','.join([self.arg]*len(remove)) |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1072 sql = 'delete from %s where nodeid=%s and linkid in (%s)'%(tn, |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1073 self.arg, s) |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1074 # XXX numeric ids |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1075 self.sql(sql, [int(nodeid)] + remove) |
| 1165 | 1076 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1077 sql_to_hyperdb_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1078 hyperdb.String : str, |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
1079 hyperdb.Date : date_to_hyperdb_value, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1080 # hyperdb.Link : int, # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1081 hyperdb.Link : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1082 hyperdb.Interval : date.Interval, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1083 hyperdb.Password : lambda x: password.Password(encrypted=x), |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
1084 hyperdb.Boolean : _bool_cvt, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1085 hyperdb.Number : _num_cvt, |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1086 hyperdb.Integer : int, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1087 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1088 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1089 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1090 def to_hyperdb_value(self, propklass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1091 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1092 fn = self.sql_to_hyperdb_value.get(propklass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1093 if fn: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1094 return fn |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1095 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1096 for k, v in self.sql_to_hyperdb_value.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1097 if issubclass(propklass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1098 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1099 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1100 raise ValueError('%r is not a hyperdb property class' % propklass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1101 |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1102 def _materialize_multilink(self, classname, nodeid, node, propname): |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1103 """ evaluation of single Multilink (lazy eval may have skipped this) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1104 """ |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1105 if propname not in node: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1106 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1107 propname, self.arg) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1108 self.sql(sql, (nodeid,)) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1109 # extract the first column from the result |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1110 # XXX numeric ids |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1111 items = [int(x[0]) for x in self.cursor.fetchall()] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1112 items.sort () |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1113 node[propname] = [str(x) for x in items] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1114 |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1115 def _materialize_multilinks(self, classname, nodeid, node, props=None): |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1116 """ get all Multilinks of a node (lazy eval may have skipped this) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1117 """ |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1118 cl = self.classes[classname] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1119 props = props or [pn for (pn, p) in cl.properties.iteritems() |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1120 if isinstance(p, Multilink)] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1121 for propname in props: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1122 if propname not in node: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1123 self._materialize_multilink(classname, nodeid, node, propname) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1124 |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1125 def getnode(self, classname, nodeid, fetch_multilinks=True): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1126 """ Get a node from the database. |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1127 For optimisation optionally we don't fetch multilinks |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1128 (lazy Multilinks). |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1129 But for internal database operations we need them. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1130 """ |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1131 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1132 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1133 if key in self.cache: |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1134 # push us back to the top of the LRU |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1135 self._cache_refresh(key) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1136 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1137 self.stats['cache_hits'] += 1 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1138 # return the cached information |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1139 if fetch_multilinks: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1140 self._materialize_multilinks(classname, nodeid, self.cache[key]) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1141 return self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1142 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1143 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1144 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1145 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1146 |
| 1165 | 1147 # figure the columns we're fetching |
| 1148 cl = self.classes[classname] | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1149 cols, mls = self.determine_columns(list(cl.properties.iteritems())) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1150 scols = ','.join([col for col,dt in cols]) |
| 1165 | 1151 |
| 1152 # perform the basic property fetch | |
| 1153 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1154 self.sql(sql, (nodeid,)) |
| 1165 | 1155 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1156 values = self.sql_fetchone() |
| 1165 | 1157 if values is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1158 raise IndexError('no such %s node %s'%(classname, nodeid)) |
| 1165 | 1159 |
| 1160 # make up the node | |
| 1161 node = {} | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1162 props = cl.getprops(protected=1) |
| 1165 | 1163 for col in range(len(cols)): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1164 name = cols[col][0][1:] |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1165 if name.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1166 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1167 # ignore the special Interval-as-seconds column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1168 continue |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1169 value = values[col] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1170 if value is not None: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1171 value = self.to_hyperdb_value(props[name].__class__)(value) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1172 node[name] = value |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1173 |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1174 if fetch_multilinks and mls: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1175 self._materialize_multilinks(classname, nodeid, node, mls) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1176 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1177 # save off in the cache |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1178 key = (classname, nodeid) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1179 self._cache_save(key, node) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1180 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1181 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1182 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1183 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1184 return node |
| 1165 | 1185 |
| 1186 def destroynode(self, classname, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1187 """Remove a node from the database. Called exclusively by the |
| 1165 | 1188 destroy() method on Class. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1189 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1190 logging.getLogger('roundup.hyperdb').info('destroynode %s%s'%( |
|
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1191 classname, nodeid)) |
| 1165 | 1192 |
| 1193 # make sure the node exists | |
| 1194 if not self.hasnode(classname, nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1195 raise IndexError('%s has no node %s'%(classname, nodeid)) |
| 1165 | 1196 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1197 # see if we have this node cached |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1198 if (classname, nodeid) in self.cache: |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1199 del self.cache[(classname, nodeid)] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1200 |
| 1165 | 1201 # see if there's any obvious commit actions that we should get rid of |
| 1202 for entry in self.transactions[:]: | |
| 1203 if entry[1][:2] == (classname, nodeid): | |
| 1204 self.transactions.remove(entry) | |
| 1205 | |
| 1206 # now do the SQL | |
| 1207 sql = 'delete from _%s where id=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1208 self.sql(sql, (nodeid,)) |
| 1165 | 1209 |
| 1210 # remove from multilnks | |
| 1211 cl = self.getclass(classname) | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1212 x, mls = self.determine_columns(list(cl.properties.iteritems())) |
| 1165 | 1213 for col in mls: |
| 1214 # get the link ids | |
| 1215 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg) | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
1216 self.sql(sql, (nodeid,)) |
| 1165 | 1217 |
| 1218 # remove journal entries | |
| 1219 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1220 self.sql(sql, (nodeid,)) |
| 1165 | 1221 |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1222 # cleanup any blob filestorage when we commit |
|
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1223 self.transactions.append((FileStorage.destroy, (self, classname, nodeid))) |
|
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1224 |
| 1165 | 1225 def hasnode(self, classname, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1226 """ Determine if the database has a given node. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1227 """ |
|
5086
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1228 # nodeid (aka link) is sql type INTEGER. max positive value |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1229 # for INTEGER is 2^31-1 for Postgres and MySQL. The max |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1230 # positive value for SqLite is 2^63 -1, so arguably this check |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1231 # needs to adapt for the type of the RDBMS. For right now, |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1232 # choose lowest common denominator. |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1233 if int(nodeid) >= 2**31: |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1234 # value out of range return false |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1235 return 0 |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1236 |
| 4031 | 1237 # If this node is in the cache, then we do not need to go to |
| 1238 # the database. (We don't consider this an LRU hit, though.) | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1239 if (classname, nodeid) in self.cache: |
| 4031 | 1240 # Return 1, not True, to match the type of the result of |
| 1241 # the SQL operation below. | |
| 1242 return 1 | |
| 1165 | 1243 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1244 self.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1245 return int(self.cursor.fetchone()[0]) |
| 1165 | 1246 |
| 1247 def countnodes(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1248 """ Count the number of nodes that exist for a particular Class. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1249 """ |
| 1165 | 1250 sql = 'select count(*) from _%s'%classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1251 self.sql(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1252 return self.cursor.fetchone()[0] |
| 1165 | 1253 |
| 1254 def addjournal(self, classname, nodeid, action, params, creator=None, | |
| 1255 creation=None): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1256 """ Journal the Action |
| 1165 | 1257 'action' may be: |
| 1258 | |
| 1259 'create' or 'set' -- 'params' is a dictionary of property values | |
| 1260 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) | |
| 1261 'retire' -- 'params' is None | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1262 """ |
| 1165 | 1263 # handle supply of the special journalling parameters (usually |
| 1264 # supplied on importing an existing database) | |
| 1265 if creator: | |
| 1266 journaltag = creator | |
| 1267 else: | |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1268 journaltag = self.getuid() |
| 1165 | 1269 if creation: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1270 journaldate = creation |
| 1165 | 1271 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1272 journaldate = date.Date() |
| 1165 | 1273 |
| 1274 # create the journal entry | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1275 cols = 'nodeid,date,tag,action,params' |
| 1165 | 1276 |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1277 self.log_debug('addjournal %s%s %r %s %s %r'%(classname, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1278 nodeid, journaldate, journaltag, action, params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1279 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1280 # make the journalled data marshallable |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1281 if isinstance(params, type({})): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1282 self._journal_marshal(params, classname) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1283 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1284 params = repr(params) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1285 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1286 dc = self.to_sql_value(hyperdb.Date) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1287 journaldate = dc(journaldate) |
| 1165 | 1288 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1289 self.save_journal(classname, cols, nodeid, journaldate, |
| 1165 | 1290 journaltag, action, params) |
| 1291 | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1292 def setjournal(self, classname, nodeid, journal): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1293 """Set the journal to the "journal" list.""" |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1294 # clear out any existing entries |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1295 self.sql('delete from %s__journal where nodeid=%s'%(classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1296 self.arg), (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1297 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1298 # create the journal entry |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1299 cols = 'nodeid,date,tag,action,params' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1300 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1301 dc = self.to_sql_value(hyperdb.Date) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1302 for nodeid, journaldate, journaltag, action, params in journal: |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1303 self.log_debug('addjournal %s%s %r %s %s %r'%( |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1304 classname, nodeid, journaldate, journaltag, action, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1305 params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1306 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1307 # make the journalled data marshallable |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1308 if isinstance(params, type({})): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1309 self._journal_marshal(params, classname) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1310 params = repr(params) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1311 |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1312 self.save_journal(classname, cols, nodeid, dc(journaldate), |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1313 journaltag, action, params) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1314 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1315 def _journal_marshal(self, params, classname): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1316 """Convert the journal params values into safely repr'able and |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1317 eval'able values.""" |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1318 properties = self.getclass(classname).getprops() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1319 for param, value in params.iteritems(): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1320 if not value: |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1321 continue |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1322 property = properties[param] |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1323 cvt = self.to_sql_value(property.__class__) |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1324 if isinstance(property, Password): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1325 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1326 elif isinstance(property, Date): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1327 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1328 elif isinstance(property, Interval): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1329 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1330 elif isinstance(property, Boolean): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1331 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1332 |
| 1165 | 1333 def getjournal(self, classname, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1334 """ get the journal for id |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1335 """ |
| 1165 | 1336 # make sure the node exists |
| 1337 if not self.hasnode(classname, nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1338 raise IndexError('%s has no node %s'%(classname, nodeid)) |
| 1165 | 1339 |
| 1340 cols = ','.join('nodeid date tag action params'.split()) | |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1341 journal = self.load_journal(classname, cols, nodeid) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1342 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1343 # now unmarshal the data |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1344 dc = self.to_hyperdb_value(hyperdb.Date) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1345 res = [] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1346 properties = self.getclass(classname).getprops() |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1347 for nodeid, date_stamp, user, action, params in journal: |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1348 params = eval(params) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1349 if isinstance(params, type({})): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1350 for param, value in params.iteritems(): |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1351 if not value: |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1352 continue |
|
2731
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1353 property = properties.get(param, None) |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1354 if property is None: |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1355 # deleted property |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1356 continue |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1357 cvt = self.to_hyperdb_value(property.__class__) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1358 if isinstance(property, Password): |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1359 params[param] = password.JournalPassword(value) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1360 elif isinstance(property, Date): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1361 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1362 elif isinstance(property, Interval): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1363 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1364 elif isinstance(property, Boolean): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1365 params[param] = cvt(value) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1366 # XXX numeric ids |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1367 res.append((str(nodeid), dc(date_stamp), user, action, params)) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1368 return res |
| 1165 | 1369 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1370 def save_journal(self, classname, cols, nodeid, journaldate, |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1371 journaltag, action, params): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1372 """ Save the journal entry to the database |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1373 """ |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1374 entry = (nodeid, journaldate, journaltag, action, params) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1375 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1376 # do the insert |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1377 a = self.arg |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1378 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1379 classname, cols, a, a, a, a, a) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1380 self.sql(sql, entry) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1381 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1382 def load_journal(self, classname, cols, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1383 """ Load the journal from the database |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1384 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1385 # now get the journal entries |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1386 sql = 'select %s from %s__journal where nodeid=%s order by date'%( |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1387 cols, classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1388 self.sql(sql, (nodeid,)) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1389 return self.cursor.fetchall() |
| 1165 | 1390 |
| 1391 def pack(self, pack_before): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1392 """ Delete all journal entries except "create" before 'pack_before'. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1393 """ |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1394 date_stamp = self.to_sql_value(Date)(pack_before) |
| 1165 | 1395 |
| 1396 # do the delete | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1397 for classname in self.classes: |
| 1165 | 1398 sql = "delete from %s__journal where date<%s and "\ |
| 1399 "action<>'create'"%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1400 self.sql(sql, (date_stamp,)) |
| 1165 | 1401 |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1402 def sql_commit(self, fail_ok=False): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1403 """ Actually commit to the database. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1404 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1405 logging.getLogger('roundup.hyperdb').info('commit') |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1406 |
| 1165 | 1407 self.conn.commit() |
| 1408 | |
|
2374
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1409 # open a new cursor for subsequent work |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1410 self.cursor = self.conn.cursor() |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1411 |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1412 def commit(self, fail_ok=False): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1413 """ Commit the current transactions. |
| 1165 | 1414 |
| 1415 Save all data changed since the database was opened or since the | |
| 1416 last commit() or rollback(). | |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1417 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1418 fail_ok indicates that the commit is allowed to fail. This is used |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1419 in the web interface when committing cleaning of the session |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1420 database. We don't care if there's a concurrency issue there. |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1421 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1422 The only backend this seems to affect is postgres. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1423 """ |
| 1165 | 1424 # commit the database |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1425 self.sql_commit(fail_ok) |
| 1165 | 1426 |
| 1427 # now, do all the other transaction stuff | |
| 1428 for method, args in self.transactions: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1429 method(*args) |
| 1165 | 1430 |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1431 # save the indexer |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1432 self.indexer.save_index() |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1433 |
| 1165 | 1434 # clear out the transactions |
| 1435 self.transactions = [] | |
| 1436 | |
|
4448
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1437 # clear the cache: Don't carry over cached values from one |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1438 # transaction to the next (there may be other changes from other |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1439 # transactions) |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1440 self.clearCache() |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1441 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1442 def sql_rollback(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1443 self.conn.rollback() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1444 |
| 1165 | 1445 def rollback(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1446 """ Reverse all actions from the current transaction. |
| 1165 | 1447 |
| 1448 Undo all the changes made since the database was opened or the last | |
| 1449 commit() or rollback() was performed. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1450 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1451 logging.getLogger('roundup.hyperdb').info('rollback') |
| 1165 | 1452 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1453 self.sql_rollback() |
| 1165 | 1454 |
| 1455 # roll back "other" transaction stuff | |
| 1456 for method, args in self.transactions: | |
| 1457 # delete temporary files | |
| 1458 if method == self.doStoreFile: | |
| 1459 self.rollbackStoreFile(*args) | |
| 1460 self.transactions = [] | |
| 1461 | |
|
1492
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1462 # clear the cache |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1463 self.clearCache() |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1464 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1465 def sql_close(self): |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1466 logging.getLogger('roundup.hyperdb').info('close') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1467 self.conn.close() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1468 |
| 1165 | 1469 def close(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1470 """ Close off the connection. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1471 """ |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
1472 self.indexer.close() |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1473 self.sql_close() |
| 1165 | 1474 |
| 1475 # | |
| 1476 # The base Class class | |
| 1477 # | |
| 1478 class Class(hyperdb.Class): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1479 """ The handle to a particular class of nodes in a hyperdatabase. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1480 |
| 1165 | 1481 All methods except __repr__ and getnode must be implemented by a |
| 1482 concrete backend Class. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1483 """ |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1484 # For many databases the LIKE operator ignores case. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1485 # Postgres and Oracle have an ILIKE operator to support this. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1486 # We define the default here, can be changed in derivative class |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1487 case_insensitive_like = 'LIKE' |
| 1165 | 1488 |
| 1489 def schema(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1490 """ A dumpable version of the schema that we can store in the |
| 1165 | 1491 database |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1492 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1493 return (self.key, [(x, repr(y)) for x,y in self.properties.iteritems()]) |
| 1165 | 1494 |
| 1495 def enableJournalling(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1496 """Turn journalling on for this class |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1497 """ |
| 1165 | 1498 self.do_journal = 1 |
| 1499 | |
| 1500 def disableJournalling(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1501 """Turn journalling off for this class |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1502 """ |
| 1165 | 1503 self.do_journal = 0 |
| 1504 | |
| 1505 # Editing nodes: | |
| 1506 def create(self, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1507 """ Create a new node of this class and return its id. |
| 1165 | 1508 |
| 1509 The keyword arguments in 'propvalues' map property names to values. | |
| 1510 | |
| 1511 The values of arguments must be acceptable for the types of their | |
| 1512 corresponding properties or a TypeError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1513 |
| 1165 | 1514 If this class has a key property, it must be present and its value |
| 1515 must not collide with other key strings or a ValueError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1516 |
| 1165 | 1517 Any other properties on this class that are missing from the |
| 1518 'propvalues' dictionary are set to None. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1519 |
| 1165 | 1520 If an id in a link or multilink property does not refer to a valid |
| 1521 node, an IndexError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1522 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1523 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1524 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1525 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1526 return newid |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1527 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1528 def create_inner(self, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1529 """ Called by create, in-between the audit and react calls. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1530 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1531 if 'id' in propvalues: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1532 raise KeyError('"id" is reserved') |
| 1165 | 1533 |
| 1534 if self.db.journaltag is None: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1535 raise DatabaseError(_('Database open read-only')) |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1536 |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1537 if ('creator' in propvalues or 'actor' in propvalues or |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1538 'creation' in propvalues or 'activity' in propvalues): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1539 raise KeyError('"creator", "actor", "creation" and ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1540 '"activity" are reserved') |
| 1165 | 1541 |
| 1542 # new node's id | |
| 1543 newid = self.db.newid(self.classname) | |
| 1544 | |
| 1545 # validate propvalues | |
| 1546 num_re = re.compile('^\d+$') | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1547 for key, value in propvalues.iteritems(): |
| 1165 | 1548 if key == self.key: |
| 1549 try: | |
| 1550 self.lookup(value) | |
| 1551 except KeyError: | |
| 1552 pass | |
| 1553 else: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1554 raise ValueError('node with key "%s" exists'%value) |
| 1165 | 1555 |
| 1556 # try to handle this property | |
| 1557 try: | |
| 1558 prop = self.properties[key] | |
| 1559 except KeyError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1560 raise KeyError('"%s" has no property "%s"'%(self.classname, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1561 key)) |
| 1165 | 1562 |
| 1563 if value is not None and isinstance(prop, Link): | |
| 1564 if type(value) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1565 raise ValueError('link value must be String') |
| 1165 | 1566 link_class = self.properties[key].classname |
| 1567 # if it isn't a number, it's a key | |
| 1568 if not num_re.match(value): | |
| 1569 try: | |
| 1570 value = self.db.classes[link_class].lookup(value) | |
| 1571 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1572 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1573 key, value, link_class)) |
| 1165 | 1574 elif not self.db.getclass(link_class).hasnode(value): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1575 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1576 value)) |
| 1165 | 1577 |
| 1578 # save off the value | |
| 1579 propvalues[key] = value | |
| 1580 | |
| 1581 # register the link with the newly linked node | |
| 1582 if self.do_journal and self.properties[key].do_journal: | |
| 1583 self.db.addjournal(link_class, value, 'link', | |
| 1584 (self.classname, newid, key)) | |
| 1585 | |
| 1586 elif isinstance(prop, Multilink): | |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1587 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1588 value = [] |
|
3869
16faac822fe5
Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
1589 if not hasattr(value, '__iter__'): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1590 raise TypeError('new property "%s" not an iterable of ids'%key) |
| 1165 | 1591 # clean up and validate the list of links |
| 1592 link_class = self.properties[key].classname | |
| 1593 l = [] | |
| 1594 for entry in value: | |
| 1595 if type(entry) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1596 raise ValueError('"%s" multilink value (%r) ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1597 'must contain Strings'%(key, value)) |
| 1165 | 1598 # if it isn't a number, it's a key |
| 1599 if not num_re.match(entry): | |
| 1600 try: | |
| 1601 entry = self.db.classes[link_class].lookup(entry) | |
| 1602 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1603 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1604 key, entry, self.properties[key].classname)) |
| 1165 | 1605 l.append(entry) |
| 1606 value = l | |
| 1607 propvalues[key] = value | |
| 1608 | |
| 1609 # handle additions | |
| 1610 for nodeid in value: | |
| 1611 if not self.db.getclass(link_class).hasnode(nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1612 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1613 nodeid)) |
| 1165 | 1614 # register the link with the newly linked node |
| 1615 if self.do_journal and self.properties[key].do_journal: | |
| 1616 self.db.addjournal(link_class, nodeid, 'link', | |
| 1617 (self.classname, newid, key)) | |
| 1618 | |
| 1619 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1620 if type(value) != type('') and type(value) != type(u''): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1621 raise TypeError('new property "%s" not a string'%key) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1622 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1623 self.db.indexer.add_text((self.classname, newid, key), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1624 value) |
| 1165 | 1625 |
| 1626 elif isinstance(prop, Password): | |
| 1627 if not isinstance(value, password.Password): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1628 raise TypeError('new property "%s" not a Password'%key) |
| 1165 | 1629 |
| 1630 elif isinstance(prop, Date): | |
| 1631 if value is not None and not isinstance(value, date.Date): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1632 raise TypeError('new property "%s" not a Date'%key) |
| 1165 | 1633 |
| 1634 elif isinstance(prop, Interval): | |
| 1635 if value is not None and not isinstance(value, date.Interval): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1636 raise TypeError('new property "%s" not an Interval'%key) |
| 1165 | 1637 |
| 1638 elif value is not None and isinstance(prop, Number): | |
| 1639 try: | |
| 1640 float(value) | |
| 1641 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1642 raise TypeError('new property "%s" not numeric'%key) |
| 1165 | 1643 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1644 elif value is not None and isinstance(prop, Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1645 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1646 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1647 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1648 raise TypeError('new property "%s" not integer'%key) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1649 |
| 1165 | 1650 elif value is not None and isinstance(prop, Boolean): |
| 1651 try: | |
| 1652 int(value) | |
| 1653 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1654 raise TypeError('new property "%s" not boolean'%key) |
| 1165 | 1655 |
| 1656 # make sure there's data where there needs to be | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1657 for key, prop in self.properties.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1658 if key in propvalues: |
| 1165 | 1659 continue |
| 1660 if key == self.key: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1661 raise ValueError('key property "%s" is required'%key) |
| 1165 | 1662 if isinstance(prop, Multilink): |
| 1663 propvalues[key] = [] | |
| 1664 else: | |
| 1665 propvalues[key] = None | |
| 1666 | |
| 1667 # done | |
| 1668 self.db.addnode(self.classname, newid, propvalues) | |
| 1669 if self.do_journal: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1670 self.db.addjournal(self.classname, newid, ''"create", {}) |
| 1165 | 1671 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1672 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1673 return str(newid) |
| 1165 | 1674 |
| 1675 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1676 """Get the value of a property on an existing node of this class. |
| 1165 | 1677 |
| 1678 'nodeid' must be the id of an existing node of this class or an | |
| 1679 IndexError is raised. 'propname' must be the name of a property | |
| 1680 of this class or a KeyError is raised. | |
| 1681 | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1682 'cache' exists for backwards compatibility, and is not used. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1683 """ |
| 1165 | 1684 if propname == 'id': |
| 1685 return nodeid | |
| 1686 | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1687 # get the node's dict |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1688 d = self.db.getnode(self.classname, nodeid, fetch_multilinks=False) |
|
4476
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1689 # handle common case -- that property is in dict -- first |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1690 # if None and one of creator/creation actor/activity return None |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1691 if propname in d: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1692 r = d [propname] |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1693 # return copy of our list |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1694 if isinstance (r, list): |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1695 return r[:] |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1696 if r is not None: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1697 return r |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1698 elif propname in ('creation', 'activity', 'creator', 'actor'): |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1699 return r |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1700 |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1701 # propname not in d: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1702 if propname == 'creation' or propname == 'activity': |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1703 return date.Date() |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1704 if propname == 'creator' or propname == 'actor': |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1705 return self.db.getuid() |
| 1165 | 1706 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1707 # get the property (raises KeyError if invalid) |
| 1165 | 1708 prop = self.properties[propname] |
| 1709 | |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1710 # lazy evaluation of Multilink |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1711 if propname not in d and isinstance(prop, Multilink): |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1712 self.db._materialize_multilink(self.classname, nodeid, d, propname) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1713 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1714 # handle there being no value in the table for the property |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1715 if propname not in d or d[propname] is None: |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
1716 if default is _marker: |
| 1165 | 1717 if isinstance(prop, Multilink): |
| 1718 return [] | |
| 1719 else: | |
| 1720 return None | |
| 1721 else: | |
| 1722 return default | |
| 1723 | |
| 1724 # don't pass our list to other code | |
| 1725 if isinstance(prop, Multilink): | |
| 1726 return d[propname][:] | |
| 1727 | |
| 1728 return d[propname] | |
| 1729 | |
| 1730 def set(self, nodeid, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1731 """Modify a property on an existing node of this class. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1732 |
| 1165 | 1733 'nodeid' must be the id of an existing node of this class or an |
| 1734 IndexError is raised. | |
| 1735 | |
| 1736 Each key in 'propvalues' must be the name of a property of this | |
| 1737 class or a KeyError is raised. | |
| 1738 | |
| 1739 All values in 'propvalues' must be acceptable types for their | |
| 1740 corresponding properties or a TypeError is raised. | |
| 1741 | |
| 1742 If the value of the key property is set, it must not collide with | |
| 1743 other key strings or a ValueError is raised. | |
| 1744 | |
| 1745 If the value of a Link or Multilink property contains an invalid | |
| 1746 node id, a ValueError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1747 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1748 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1749 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1750 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1751 self.fireReactors('set', nodeid, oldvalues) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1752 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1753 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1754 def set_inner(self, nodeid, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1755 """ Called by set, in-between the audit and react calls. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1756 """ |
| 1165 | 1757 if not propvalues: |
| 1758 return propvalues | |
| 1759 | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1760 if ('creator' in propvalues or 'actor' in propvalues or |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1761 'creation' in propvalues or 'activity' in propvalues): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1762 raise KeyError('"creator", "actor", "creation" and ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1763 '"activity" are reserved') |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1764 |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1765 if 'id' in propvalues: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1766 raise KeyError('"id" is reserved') |
| 1165 | 1767 |
| 1768 if self.db.journaltag is None: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1769 raise DatabaseError(_('Database open read-only')) |
| 1165 | 1770 |
| 1771 node = self.db.getnode(self.classname, nodeid) | |
| 1772 if self.is_retired(nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1773 raise IndexError('Requested item is retired') |
| 1165 | 1774 num_re = re.compile('^\d+$') |
| 1775 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1776 # make a copy of the values dictionary - we'll modify the contents |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1777 propvalues = propvalues.copy() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1778 |
| 1165 | 1779 # if the journal value is to be different, store it in here |
| 1780 journalvalues = {} | |
| 1781 | |
| 1782 # remember the add/remove stuff for multilinks, making it easier | |
| 1783 # for the Database layer to do its stuff | |
| 1784 multilink_changes = {} | |
| 1785 | |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1786 # omit quiet properties from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1787 quiet_props = [] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1788 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1789 for propname, value in list(propvalues.items()): |
| 1165 | 1790 # check to make sure we're not duplicating an existing key |
| 1791 if propname == self.key and node[propname] != value: | |
| 1792 try: | |
| 1793 self.lookup(value) | |
| 1794 except KeyError: | |
| 1795 pass | |
| 1796 else: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1797 raise ValueError('node with key "%s" exists'%value) |
| 1165 | 1798 |
| 1799 # this will raise the KeyError if the property isn't valid | |
| 1800 # ... we don't use getprops() here because we only care about | |
| 1801 # the writeable properties. | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1802 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1803 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1804 except KeyError: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1805 raise KeyError('"%s" has no property named "%s"'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1806 self.classname, propname)) |
| 1165 | 1807 |
| 1808 # if the value's the same as the existing value, no sense in | |
| 1809 # doing anything | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1810 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1811 if value == current: |
| 1165 | 1812 del propvalues[propname] |
| 1813 continue | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1814 journalvalues[propname] = current |
| 1165 | 1815 |
| 1816 # do stuff based on the prop type | |
| 1817 if isinstance(prop, Link): | |
| 1818 link_class = prop.classname | |
| 1819 # if it isn't a number, it's a key | |
| 1820 if value is not None and not isinstance(value, type('')): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1821 raise ValueError('property "%s" link value be a string'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1822 propname)) |
| 1165 | 1823 if isinstance(value, type('')) and not num_re.match(value): |
| 1824 try: | |
| 1825 value = self.db.classes[link_class].lookup(value) | |
| 1826 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1827 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1828 propname, value, prop.classname)) |
| 1165 | 1829 |
| 1830 if (value is not None and | |
| 1831 not self.db.getclass(link_class).hasnode(value)): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1832 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1833 value)) |
| 1165 | 1834 |
| 1835 if self.do_journal and prop.do_journal: | |
| 1836 # register the unlink with the old linked node | |
| 1837 if node[propname] is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1838 self.db.addjournal(link_class, node[propname], |
|
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1839 ''"unlink", (self.classname, nodeid, propname)) |
| 1165 | 1840 |
| 1841 # register the link with the newly linked node | |
| 1842 if value is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1843 self.db.addjournal(link_class, value, ''"link", |
| 1165 | 1844 (self.classname, nodeid, propname)) |
| 1845 | |
| 1846 elif isinstance(prop, Multilink): | |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1847 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1848 value = [] |
|
3869
16faac822fe5
Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
1849 if not hasattr(value, '__iter__'): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1850 raise TypeError('new property "%s" not an iterable of' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1851 ' ids'%propname) |
| 1165 | 1852 link_class = self.properties[propname].classname |
| 1853 l = [] | |
| 1854 for entry in value: | |
| 1855 # if it isn't a number, it's a key | |
| 1856 if type(entry) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1857 raise ValueError('new property "%s" link value ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1858 'must be a string'%propname) |
| 1165 | 1859 if not num_re.match(entry): |
| 1860 try: | |
| 1861 entry = self.db.classes[link_class].lookup(entry) | |
| 1862 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1863 raise IndexError('new property "%s": %s not a %s'%( |
| 1165 | 1864 propname, entry, |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1865 self.properties[propname].classname)) |
| 1165 | 1866 l.append(entry) |
| 1867 value = l | |
| 1868 propvalues[propname] = value | |
| 1869 | |
| 1870 # figure the journal entry for this property | |
| 1871 add = [] | |
| 1872 remove = [] | |
| 1873 | |
| 1874 # handle removals | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1875 if propname in node: |
| 1165 | 1876 l = node[propname] |
| 1877 else: | |
| 1878 l = [] | |
| 1879 for id in l[:]: | |
| 1880 if id in value: | |
| 1881 continue | |
| 1882 # register the unlink with the old linked node | |
| 1883 if self.do_journal and self.properties[propname].do_journal: | |
| 1884 self.db.addjournal(link_class, id, 'unlink', | |
| 1885 (self.classname, nodeid, propname)) | |
| 1886 l.remove(id) | |
| 1887 remove.append(id) | |
| 1888 | |
| 1889 # handle additions | |
| 1890 for id in value: | |
|
4095
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1891 if id in l: |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1892 continue |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1893 # We can safely check this condition after |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1894 # checking that this is an addition to the |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1895 # multilink since the condition was checked for |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1896 # existing entries at the point they were added to |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1897 # the multilink. Since the hasnode call will |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1898 # result in a SQL query, it is more efficient to |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1899 # avoid the check if possible. |
| 1165 | 1900 if not self.db.getclass(link_class).hasnode(id): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1901 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1902 id)) |
| 1165 | 1903 # register the link with the newly linked node |
| 1904 if self.do_journal and self.properties[propname].do_journal: | |
| 1905 self.db.addjournal(link_class, id, 'link', | |
| 1906 (self.classname, nodeid, propname)) | |
| 1907 l.append(id) | |
| 1908 add.append(id) | |
| 1909 | |
| 1910 # figure the journal entry | |
| 1911 l = [] | |
| 1912 if add: | |
| 1913 l.append(('+', add)) | |
| 1914 if remove: | |
| 1915 l.append(('-', remove)) | |
| 1916 multilink_changes[propname] = (add, remove) | |
| 1917 if l: | |
| 1918 journalvalues[propname] = tuple(l) | |
| 1919 | |
| 1920 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1921 if value is not None and type(value) != type('') and type(value) != type(u''): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1922 raise TypeError('new property "%s" not a string'%propname) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1923 if prop.indexme: |
|
3399
3a0d4e4a0f34
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
1924 if value is None: value = '' |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1925 self.db.indexer.add_text((self.classname, nodeid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1926 value) |
| 1165 | 1927 |
| 1928 elif isinstance(prop, Password): | |
| 1929 if not isinstance(value, password.Password): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1930 raise TypeError('new property "%s" not a Password'%propname) |
| 1165 | 1931 propvalues[propname] = value |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1932 journalvalues[propname] = \ |
|
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1933 current and password.JournalPassword(current) |
| 1165 | 1934 |
| 1935 elif value is not None and isinstance(prop, Date): | |
| 1936 if not isinstance(value, date.Date): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1937 raise TypeError('new property "%s" not a Date'% propname) |
| 1165 | 1938 propvalues[propname] = value |
| 1939 | |
| 1940 elif value is not None and isinstance(prop, Interval): | |
| 1941 if not isinstance(value, date.Interval): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1942 raise TypeError('new property "%s" not an ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1943 'Interval'%propname) |
| 1165 | 1944 propvalues[propname] = value |
| 1945 | |
| 1946 elif value is not None and isinstance(prop, Number): | |
| 1947 try: | |
| 1948 float(value) | |
| 1949 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1950 raise TypeError('new property "%s" not numeric'%propname) |
| 1165 | 1951 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1952 elif value is not None and isinstance(prop, Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1953 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1954 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1955 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1956 raise TypeError('new property "%s" not integer'%propname) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1957 |
| 1165 | 1958 elif value is not None and isinstance(prop, Boolean): |
| 1959 try: | |
| 1960 int(value) | |
| 1961 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1962 raise TypeError('new property "%s" not boolean'%propname) |
| 1165 | 1963 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1964 # record quiet properties to omit from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1965 if prop.quiet: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1966 quiet_props.append(propname) |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1967 |
| 1165 | 1968 # nothing to do? |
| 1969 if not propvalues: | |
| 1970 return propvalues | |
| 1971 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1972 # update the activity time |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1973 propvalues['activity'] = date.Date() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1974 propvalues['actor'] = self.db.getuid() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1975 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1976 # do the set |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1977 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes) |
| 1165 | 1978 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1979 # remove the activity props now they're handled |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1980 del propvalues['activity'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1981 del propvalues['actor'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1982 |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1983 # journal the set |
| 1165 | 1984 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1985 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues) |
| 1165 | 1986 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1987 # remove quiet properties from output |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1988 for propname in quiet_props: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1989 if propname in propvalues: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1990 del propvalues[propname] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
1991 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1992 return propvalues |
| 1165 | 1993 |
| 1994 def retire(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1995 """Retire a node. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1996 |
| 1165 | 1997 The properties on the node remain available from the get() method, |
| 1998 and the node's id is never reused. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1999 |
| 1165 | 2000 Retired nodes are not returned by the find(), list(), or lookup() |
| 2001 methods, and other nodes may reuse the values of their key properties. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2002 """ |
| 1165 | 2003 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2004 raise DatabaseError(_('Database open read-only')) |
| 1165 | 2005 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2006 self.fireAuditors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2007 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2008 # use the arg for __retired__ to cope with any odd database type |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2009 # conversion (hello, sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2010 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2011 self.db.arg, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2012 self.db.sql(sql, (nodeid, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2013 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
2014 self.db.addjournal(self.classname, nodeid, ''"retired", None) |
| 1165 | 2015 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2016 self.fireReactors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2017 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2018 def restore(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2019 """Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2020 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2021 Make node available for all operations like it was before retirement. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2022 """ |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2023 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2024 raise DatabaseError(_('Database open read-only')) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2025 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2026 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2027 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2028 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2029 try: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2030 id = self.lookup(node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2031 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2032 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2033 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2034 raise KeyError("Key property (%s) of retired node clashes " |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2035 "with existing one (%s)" % (key, node[key])) |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2036 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2037 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2038 # use the arg for __retired__ to cope with any odd database type |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2039 # conversion (hello, sqlite) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2040 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2041 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2042 self.db.sql(sql, (0, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2043 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
2044 self.db.addjournal(self.classname, nodeid, ''"restored", None) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2045 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2046 self.fireReactors('restore', nodeid, None) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2047 |
| 1165 | 2048 def is_retired(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2049 """Return true if the node is rerired |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2050 """ |
| 1165 | 2051 sql = 'select __retired__ from _%s where id=%s'%(self.classname, |
| 2052 self.db.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2053 self.db.sql(sql, (nodeid,)) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2054 return int(self.db.sql_fetchone()[0]) > 0 |
| 1165 | 2055 |
| 2056 def destroy(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2057 """Destroy a node. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2058 |
| 1165 | 2059 WARNING: this method should never be used except in extremely rare |
| 2060 situations where there could never be links to the node being | |
| 2061 deleted | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2062 |
| 1165 | 2063 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2064 |
| 1165 | 2065 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2066 |
| 1165 | 2067 WARNING: really, use retire() instead |
| 2068 | |
| 2069 Well, I think that's enough warnings. This method exists mostly to | |
| 2070 support the session storage of the cgi interface. | |
| 2071 | |
| 2072 The node is completely removed from the hyperdb, including all journal | |
| 2073 entries. It will no longer be available, and will generally break code | |
| 2074 if there are any references to the node. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2075 """ |
| 1165 | 2076 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2077 raise DatabaseError(_('Database open read-only')) |
| 1165 | 2078 self.db.destroynode(self.classname, nodeid) |
| 2079 | |
| 2080 # Locating nodes: | |
| 2081 def hasnode(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2082 """Determine if the given nodeid actually exists |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2083 """ |
| 1165 | 2084 return self.db.hasnode(self.classname, nodeid) |
| 2085 | |
| 2086 def setkey(self, propname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2087 """Select a String property of this class to be the key property. |
| 1165 | 2088 |
| 2089 'propname' must be the name of a String property of this class or | |
| 2090 None, or a TypeError is raised. The values of the key property on | |
| 2091 all existing nodes must be unique or a ValueError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2092 """ |
| 1165 | 2093 prop = self.getprops()[propname] |
| 2094 if not isinstance(prop, String): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2095 raise TypeError('key properties must be String') |
| 1165 | 2096 self.key = propname |
| 2097 | |
| 2098 def getkey(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2099 """Return the name of the key property for this class or None.""" |
| 1165 | 2100 return self.key |
| 2101 | |
| 2102 def lookup(self, keyvalue): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2103 """Locate a particular node by its key property and return its id. |
| 1165 | 2104 |
| 2105 If this class has no key property, a TypeError is raised. If the | |
| 2106 'keyvalue' matches one of the values for the key property among | |
| 2107 the nodes in this class, the matching node's id is returned; | |
| 2108 otherwise a KeyError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2109 """ |
| 1165 | 2110 if not self.key: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2111 raise TypeError('No key property set for class %s'%self.classname) |
| 1165 | 2112 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2113 # use the arg to handle any odd database type conversion (hello, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2114 # sqlite) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2115 sql = "select id from _%s where _%s=%s and __retired__=%s"%( |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2116 self.classname, self.key, self.db.arg, self.db.arg) |
|
3977
732a37da3a10
Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479])
Richard Jones <richard@users.sourceforge.net>
parents:
3969
diff
changeset
|
2117 self.db.sql(sql, (str(keyvalue), 0)) |
| 1165 | 2118 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
2119 # see if there was a result that's not retired |
|
1203
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
2120 row = self.db.sql_fetchone() |
|
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
2121 if not row: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2122 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2123 keyvalue, self.classname)) |
| 1165 | 2124 |
| 2125 # return the id | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2126 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2127 return str(row[0]) |
| 1165 | 2128 |
| 2129 def find(self, **propspec): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2130 """Get the ids of nodes in this class which link to the given nodes. |
| 1165 | 2131 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2132 'propspec' consists of keyword args propname=nodeid or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2133 propname={nodeid:1, } |
| 1165 | 2134 'propname' must be the name of a property in this class, or a |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2135 KeyError is raised. That property must be a Link or |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2136 Multilink property, or a TypeError is raised. |
| 1165 | 2137 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2138 Any node in this class whose 'propname' property links to any of |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2139 the nodeids will be returned. Examples:: |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2140 |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2141 db.issue.find(messages='1') |
| 1165 | 2142 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2143 """ |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2144 # shortcut |
| 1165 | 2145 if not propspec: |
| 2146 return [] | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2147 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2148 # validate the args |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2149 props = self.getprops() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2150 for propname, nodeids in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2151 # check the prop is OK |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2152 prop = props[propname] |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2153 if not isinstance(prop, Link) and not isinstance(prop, Multilink): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2154 raise TypeError("'%s' not a Link/Multilink property"%propname) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2155 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2156 # first, links |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2157 a = self.db.arg |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2158 allvalues = () |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2159 sql = [] |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2160 where = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2161 for prop, values in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2162 if not isinstance(props[prop], hyperdb.Link): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2163 continue |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2164 if type(values) is type({}) and len(values) == 1: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2165 values = list(values)[0] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2166 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2167 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2168 where.append('_%s = %s'%(prop, a)) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
2169 elif values is None: |
|
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
2170 where.append('_%s is NULL'%prop) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2171 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2172 values = list(values) |
|
2494
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2173 s = '' |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2174 if None in values: |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2175 values.remove(None) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2176 s = '_%s is NULL or '%prop |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2177 allvalues += tuple(values) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2178 s += '_%s in (%s)'%(prop, ','.join([a]*len(values))) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2179 where.append('(' + s +')') |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2180 if where: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2181 allvalues = (0, ) + allvalues |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2182 sql.append("""select id from _%s where __retired__=%s |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2183 and %s"""%(self.classname, a, ' and '.join(where))) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2184 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2185 # now multilinks |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2186 for prop, values in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2187 if not isinstance(props[prop], hyperdb.Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2188 continue |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2189 if not values: |
|
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2190 continue |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2191 allvalues += (0, ) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2192 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2193 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2194 s = a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2195 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2196 allvalues += tuple(values) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2197 s = ','.join([a]*len(values)) |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2198 tn = '%s_%s'%(self.classname, prop) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2199 sql.append("""select id from _%s, %s where __retired__=%s |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2200 and id = %s.nodeid and %s.linkid in (%s)"""%(self.classname, |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2201 tn, a, tn, tn, s)) |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2202 |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2203 if not sql: |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2204 return [] |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2205 sql = ' union '.join(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
2206 self.db.sql(sql, allvalues) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2207 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2208 l = [str(x[0]) for x in self.db.sql_fetchall()] |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2209 return l |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2210 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2211 def stringFind(self, **requirements): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2212 """Locate a particular node by matching a set of its String |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2213 properties in a caseless search. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2214 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2215 If the property is not a String property, a TypeError is raised. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2216 |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2217 The return is a list of the id of all nodes that match. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2218 """ |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2219 where = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2220 args = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2221 for propname in requirements: |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2222 prop = self.properties[propname] |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
2223 if not isinstance(prop, String): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2224 raise TypeError("'%s' not a String property"%propname) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2225 where.append(propname) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2226 args.append(requirements[propname].lower()) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2227 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2228 # generate the where clause |
|
1549
a53a7e197360
fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents:
1539
diff
changeset
|
2229 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where]) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2230 sql = 'select id from _%s where %s and __retired__=%s'%( |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2231 self.classname, s, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2232 args.append(0) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2233 self.db.sql(sql, tuple(args)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2234 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2235 l = [str(x[0]) for x in self.db.sql_fetchall()] |
| 1165 | 2236 return l |
| 2237 | |
| 2238 def list(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2239 """ Return a list of the ids of the active nodes in this class. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2240 """ |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2241 return self.getnodeids(retired=0) |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2242 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2243 def getnodeids(self, retired=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2244 """ Retrieve all the ids of the nodes for a particular Class. |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2245 |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2246 Set retired=None to get all nodes. Otherwise it'll get all the |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2247 retired or non-retired nodes, depending on the flag. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2248 """ |
|
1707
3d627e34f18e
sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1599
diff
changeset
|
2249 # flip the sense of the 'retired' flag if we don't want all of them |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2250 if retired is not None: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2251 args = (0, ) |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2252 if retired: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2253 compare = '>' |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2254 else: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2255 compare = '=' |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2256 sql = 'select id from _%s where __retired__%s%s'%(self.classname, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2257 compare, self.db.arg) |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2258 else: |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2259 args = () |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2260 sql = 'select id from _%s'%self.classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2261 self.db.sql(sql, args) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2262 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2263 ids = [str(x[0]) for x in self.db.cursor.fetchall()] |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2264 return ids |
| 1165 | 2265 |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2266 def _subselect(self, classname, multilink_table): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2267 """Create a subselect. This is factored out because some |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2268 databases (hmm only one, so far) doesn't support subselects |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2269 look for "I can't believe it's not a toy RDBMS" in the mysql |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2270 backend. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2271 """ |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2272 return '_%s.id not in (select nodeid from %s)'%(classname, |
|
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2273 multilink_table) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2274 |
|
3685
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2275 # Some DBs order NULL values last. Set this variable in the backend |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2276 # for prepending an order by clause for each attribute that causes |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2277 # correct sort order for NULLs. Examples: |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2278 # order_by_null_values = '(%s is not NULL)' |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2279 # order_by_null_values = 'notnull(%s)' |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2280 # The format parameter is replaced with the attribute. |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2281 order_by_null_values = None |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2282 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2283 def supports_subselects(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2284 '''Assuming DBs can do subselects, overwrite if they cannot. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2285 ''' |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2286 return True |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2287 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2288 def _filter_multilink_expression_fallback( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2289 self, classname, multilink_table, expr): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2290 '''This is a fallback for database that do not support |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2291 subselects.''' |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2292 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2293 is_valid = expr.evaluate |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2294 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2295 last_id, kws = None, [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2296 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2297 ids = IdListOptimizer() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2298 append = ids.append |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2299 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2300 # This join and the evaluation in program space |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2301 # can be expensive for larger databases! |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2302 # TODO: Find a faster way to collect the data needed |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2303 # to evalute the expression. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2304 # Moving the expression evaluation into the database |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2305 # would be nice but this tricky: Think about the cases |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2306 # where the multilink table does not have join values |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2307 # needed in evaluation. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2308 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2309 stmnt = "SELECT c.id, m.linkid FROM _%s c " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2310 "LEFT OUTER JOIN %s m " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2311 "ON c.id = m.nodeid ORDER BY c.id" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2312 classname, multilink_table) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2313 self.db.sql(stmnt) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2314 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2315 # collect all multilink items for a class item |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2316 for nid, kw in self.db.sql_fetchiter(): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2317 if nid != last_id: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2318 if last_id is None: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2319 last_id = nid |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2320 else: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2321 # we have all multilink items -> evaluate! |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2322 if is_valid(kws): append(last_id) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2323 last_id, kws = nid, [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2324 if kw is not None: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2325 kws.append(kw) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2326 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2327 if last_id is not None and is_valid(kws): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2328 append(last_id) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2329 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2330 # we have ids of the classname table |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2331 return ids.where("_%s.id" % classname, self.db.arg) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2332 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2333 def _filter_multilink_expression(self, classname, multilink_table, v): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2334 """ Filters out elements of the classname table that do not |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2335 match the given expression. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2336 Returns tuple of 'WHERE' introns for the overall filter. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2337 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2338 try: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2339 opcodes = [int(x) for x in v] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2340 if min(opcodes) >= -1: raise ValueError() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2341 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2342 expr = compile_expression(opcodes) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2343 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2344 if not self.supports_subselects(): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2345 # We heavily rely on subselects. If there is |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2346 # no decent support fall back to slower variant. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2347 return self._filter_multilink_expression_fallback( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2348 classname, multilink_table, expr) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2349 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2350 atom = \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2351 "%s IN(SELECT linkid FROM %s WHERE nodeid=a.id)" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2352 self.db.arg, |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2353 multilink_table) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2354 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2355 intron = \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2356 "_%(classname)s.id in (SELECT id " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2357 "FROM _%(classname)s AS a WHERE %(condition)s) " % { |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2358 'classname' : classname, |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2359 'condition' : expr.generate(lambda n: atom) } |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2360 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2361 values = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2362 def collect_values(n): values.append(n.x) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2363 expr.visit(collect_values) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2364 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2365 return intron, values |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2366 except: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2367 # original behavior |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2368 where = "%s.linkid in (%s)" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2369 multilink_table, ','.join([self.db.arg] * len(v))) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2370 return where, v, True # True to indicate original |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2371 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2372 def _filter_sql (self, search_matches, filterspec, srt=[], grp=[], retr=0): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2373 """ Compute the proptree and the SQL/ARGS for a filter. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2374 For argument description see filter below. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2375 We return a 3-tuple, the proptree, the sql and the sql-args |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2376 or None if no SQL is necessary. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2377 The flag retr serves to retrieve *all* non-Multilink properties |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2378 (for filling the cache during a filter_iter) |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2379 """ |
|
2578
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2380 # we can't match anything if search_matches is empty |
| 4044 | 2381 if not search_matches and search_matches is not None: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2382 return None |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2383 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2384 icn = self.classname |
| 1165 | 2385 |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2386 # vars to hold the components of the SQL statement |
|
2586
3c67f4bfa225
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
2387 frum = [] # FROM clauses |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2388 loj = [] # LEFT OUTER JOIN clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2389 where = [] # WHERE clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2390 args = [] # *any* positional arguments |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2391 a = self.db.arg |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2392 |
| 1165 | 2393 # figure the WHERE clause from the filterspec |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2394 mlfilt = 0 # are we joining with Multilink tables? |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2395 sortattr = self._sortattr (group = grp, sort = srt) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2396 proptree = self._proptree(filterspec, sortattr, retr) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2397 mlseen = 0 |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2398 for pt in reversed(proptree.sortattr): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2399 p = pt |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2400 while p.parent: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2401 if isinstance (p.propclass, Multilink): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2402 mlseen = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2403 if mlseen: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2404 p.sort_ids_needed = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2405 p.tree_sort_done = False |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2406 p = p.parent |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2407 if not mlseen: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2408 pt.attr_sort_done = pt.tree_sort_done = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2409 proptree.compute_sort_done() |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2410 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2411 cols = ['_%s.id'%icn] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2412 mlsort = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2413 rhsnum = 0 |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2414 for p in proptree: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2415 rc = ac = oc = None |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2416 cn = p.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2417 ln = p.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2418 pln = p.parent.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2419 pcn = p.parent.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2420 k = p.name |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2421 v = p.val |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2422 propclass = p.propclass |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2423 if p.parent == proptree and p.name == 'id' \ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2424 and 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2425 p.sql_idx = 0 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2426 if 'sort' in p.need_for or 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2427 rc = oc = ac = '_%s._%s'%(pln, k) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2428 if isinstance(propclass, Multilink): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2429 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2430 mlfilt = 1 |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2431 tn = '%s_%s'%(pcn, k) |
|
4061
52431f2ad76b
Correct handling of a Multilink filterspec with an empty list of linked-to items.
Stefan Seefeld <stefan@seefeld.name>
parents:
4060
diff
changeset
|
2432 if v in ('-1', ['-1'], []): |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2433 # only match rows that have count(linkid)=0 in the |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2434 # corresponding multilink table) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2435 where.append(self._subselect(pcn, tn)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2436 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2437 frum.append(tn) |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2438 gen_join = True |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2439 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2440 if p.has_values and isinstance(v, type([])): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2441 result = self._filter_multilink_expression(pln, tn, v) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2442 # XXX: We dont need an id join if we used the filter |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2443 gen_join = len(result) == 3 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2444 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2445 if gen_join: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2446 where.append('_%s.id=%s.nodeid'%(pln,tn)) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2447 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2448 if p.children: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2449 frum.append('_%s as _%s' % (cn, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2450 where.append('%s.linkid=_%s.id'%(tn, ln)) |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2451 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2452 if p.has_values: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2453 if isinstance(v, type([])): |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2454 where.append(result[0]) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2455 args += result[1] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2456 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2457 where.append('%s.linkid=%s'%(tn, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2458 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2459 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2460 assert not p.attr_sort_done and not p.sort_ids_needed |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2461 elif k == 'id': |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2462 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2463 if isinstance(v, type([])): |
|
4462
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2464 # If there are no permitted values, then the |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2465 # where clause will always be false, and we |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2466 # can optimize the query away. |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2467 if not v: |
|
4803
5e679e364f9a
Bug-fix: need to return None for empty query
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4787
diff
changeset
|
2468 return None |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2469 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2470 where.append('_%s.%s in (%s)'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2471 args = args + v |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2472 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2473 where.append('_%s.%s=%s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2474 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2475 if 'sort' in p.need_for or 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2476 rc = oc = ac = '_%s.id'%pln |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2477 elif isinstance(propclass, String): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2478 if 'search' in p.need_for: |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2479 if not isinstance(v, type([])): |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2480 v = [v] |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2481 |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2482 # Quote special search characters '%' and '_' for |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2483 # correct matching with LIKE/ILIKE |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2484 # Note that we now pass the elements of v as query |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2485 # arguments and don't interpolate the quoted string |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2486 # into the sql statement. Should be safer. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2487 v = [self.db.search_stringquote(s) for s in v] |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2488 |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2489 # now add to the where clause |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2490 where.append('(' |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2491 +' and '.join(["_%s._%s %s %s ESCAPE %s"%( |
|
4787
4a017661e414
Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents:
4774
diff
changeset
|
2492 pln, |
|
4a017661e414
Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents:
4774
diff
changeset
|
2493 k, |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2494 self.case_insensitive_like, |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2495 a, |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2496 a) for s in v]) |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2497 +')') |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2498 for vv in v: |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2499 args.extend((vv, '\\')) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2500 if 'sort' in p.need_for: |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2501 oc = ac = 'lower(_%s._%s)'%(pln, k) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2502 elif isinstance(propclass, Link): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2503 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2504 if p.children: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2505 if 'sort' not in p.need_for: |
|
3694
595041d78104
Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3693
diff
changeset
|
2506 frum.append('_%s as _%s' % (cn, ln)) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2507 where.append('_%s._%s=_%s.id'%(pln, k, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2508 if p.has_values: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2509 if isinstance(v, type([])): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2510 d = {} |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2511 for entry in v: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2512 if entry == '-1': |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2513 entry = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2514 d[entry] = entry |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2515 l = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2516 if None in d or not d: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2517 if None in d: del d[None] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2518 l.append('_%s._%s is NULL'%(pln, k)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2519 if d: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2520 v = list(d) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2521 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2522 l.append('(_%s._%s in (%s))'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2523 args = args + v |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2524 if l: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2525 where.append('(' + ' or '.join(l) +')') |
|
3636
fa7becc62534
Of course for Links there is the same bug as I fixed before for multilinks.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3635
diff
changeset
|
2526 else: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2527 if v in ('-1', None): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2528 v = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2529 where.append('_%s._%s is NULL'%(pln, k)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2530 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2531 where.append('_%s._%s=%s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2532 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2533 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2534 lp = p.cls.labelprop() |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2535 oc = ac = '_%s._%s'%(pln, k) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2536 if lp != 'id': |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2537 if p.tree_sort_done: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2538 loj.append( |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2539 'LEFT OUTER JOIN _%s as _%s on _%s._%s=_%s.id'%( |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2540 cn, ln, pln, k, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2541 oc = '_%s._%s'%(ln, lp) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2542 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2543 rc = '_%s._%s'%(pln, k) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2544 elif isinstance(propclass, Date) and 'search' in p.need_for: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
2545 dc = self.db.to_sql_value(hyperdb.Date) |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2546 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2547 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2548 where.append('_%s._%s in (%s)'%(pln, k, s)) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2549 args = args + [dc(date.Date(x)) for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2550 else: |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2551 try: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2552 # Try to filter on range of dates |
|
3586
f47bddab5a49
date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents:
3554
diff
changeset
|
2553 date_rng = propclass.range_from_raw(v, self.db) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2554 if date_rng.from_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2555 where.append('_%s._%s >= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2556 args.append(dc(date_rng.from_value)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2557 if date_rng.to_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2558 where.append('_%s._%s <= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2559 args.append(dc(date_rng.to_value)) |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2560 except ValueError: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2561 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2562 pass |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2563 elif isinstance(propclass, Interval): |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2564 # filter/sort using the __<prop>_int__ column |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2565 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2566 if isinstance(v, type([])): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2567 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2568 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2569 args = args + [date.Interval(x).as_seconds() for x in v] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2570 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2571 try: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2572 # Try to filter on range of intervals |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2573 date_rng = Range(v, date.Interval) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2574 if date_rng.from_value: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2575 where.append('_%s.__%s_int__ >= %s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2576 args.append(date_rng.from_value.as_seconds()) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2577 if date_rng.to_value: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2578 where.append('_%s.__%s_int__ <= %s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2579 args.append(date_rng.to_value.as_seconds()) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2580 except ValueError: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2581 # If range creation fails - ignore search parameter |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2582 pass |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2583 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2584 oc = ac = '_%s.__%s_int__'%(pln,k) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2585 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2586 rc = '_%s._%s'%(pln,k) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2587 elif isinstance(propclass, Boolean) and 'search' in p.need_for: |
|
4365
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2588 if type(v) == type(""): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2589 v = v.split(',') |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2590 if type(v) != type([]): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2591 v = [v] |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2592 bv = [] |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2593 for val in v: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2594 if type(val) is type(''): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2595 bv.append(propclass.from_raw (val)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2596 else: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2597 bv.append(bool(val)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2598 if len(bv) == 1: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2599 where.append('_%s._%s=%s'%(pln, k, a)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2600 args = args + bv |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2601 else: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2602 s = ','.join([a for x in v]) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2603 where.append('_%s._%s in (%s)'%(pln, k, s)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2604 args = args + bv |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2605 elif 'search' in p.need_for: |
| 1165 | 2606 if isinstance(v, type([])): |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2607 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2608 where.append('_%s._%s in (%s)'%(pln, k, s)) |
| 1165 | 2609 args = args + v |
| 2610 else: | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2611 where.append('_%s._%s=%s'%(pln, k, a)) |
| 1165 | 2612 args.append(v) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2613 if oc: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2614 if p.sort_ids_needed: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2615 if rc == ac: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2616 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2617 p.auxcol = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2618 cols.append(ac) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2619 if p.tree_sort_done and p.sort_direction: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2620 # Don't select top-level id or multilink twice |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2621 if (not p.sort_ids_needed or ac != oc) and (p.name != 'id' |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2622 or p.parent != proptree): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2623 if rc == oc: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2624 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2625 cols.append(oc) |
|
3685
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2626 desc = ['', ' desc'][p.sort_direction == '-'] |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2627 # Some SQL dbs sort NULL values last -- we want them first. |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2628 if (self.order_by_null_values and p.name != 'id'): |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2629 nv = self.order_by_null_values % oc |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2630 cols.append(nv) |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2631 p.orderby.append(nv + desc) |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2632 p.orderby.append(oc + desc) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2633 if 'retrieve' in p.need_for and p.sql_idx is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2634 assert(rc) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2635 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2636 cols.append (rc) |
| 1165 | 2637 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2638 props = self.getprops() |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2639 |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2640 # don't match retired nodes |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2641 where.append('_%s.__retired__=0'%icn) |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2642 |
| 1165 | 2643 # add results of full text search |
| 2644 if search_matches is not None: | |
| 4044 | 2645 s = ','.join([a for x in search_matches]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2646 where.append('_%s.id in (%s)'%(icn, s)) |
| 4113 | 2647 args = args + [x for x in search_matches] |
| 1165 | 2648 |
| 2649 # construct the SQL | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2650 frum.append('_'+icn) |
| 1165 | 2651 frum = ','.join(frum) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2652 if where: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2653 where = ' where ' + (' and '.join(where)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2654 else: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2655 where = '' |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2656 if mlfilt: |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2657 # we're joining tables on the id, so we will get dupes if we |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2658 # don't distinct() |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2659 cols[0] = 'distinct(_%s.id)'%icn |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2660 |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2661 order = [] |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2662 # keep correct sequence of order attributes. |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2663 for sa in proptree.sortattr: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2664 if not sa.attr_sort_done: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2665 continue |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2666 order.extend(sa.orderby) |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2667 if order: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2668 order = ' order by %s'%(','.join(order)) |
| 1165 | 2669 else: |
| 2670 order = '' | |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2671 |
| 1165 | 2672 cols = ','.join(cols) |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2673 loj = ' '.join(loj) |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2674 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order) |
| 1165 | 2675 args = tuple(args) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2676 __traceback_info__ = (sql, args) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2677 return proptree, sql, args |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2678 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2679 def filter(self, search_matches, filterspec, sort=[], group=[]): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2680 """Return a list of the ids of the active nodes in this class that |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2681 match the 'filter' spec, sorted by the group spec and then the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2682 sort spec |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2683 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2684 "filterspec" is {propname: value(s)} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2685 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2686 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-' |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2687 or None and prop is a prop name or None. Note that for |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2688 backward-compatibility reasons a single (dir, prop) tuple is |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2689 also allowed. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2690 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2691 "search_matches" is a container type or None |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2692 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2693 The filter must match all properties specificed. If the property |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2694 value to match is a list: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2695 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2696 1. String properties must match all elements in the list, and |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2697 2. Other properties must match any of the elements in the list. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2698 """ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2699 if __debug__: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2700 start_t = time.time() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2701 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2702 sq = self._filter_sql (search_matches, filterspec, sort, group) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2703 # nothing to match? |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2704 if sq is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2705 return [] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2706 proptree, sql, args = sq |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2707 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2708 self.db.sql(sql, args) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
2709 l = self.db.sql_fetchall() |
| 1165 | 2710 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2711 # Compute values needed for sorting in proptree.sort |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2712 for p in proptree: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2713 if hasattr(p, 'auxcol'): |
|
4886
a116de39e38c
Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4849
diff
changeset
|
2714 p.sort_ids = [row[p.auxcol] for row in l] |
|
a116de39e38c
Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4849
diff
changeset
|
2715 p.sort_result = p._sort_repr (p.propclass.sort_repr, p.sort_ids) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2716 # return the IDs (the first column) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2717 # XXX numeric ids |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2718 l = [str(row[0]) for row in l] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2719 l = proptree.sort (l) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2720 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2721 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2722 self.db.stats['filtering'] += (time.time() - start_t) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2723 return l |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2724 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2725 def filter_iter(self, search_matches, filterspec, sort=[], group=[]): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2726 """Iterator similar to filter above with same args. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2727 Limitation: We don't sort on multilinks. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2728 This uses an optimisation: We put all nodes that are in the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2729 current row into the node cache. Then we return the node id. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2730 That way a fetch of a node won't create another sql-fetch (with |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2731 a join) from the database because the nodes are already in the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2732 cache. We're using our own temporary cursor. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2733 """ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2734 sq = self._filter_sql(search_matches, filterspec, sort, group, retr=1) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2735 # nothing to match? |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2736 if sq is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2737 return |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2738 proptree, sql, args = sq |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2739 cursor = self.db.conn.cursor() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2740 self.db.sql(sql, args, cursor) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2741 classes = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2742 for p in proptree: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2743 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2744 cn = p.parent.classname |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2745 ptid = p.parent.id # not the nodeid! |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2746 key = (cn, ptid) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2747 if key not in classes: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2748 classes[key] = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2749 name = p.name |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2750 assert (name) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2751 classes[key][name] = p |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
2752 p.to_hyperdb = self.db.to_hyperdb_value(p.propclass.__class__) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2753 while True: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2754 row = cursor.fetchone() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2755 if not row: break |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2756 # populate cache with current items |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2757 for (classname, ptid), pt in classes.iteritems(): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2758 nodeid = str(row[pt['id'].sql_idx]) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2759 key = (classname, nodeid) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2760 if key in self.db.cache: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2761 self.db._cache_refresh(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2762 continue |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2763 node = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2764 for propname, p in pt.iteritems(): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2765 value = row[p.sql_idx] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2766 if value is not None: |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
2767 value = p.to_hyperdb(value) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2768 node[propname] = value |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2769 self.db._cache_save(key, node) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2770 yield str(row[0]) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2771 |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2772 def filter_sql(self, sql): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2773 """Return a list of the ids of the items in this class that match |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2774 the SQL provided. The SQL is a complete "select" statement. |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2775 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2776 The SQL select must include the item id as the first column. |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2777 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2778 This function DOES NOT filter out retired items, add on a where |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2779 clause "__retired__=0" if you don't want retired nodes. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2780 """ |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2781 if __debug__: |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2782 start_t = time.time() |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2783 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2784 self.db.sql(sql) |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2785 l = self.db.sql_fetchall() |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2786 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2787 if __debug__: |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2788 self.db.stats['filtering'] += (time.time() - start_t) |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2789 return l |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2790 |
| 1165 | 2791 def count(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2792 """Get the number of nodes in this class. |
| 1165 | 2793 |
| 2794 If the returned integer is 'numnodes', the ids of all the nodes | |
| 2795 in this class run from 1 to numnodes, and numnodes+1 will be the | |
| 2796 id of the next node to be created in this class. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2797 """ |
| 1165 | 2798 return self.db.countnodes(self.classname) |
| 2799 | |
| 2800 # Manipulating properties: | |
| 2801 def getprops(self, protected=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2802 """Return a dictionary mapping property names to property objects. |
| 1165 | 2803 If the "protected" flag is true, we include protected properties - |
| 2804 those which may not be modified. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2805 """ |
| 1165 | 2806 d = self.properties.copy() |
| 2807 if protected: | |
| 2808 d['id'] = String() | |
| 2809 d['creation'] = hyperdb.Date() | |
| 2810 d['activity'] = hyperdb.Date() | |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1175
diff
changeset
|
2811 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2812 d['actor'] = hyperdb.Link('user') |
| 1165 | 2813 return d |
| 2814 | |
| 2815 def addprop(self, **properties): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2816 """Add properties to this class. |
| 1165 | 2817 |
| 2818 The keyword arguments in 'properties' must map names to property | |
| 2819 objects, or a TypeError is raised. None of the keys in 'properties' | |
| 2820 may collide with the names of existing properties, or a ValueError | |
| 2821 is raised before any properties have been added. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2822 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2823 for key in properties: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2824 if key in self.properties: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2825 raise ValueError(key) |
| 1165 | 2826 self.properties.update(properties) |
| 2827 | |
| 2828 def index(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2829 """Add (or refresh) the node to search indexes |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2830 """ |
| 1165 | 2831 # find all the String properties that have indexme |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2832 for prop, propclass in self.getprops().iteritems(): |
| 1165 | 2833 if isinstance(propclass, String) and propclass.indexme: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2834 self.db.indexer.add_text((self.classname, nodeid, prop), |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2835 str(self.get(nodeid, prop))) |
| 1165 | 2836 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2837 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2838 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2839 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2840 def export_list(self, propnames, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2841 """ Export a node - generate a list of CSV-able data in the order |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2842 specified by propnames for the given node. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2843 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2844 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2845 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2846 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2847 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2848 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2849 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2850 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2851 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2852 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2853 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2854 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2855 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2856 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2857 value = str(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2858 l.append(repr(value)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2859 l.append(repr(self.is_retired(nodeid))) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2860 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2861 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2862 def import_list(self, propnames, proplist): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2863 """ Import a node - all information including "id" is present and |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2864 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2865 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2866 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2867 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2868 Return the nodeid of the node imported. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2869 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2870 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2871 raise DatabaseError(_('Database open read-only')) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2872 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2873 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2874 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2875 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2876 retire = 0 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2877 if not "id" in propnames: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2878 newid = self.db.newid(self.classname) |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2879 else: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2880 newid = eval(proplist[propnames.index("id")]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2881 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2882 # Use eval to reverse the repr() used to output the CSV |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2883 value = eval(proplist[i]) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2884 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2885 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2886 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2887 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2888 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2889 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2890 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2891 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2892 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2893 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2894 retire = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2895 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2896 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2897 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2898 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2899 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2900 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2901 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2902 # don't set Nones |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2903 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2904 elif isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2905 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2906 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2907 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2908 elif isinstance(prop, hyperdb.Password): |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4476
diff
changeset
|
2909 value = password.Password(encrypted=value) |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2910 elif isinstance(prop, String): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2911 if isinstance(value, unicode): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2912 value = value.encode('utf8') |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2913 if not isinstance(value, str): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2914 raise TypeError('new property "%(propname)s" not a ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2915 'string: %(value)r'%locals()) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2916 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2917 self.db.indexer.add_text((self.classname, newid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2918 value) |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2919 d[propname] = value |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2920 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2921 # get a new id if necessary |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2922 if newid is None: |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2923 newid = self.db.newid(self.classname) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2924 |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2925 # insert new node or update existing? |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2926 if not self.hasnode(newid): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2927 self.db.addnode(self.classname, newid, d) # insert |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2928 else: |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2929 self.db.setnode(self.classname, newid, d) # update |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2930 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2931 # retire? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2932 if retire: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2933 # use the arg for __retired__ to cope with any odd database type |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2934 # conversion (hello, sqlite) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2935 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2936 self.db.arg, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2937 self.db.sql(sql, (newid, newid)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2938 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2939 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2940 def export_journals(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2941 """Export a class's journal - generate a list of lists of |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2942 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2943 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2944 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2945 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2946 No heading here - the columns are fixed. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2947 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2948 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2949 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2950 for nodeid in self.getnodeids(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2951 for nodeid, date, user, action, params in self.history(nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2952 date = date.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2953 if action == 'set': |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2954 export_data = {} |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2955 for propname, value in params.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2956 if propname not in properties: |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2957 # property no longer in the schema |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2958 continue |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2959 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2960 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2961 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2962 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2963 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2964 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2965 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2966 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2967 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2968 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2969 value = str(value) |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2970 export_data[propname] = value |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2971 params = export_data |
|
3933
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2972 elif action == 'create' and params: |
|
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2973 # old tracker with data stored in the create! |
|
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2974 params = {} |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2975 l = [nodeid, date, user, action, params] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2976 r.append(list(map(repr, l))) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2977 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2978 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2586
diff
changeset
|
2979 class FileClass(hyperdb.FileClass, Class): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2980 """This class defines a large chunk of data. To support this, it has a |
| 1165 | 2981 mandatory String property "content" which is typically saved off |
| 2982 externally to the hyperdb. | |
| 2983 | |
| 2984 The default MIME type of this data is defined by the | |
| 2985 "default_mime_type" class attribute, which may be overridden by each | |
| 2986 node if the class defines a "type" String property. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2987 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2988 def __init__(self, db, classname, **properties): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2989 """The newly-created class automatically includes the "content" |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2990 and "type" properties. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2991 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2992 if 'content' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2993 properties['content'] = hyperdb.String(indexme='yes') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2994 if 'type' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2995 properties['type'] = hyperdb.String() |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2996 Class.__init__(self, db, classname, **properties) |
| 1165 | 2997 |
| 2998 def create(self, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2999 """ snaffle the file propvalue and store in a file |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3000 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3001 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3002 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3003 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3004 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3005 # now remove the content property so it's not stored in the db |
| 1165 | 3006 content = propvalues['content'] |
| 3007 del propvalues['content'] | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3008 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3009 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3010 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3011 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3012 # figure the mime type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3013 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3014 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3015 # and index! |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3016 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3017 self.db.indexer.add_text((self.classname, newid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3018 content, mime_type) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3019 |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3020 # store off the content as a file |
|
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3021 self.db.storefile(self.classname, newid, None, content) |
|
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3022 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3023 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3024 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3025 |
| 1165 | 3026 return newid |
| 3027 | |
| 3028 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3029 """ Trap the content propname and get it from the file |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3030 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3031 'cache' exists for backwards compatibility, and is not used. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3032 """ |
| 1165 | 3033 poss_msg = 'Possibly a access right configuration problem.' |
| 3034 if propname == 'content': | |
| 3035 try: | |
| 3036 return self.db.getfile(self.classname, nodeid, None) | |
|
4410
de00a238a9ad
remove unnecessary brackets
Richard Jones <richard@users.sourceforge.net>
parents:
4365
diff
changeset
|
3037 except IOError, strerror: |
| 1165 | 3038 # BUG: by catching this we donot see an error in the log. |
| 3039 return 'ERROR reading file: %s%s\n%s\n%s'%( | |
| 3040 self.classname, nodeid, poss_msg, strerror) | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
3041 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3042 return Class.get(self, nodeid, propname, default) |
| 1165 | 3043 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3044 return Class.get(self, nodeid, propname) |
| 1165 | 3045 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3046 def set(self, itemid, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3047 """ Snarf the "content" propvalue and update it in a file |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3048 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3049 self.fireAuditors('set', itemid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3050 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
| 1165 | 3051 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3052 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3053 content = None |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3054 if 'content' in propvalues: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3055 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3056 del propvalues['content'] |
| 1165 | 3057 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3058 # do the database create |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3059 propvalues = self.set_inner(itemid, **propvalues) |
| 1165 | 3060 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3061 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3062 if content: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3063 # store and possibly index |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3064 self.db.storefile(self.classname, itemid, None, content) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3065 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3066 mime_type = self.get(itemid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3067 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3068 content, mime_type) |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
3069 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
3070 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3071 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3072 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3073 return propvalues |
| 1165 | 3074 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
3075 def index(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3076 """ Add (or refresh) the node to search indexes. |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3077 |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3078 Use the content-type property for the content property. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3079 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3080 # find all the String properties that have indexme |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3081 for prop, propclass in self.getprops().iteritems(): |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3082 if prop == 'content' and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3083 mime_type = self.get(nodeid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3084 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3085 str(self.get(nodeid, 'content')), mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3086 elif isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3087 # index them under (classname, nodeid, property) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3088 try: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3089 value = str(self.get(nodeid, prop)) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3090 except IndexError: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3091 # node has been destroyed |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3092 continue |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3093 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
3094 |
| 1165 | 3095 # XXX deviation from spec - was called ItemClass |
| 3096 class IssueClass(Class, roundupdb.IssueClass): | |
| 3097 # Overridden methods: | |
| 3098 def __init__(self, db, classname, **properties): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3099 """The newly-created class automatically includes the "messages", |
| 1165 | 3100 "files", "nosy", and "superseder" properties. If the 'properties' |
| 3101 dictionary attempts to specify any of these properties or a | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
3102 "creation", "creator", "activity" or "actor" property, a ValueError |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
3103 is raised. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3104 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3105 if 'title' not in properties: |
| 1165 | 3106 properties['title'] = hyperdb.String(indexme='yes') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3107 if 'messages' not in properties: |
| 1165 | 3108 properties['messages'] = hyperdb.Multilink("msg") |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3109 if 'files' not in properties: |
| 1165 | 3110 properties['files'] = hyperdb.Multilink("file") |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3111 if 'nosy' not in properties: |
| 1165 | 3112 # note: journalling is turned off as it really just wastes |
| 3113 # space. this behaviour may be overridden in an instance | |
| 3114 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3115 if 'superseder' not in properties: |
| 1165 | 3116 properties['superseder'] = hyperdb.Multilink(classname) |
| 3117 Class.__init__(self, db, classname, **properties) | |
| 3118 | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
3119 # vim: set et sts=4 sw=4 : |