Mercurial > p > roundup > code
annotate roundup/backends/rdbms_common.py @ 5096:e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
Also adds new config.ini setting called indexer to select
indexer. See ``doc/upgrading.txt`` for details. Initial patch
done by David Wolever. Patch modified (see ticket or below for
changes), docs updated and committed.
I have an outstanding issue with test/test_indexer.py. I have to
comment out all imports and tests for indexers I don't have (i.e.
mysql, postgres) otherwise no tests run.
With that change made, dbm, sqlite (rdbms), xapian and whoosh indexes
are all passing the indexer tests.
Changes summary:
1) support native back ends dbm and rdbms. (original patch only fell
through to dbm)
2) Developed whoosh stopfilter to not index stopwords or words outside
the the maxlength and minlength limits defined in index_common.py.
Required to pass the extremewords test_indexer test. Also I
removed a call to .lower on the input text as the tokenizer I chose
automatically does the lowercase.
3) Added support for max/min length to find. This was needed to pass
extremewords test.
4) Added back a call to save_index in add_text. This allowed all but
two tests to pass.
5) Fixed a call to:
results = searcher.search(query.Term("identifier", identifier))
which had an extra parameter that is an error under current whoosh.
6) Set limit=None in search call for find() otherwise it only return
10 items. This allowed it to pass manyresults test
Also due to changes in the roundup code removed the call in
indexer_whoosh to
from roundup.anypy.sets_ import set
since we use the python builtin set.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Sat, 25 Jun 2016 20:10:03 -0400 |
| parents | 2b551b6b0830 |
| children | 8901cc4ef0e0 |
| rev | line source |
|---|---|
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
1 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
6 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
11 # |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
17 # |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
18 """ Relational database (SQL) backend common code. |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
19 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
20 Basics: |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
21 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
22 - map roundup classes to relational tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
23 - automatically detect schema changes and modify the table schemas |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
24 appropriately (we store the "database version" of the schema in the |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
25 database itself as the only row of the "schema" table) |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
26 - multilinks (which represent a many-to-many relationship) are handled through |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
27 intermediate tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
28 - journals are stored adjunct to the per-class tables |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
29 - table names and columns have "_" prepended so the names can't clash with |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
30 restricted names (like "order") |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
31 - retirement is determined by the __retired__ column being > 0 |
|
1244
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
32 |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
33 Database-specific changes may generally be pushed out to the overridable |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
34 sql_* methods, since everything else should be fairly generic. There's |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
35 probably a bit of work to be done if a database is used that actually |
|
8dd4f736370b
merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents:
1222
diff
changeset
|
36 honors column typing, since the initial databases don't (sqlite stores |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
37 everything as a string.) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
38 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
39 The schema of the hyperdb being mapped to the database is stored in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
40 database itself as a repr()'ed dictionary of information about each Class |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
41 that maps to a table. If that information differs from the hyperdb schema, |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
42 then we update it. We also store in the schema dict a version which |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
43 allows us to upgrade the database schema when necessary. See upgrade_db(). |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
44 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
45 To force a unqiueness constraint on the key properties we put the item |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
46 id into the __retired__ column duing retirement (so it's 0 for "active" |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
47 items) and place a unqiueness constraint on key + __retired__. This is |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
48 particularly important for the users class where multiple users may |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
49 try to have the same username, with potentially many retired users with |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
50 the same name. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
51 """ |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
52 __docformat__ = 'restructuredtext' |
| 1165 | 53 |
| 54 # standard python modules | |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
55 import sys, os, time, re, errno, weakref, copy, logging, datetime |
| 1165 | 56 |
| 57 # roundup modules | |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
58 from roundup import hyperdb, date, password, roundupdb, security, support |
| 1165 | 59 from roundup.hyperdb import String, Password, Date, Interval, Link, \ |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
60 Multilink, DatabaseError, Boolean, Number, Integer, Node |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
61 from roundup.backends import locking |
|
3925
603ec9630b08
i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3924
diff
changeset
|
62 from roundup.i18n import _ |
| 1165 | 63 |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
64 |
| 1165 | 65 # support |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
66 from roundup.backends.blobfiles import FileStorage |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5086
diff
changeset
|
67 from roundup.backends.indexer_common import get_indexer |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
68 from roundup.backends.sessions_rdbms import Sessions, OneTimeKeys |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
69 from roundup.date import Range |
| 1165 | 70 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
71 from roundup.backends.back_anydbm import compile_expression |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
72 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
73 |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
74 # dummy value meaning "argument not passed" |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
75 _marker = [] |
|
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
76 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
77 def _num_cvt(num): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
78 num = str(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
79 try: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
80 return int(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
81 except: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
82 return float(num) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
83 |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
84 def _bool_cvt(value): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
85 if value in ('TRUE', 'FALSE'): |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
86 return {'TRUE': 1, 'FALSE': 0}[value] |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
87 # assume it's a number returned from the db API |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
88 return int(value) |
|
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
89 |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
90 def date_to_hyperdb_value(d): |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
91 """ convert date d to a roundup date """ |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
92 if isinstance (d, datetime.datetime): |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
93 return date.Date(d) |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
94 return date.Date (str(d).replace(' ', '.')) |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
95 |
|
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
96 |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
97 def connection_dict(config, dbnamestr=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
98 """ Used by Postgresql and MySQL to detemine the keyword args for |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
99 opening the database connection.""" |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
100 d = { } |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
101 if dbnamestr: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
102 d[dbnamestr] = config.RDBMS_NAME |
|
3099
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
103 for name in ('host', 'port', 'password', 'user', 'read_default_group', |
|
519b92df37dc
handle ~/.my.cnf files for MySQL defaults [SF#1096031]
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
104 'read_default_file'): |
|
2634
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
105 cvar = 'RDBMS_'+name.upper() |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
106 if config[cvar] is not None: |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
107 d[name] = config[cvar] |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
108 return d |
|
f47ca4541770
Both RDBMS backends now use the same config.ini section, [rdbms].
Richard Jones <richard@users.sourceforge.net>
parents:
2597
diff
changeset
|
109 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
110 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
111 class IdListOptimizer: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
112 """ To prevent flooding the SQL parser of the underlaying |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
113 db engine with "x IN (1, 2, 3, ..., <large number>)" collapses |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
114 these cases to "x BETWEEN 1 AND <large number>". |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
115 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
116 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
117 def __init__(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
118 self.ranges = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
119 self.singles = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
120 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
121 def append(self, nid): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
122 """ Invariant: nid are ordered ascending """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
123 if self.ranges: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
124 last = self.ranges[-1] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
125 if last[1] == nid-1: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
126 last[1] = nid |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
127 return |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
128 if self.singles: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
129 last = self.singles[-1] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
130 if last == nid-1: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
131 self.singles.pop() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
132 self.ranges.append([last, nid]) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
133 return |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
134 self.singles.append(nid) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
135 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
136 def where(self, field, placeholder): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
137 ranges = self.ranges |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
138 singles = self.singles |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
139 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
140 if not singles and not ranges: return "(1=0)", [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
141 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
142 if ranges: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
143 between = '%s BETWEEN %s AND %s' % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
144 field, placeholder, placeholder) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
145 stmnt = [between] * len(ranges) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
146 else: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
147 stmnt = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
148 if singles: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
149 stmnt.append('%s in (%s)' % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
150 field, ','.join([placeholder]*len(singles)))) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
151 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
152 return '(%s)' % ' OR '.join(stmnt), sum(ranges, []) + singles |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
153 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
154 def __str__(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
155 return "ranges: %r / singles: %r" % (self.ranges, self.singles) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
156 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
157 |
| 1165 | 158 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
159 """ Wrapper around an SQL database that presents a hyperdb interface. |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
160 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
161 - some functionality is specific to the actual SQL database, hence |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
162 the sql_* methods that are NotImplemented |
|
4074
e039f3cbbb96
Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents:
4062
diff
changeset
|
163 - we keep a cache of the latest N row fetches (where N is configurable). |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
164 """ |
| 1165 | 165 def __init__(self, config, journaltag=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
166 """ Open the database and load the schema from it. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
167 """ |
|
3897
e2c2d91932ad
respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3872
diff
changeset
|
168 FileStorage.__init__(self, config.UMASK) |
| 1165 | 169 self.config, self.journaltag = config, journaltag |
| 170 self.dir = config.DATABASE | |
| 171 self.classes = {} | |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5086
diff
changeset
|
172 self.indexer = get_indexer(config, self) |
| 1165 | 173 self.security = security.Security(self) |
| 174 | |
| 175 # additional transaction support for external files and the like | |
| 176 self.transactions = [] | |
| 177 | |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
178 # keep a cache of the N most recently retrieved rows of any kind |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
179 # (classname, nodeid) = row |
|
4074
e039f3cbbb96
Make RDBMS cache-size configurable.
Stefan Seefeld <stefan@seefeld.name>
parents:
4062
diff
changeset
|
180 self.cache_size = config.RDBMS_CACHE_SIZE |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
181 self.clearCache() |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
182 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
183 'filtering': 0} |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
184 |
|
5041
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
185 # make sure the database directory exists |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
186 if not os.path.isdir(self.config.DATABASE): |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
187 os.makedirs(self.config.DATABASE) |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
188 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
189 # database lock |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
190 self.lockfile = None |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1304
diff
changeset
|
191 |
| 1165 | 192 # open a connection to the database, creating the "conn" attribute |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
193 self.open_connection() |
| 1165 | 194 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
195 def clearCache(self): |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
196 self.cache = {} |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
197 self.cache_lru = [] |
|
4652
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4517
diff
changeset
|
198 # upcall is necessary! |
|
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4517
diff
changeset
|
199 roundupdb.Database.clearCache(self) |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
200 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
201 def getSessionManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
202 return Sessions(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
203 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
204 def getOTKManager(self): |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
205 return OneTimeKeys(self) |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
206 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2081
diff
changeset
|
207 def open_connection(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
208 """ Open a connection to the database, creating it if necessary. |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
209 |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
210 Must call self.load_dbschema() |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
211 """ |
| 1165 | 212 raise NotImplemented |
| 213 | |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
214 def sql(self, sql, args=None, cursor=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
215 """ Execute the sql with the optional args. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
216 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
217 self.log_debug('SQL %r %r'%(sql, args)) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
218 if not cursor: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
219 cursor = self.cursor |
| 1165 | 220 if args: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
221 cursor.execute(sql, args) |
| 1165 | 222 else: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
223 cursor.execute(sql) |
| 1165 | 224 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
225 def sql_fetchone(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
226 """ Fetch a single row. If there's nothing to fetch, return None. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
227 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
228 return self.cursor.fetchone() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
229 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
230 def sql_fetchall(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
231 """ Fetch all rows. If there's nothing to fetch, return []. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
232 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
233 return self.cursor.fetchall() |
| 1165 | 234 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
235 def sql_fetchiter(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
236 """ Fetch all row as a generator |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
237 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
238 while True: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
239 row = self.cursor.fetchone() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
240 if not row: break |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
241 yield row |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
242 |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
243 def search_stringquote(self, value): |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
244 """ Quote a search string to escape magic search characters |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
245 '%' and '_', also need to quote '\' (first) |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
246 Then put '%' around resulting string for LIKE (or ILIKE) operator |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
247 """ |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
248 v = value.replace('\\', '\\\\') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
249 v = v.replace('%', '\\%') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
250 v = v.replace('_', '\\_') |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
251 return '%' + v + '%' |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
252 |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
253 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
254 def init_dbschema(self): |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
255 self.database_schema = { |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
256 'version': self.current_db_version, |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
257 'tables': {} |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
258 } |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
259 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
260 def load_dbschema(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
261 """ Load the schema definition that the database currently implements |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
262 """ |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
263 self.cursor.execute('select schema from schema') |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
264 schema = self.cursor.fetchone() |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
265 if schema: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
266 self.database_schema = eval(schema[0]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
267 else: |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
268 self.database_schema = {} |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
269 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
270 def save_dbschema(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
271 """ Save the schema definition that the database currently implements |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
272 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
273 s = repr(self.database_schema) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
274 self.sql('delete from schema') |
|
3718
0d561b24ceff
support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents:
3694
diff
changeset
|
275 self.sql('insert into schema values (%s)'%self.arg, (s,)) |
| 1165 | 276 |
| 277 def post_init(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
278 """ Called once the schema initialisation has finished. |
| 1165 | 279 |
| 280 We should now confirm that the schema defined by our "classes" | |
| 281 attribute actually matches the schema in the database. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
282 """ |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
283 |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
284 # upgrade the database for column type changes, new internal |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
285 # tables, etc. |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
286 save = self.upgrade_db() |
| 3553 | 287 |
| 288 # handle changes in the schema | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
289 tables = self.database_schema['tables'] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
290 for classname, spec in self.classes.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
291 if classname in tables: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
292 dbspec = tables[classname] |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
293 if self.update_class(spec, dbspec): |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
294 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
295 save = 1 |
| 1165 | 296 else: |
| 297 self.create_class(spec) | |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
298 tables[classname] = spec.schema() |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
299 save = 1 |
| 1165 | 300 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
301 for classname, spec in list(tables.items()): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
302 if classname not in self.classes: |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
303 self.drop_class(classname, tables[classname]) |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
304 del tables[classname] |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
305 save = 1 |
| 1165 | 306 |
| 307 # update the database version of the schema | |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
308 if save: |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
309 self.save_dbschema() |
| 1165 | 310 |
| 311 # reindex the db if necessary | |
| 312 if self.indexer.should_reindex(): | |
| 313 self.reindex() | |
| 314 | |
| 315 # commit | |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
316 self.sql_commit() |
| 1165 | 317 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
318 # update this number when we need to make changes to the SQL structure |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
319 # of the backen database |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
320 current_db_version = 5 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
321 db_version_updated = False |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
322 def upgrade_db(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
323 """ Update the SQL database to reflect changes in the backend code. |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
324 |
|
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
325 Return boolean whether we need to save the schema. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
326 """ |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
327 version = self.database_schema.get('version', 1) |
|
3993
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
328 if version > self.current_db_version: |
|
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
329 raise DatabaseError('attempting to run rev %d DATABASE with rev ' |
|
1eba65a4b0aa
Don't run old code on newer database
Richard Jones <richard@users.sourceforge.net>
parents:
3979
diff
changeset
|
330 '%d CODE!'%(version, self.current_db_version)) |
|
2081
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
331 if version == self.current_db_version: |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
332 # nothing to do |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
333 return 0 |
|
fb4bf55b94d7
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
334 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
335 if version < 2: |
| 4062 | 336 self.log_info('upgrade to version 2') |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
337 # change the schema structure |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
338 self.database_schema = {'tables': self.database_schema} |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
339 |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
340 # version 1 didn't have the actor column (note that in |
|
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
341 # MySQL this will also transition the tables to typed columns) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
342 self.add_new_columns_v2() |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
343 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
344 # version 1 doesn't have the OTK, session and indexing in the |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
345 # database |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
346 self.create_version_2_tables() |
|
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
347 |
|
2429
e28a5d5d95a7
better upgrading
Richard Jones <richard@users.sourceforge.net>
parents:
2424
diff
changeset
|
348 if version < 3: |
| 4062 | 349 self.log_info('upgrade to version 3') |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
350 self.fix_version_2_tables() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
351 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
352 if version < 4: |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
353 self.log_info('upgrade to version 4') |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
354 self.fix_version_3_tables() |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
355 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
356 if version < 5: |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
357 self.log_info('upgrade to version 5') |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
358 self.fix_version_4_tables() |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
359 |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
360 self.database_schema['version'] = self.current_db_version |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
361 self.db_version_updated = True |
|
2075
b1704ba7be41
make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents:
2073
diff
changeset
|
362 return 1 |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
363 |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
364 def fix_version_3_tables(self): |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
365 # drop the shorter VARCHAR OTK column and add a new TEXT one |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
366 for name in ('otk', 'session'): |
|
2755
e3cd28cec23d
Clean out sessions / otks tables when migrating
Richard Jones <richard@users.sourceforge.net>
parents:
2733
diff
changeset
|
367 self.sql('DELETE FROM %ss'%name) |
|
2721
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
368 self.sql('ALTER TABLE %ss DROP %s_value'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
369 self.sql('ALTER TABLE %ss ADD %s_value TEXT'%(name, name)) |
|
1cd01cf106e1
extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents:
2720
diff
changeset
|
370 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
371 def fix_version_2_tables(self): |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
372 # Default (used by sqlite): NOOP |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
373 pass |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
374 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
375 def fix_version_4_tables(self): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
376 # note this is an explicit call now |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
377 c = self.cursor |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
378 for cn, klass in self.classes.iteritems(): |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
379 c.execute('select id from _%s where __retired__<>0'%(cn,)) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
380 for (id,) in c.fetchall(): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
381 c.execute('update _%s set __retired__=%s where id=%s'%(cn, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
382 self.arg, self.arg), (id, id)) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
383 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
384 if klass.key: |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
385 self.add_class_key_required_unique_constraint(cn, klass.key) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
386 |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
387 def _convert_journal_tables(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
388 """Get current journal table contents, drop the table and re-create""" |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
389 c = self.cursor |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
390 cols = ','.join('nodeid date tag action params'.split()) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
391 for klass in self.classes.itervalues(): |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
392 # slurp and drop |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
393 sql = 'select %s from %s__journal order by date'%(cols, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
394 klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
395 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
396 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
397 self.drop_journal_table_indexes(klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
398 c.execute('drop table %s__journal'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
399 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
400 # re-create and re-populate |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
401 self.create_journal_table(klass) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
402 a = self.arg |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
403 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
404 klass.classname, cols, a, a, a, a, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
405 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
406 # no data conversion needed |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
407 self.cursor.execute(sql, row) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
408 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
409 def _convert_string_properties(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
410 """Get current Class tables that contain String properties, and |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
411 convert the VARCHAR columns to TEXT""" |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
412 c = self.cursor |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
413 for klass in self.classes.itervalues(): |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
414 # slurp and drop |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
415 cols, mls = self.determine_columns(list(klass.properties.iteritems())) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
416 scols = ','.join([i[0] for i in cols]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
417 sql = 'select id,%s from _%s'%(scols, klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
418 c.execute(sql) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
419 contents = c.fetchall() |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
420 self.drop_class_table_indexes(klass.classname, klass.getkey()) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
421 c.execute('drop table _%s'%klass.classname) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
422 |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
423 # re-create and re-populate |
|
2424
74474ec41050
argh! backwards compat
Richard Jones <richard@users.sourceforge.net>
parents:
2416
diff
changeset
|
424 self.create_class_table(klass, create_sequence=0) |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
425 a = ','.join([self.arg for i in range(len(cols)+1)]) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
426 sql = 'insert into _%s (id,%s) values (%s)'%(klass.classname, |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
427 scols, a) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
428 for row in contents: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
429 l = [] |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
430 for entry in row: |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
431 # mysql will already be a string - psql needs "help" |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
432 if entry is not None and not isinstance(entry, type('')): |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
433 entry = str(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
434 l.append(entry) |
|
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
435 self.cursor.execute(sql, l) |
|
2073
261c2e6ceb1e
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
436 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
437 def refresh_database(self): |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
438 self.post_init() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
439 |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
440 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
441 def reindex(self, classname=None, show_progress=False): |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
442 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
443 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
444 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
445 classes = list(self.classes.itervalues()) |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
446 for klass in classes: |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
447 if show_progress: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
448 for nodeid in support.Progress('Reindex %s'%klass.classname, |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
449 klass.list()): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
450 klass.index(nodeid) |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
451 else: |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
452 for nodeid in klass.list(): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
453 klass.index(nodeid) |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
454 self.indexer.save_index() |
| 1165 | 455 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
456 hyperdb_to_sql_datatypes = { |
|
2413
7d0bb6601809
fix some column datatypes in postgresql and mysql
Richard Jones <richard@users.sourceforge.net>
parents:
2379
diff
changeset
|
457 hyperdb.String : 'TEXT', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
458 hyperdb.Date : 'TIMESTAMP', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
459 hyperdb.Link : 'INTEGER', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
460 hyperdb.Interval : 'VARCHAR(255)', |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
461 hyperdb.Password : 'VARCHAR(255)', |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
462 hyperdb.Boolean : 'BOOLEAN', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
463 hyperdb.Number : 'REAL', |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
464 hyperdb.Integer : 'INTEGER', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
465 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
466 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
467 def hyperdb_to_sql_datatype(self, propclass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
468 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
469 datatype = self.hyperdb_to_sql_datatypes.get(propclass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
470 if datatype: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
471 return datatype |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
472 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
473 for k, v in self.hyperdb_to_sql_datatypes.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
474 if issubclass(propclass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
475 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
476 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
477 raise ValueError('%r is not a hyperdb property class' % propclass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
478 |
| 1165 | 479 def determine_columns(self, properties): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
480 """ Figure the column names and multilink properties from the spec |
| 1165 | 481 |
| 482 "properties" is a list of (name, prop) where prop may be an | |
| 483 instance of a hyperdb "type" _or_ a string repr of that type. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
484 """ |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
485 cols = [ |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
486 ('_actor', self.hyperdb_to_sql_datatype(hyperdb.Link)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
487 ('_activity', self.hyperdb_to_sql_datatype(hyperdb.Date)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
488 ('_creator', self.hyperdb_to_sql_datatype(hyperdb.Link)), |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
489 ('_creation', self.hyperdb_to_sql_datatype(hyperdb.Date)), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
490 ] |
| 1165 | 491 mls = [] |
| 492 # add the multilinks separately | |
| 493 for col, prop in properties: | |
| 494 if isinstance(prop, Multilink): | |
| 495 mls.append(col) | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
496 continue |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
497 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
498 if isinstance(prop, type('')): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
499 raise ValueError("string property spec!") |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
500 #and prop.find('Multilink') != -1: |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
501 #mls.append(col) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
502 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
503 datatype = self.hyperdb_to_sql_datatype(prop.__class__) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
504 cols.append(('_'+col, datatype)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
505 |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
506 # Intervals stored as two columns |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
507 if isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
508 cols.append(('__'+col+'_int__', 'BIGINT')) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
509 |
| 1165 | 510 cols.sort() |
| 511 return cols, mls | |
| 512 | |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
513 def update_class(self, spec, old_spec, force=0): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
514 """ Determine the differences between the current spec and the |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
515 database version of the spec, and update where necessary. |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
516 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
517 If 'force' is true, update the database anyway. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
518 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
519 new_spec = spec.schema() |
|
1515
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
520 new_spec[1].sort() |
|
a516bbb9896b
fixed rdbms table update detection logic [SF#703297]
Richard Jones <richard@users.sourceforge.net>
parents:
1500
diff
changeset
|
521 old_spec[1].sort() |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1838
diff
changeset
|
522 if not force and new_spec == old_spec: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
523 # no changes |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
524 return 0 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
525 |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
526 if not self.config.RDBMS_ALLOW_ALTER: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
527 raise DatabaseError(_('ALTER operation disallowed: %r -> %r.'%(old_spec, new_spec))) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
528 |
|
4420
9655a1b65974
- more logger fixes -- use correct hierarchical logger names...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4410
diff
changeset
|
529 logger = logging.getLogger('roundup.hyperdb') |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
530 logger.info('update_class %s'%spec.classname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
531 |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
532 logger.debug('old_spec %r'%(old_spec,)) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
533 logger.debug('new_spec %r'%(new_spec,)) |
| 1165 | 534 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
535 # detect key prop change for potential index change |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
536 keyprop_changes = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
537 if new_spec[0] != old_spec[0]: |
|
3554
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
538 if old_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
539 keyprop_changes['remove'] = old_spec[0] |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
540 if new_spec[0]: |
|
5e70726a86dd
fixed schema migration problem when Class keys were removed
Richard Jones <richard@users.sourceforge.net>
parents:
3553
diff
changeset
|
541 keyprop_changes['add'] = new_spec[0] |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
542 |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
543 # detect multilinks that have been removed, and drop their table |
|
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
544 old_has = {} |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
545 for name, prop in old_spec[1]: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
546 old_has[name] = 1 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
547 if name in spec.properties: |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
548 continue |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
549 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
550 if prop.find('Multilink to') != -1: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
551 # first drop indexes. |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
552 self.drop_multilink_table_indexes(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
553 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
554 # now the multilink table itself |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
555 sql = 'drop table %s_%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
556 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
557 # if this is the key prop, drop the index first |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
558 if old_spec[0] == prop: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
559 self.drop_class_table_key_index(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
560 del keyprop_changes['remove'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
561 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
562 # drop the column |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
563 sql = 'alter table _%s drop column _%s'%(spec.classname, name) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
564 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
565 self.sql(sql) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
566 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
567 # if we didn't remove the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
568 # changed, we still need to remove the old index |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
569 if 'remove' in keyprop_changes: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
570 self.drop_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
571 keyprop_changes['remove']) |
| 1165 | 572 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
573 # add new columns |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
574 for propname, prop in new_spec[1]: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
575 if propname in old_has: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
576 continue |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
577 prop = spec.properties[propname] |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
578 if isinstance(prop, Multilink): |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
579 self.create_multilink_table(spec, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
580 else: |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
581 # add the column |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
582 coltype = self.hyperdb_to_sql_datatype(prop.__class__) |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
583 sql = 'alter table _%s add column _%s %s'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
584 spec.classname, propname, coltype) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
585 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
586 |
|
2727
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
587 # extra Interval column |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
588 if isinstance(prop, Interval): |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
589 sql = 'alter table _%s add column __%s_int__ BIGINT'%( |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
590 spec.classname, propname) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
591 self.sql(sql) |
|
93e2e5b55a3c
new Interval props weren't created properly in rdbms
Richard Jones <richard@users.sourceforge.net>
parents:
2721
diff
changeset
|
592 |
|
2196
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
593 # if the new column is a key prop, we need an index! |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
594 if new_spec[0] == propname: |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
595 self.create_class_table_key_index(spec.classname, propname) |
|
85954067e496
mysql and postgresql schema mutation now handle added Multilinks; fixed test too
Richard Jones <richard@users.sourceforge.net>
parents:
2185
diff
changeset
|
596 del keyprop_changes['add'] |
| 1165 | 597 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
598 # if we didn't add the key prop just then, but the key prop has |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
599 # changed, we still need to add the new index |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
600 if 'add' in keyprop_changes: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
601 self.create_class_table_key_index(spec.classname, |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
602 keyprop_changes['add']) |
|
1479
405e91b5be46
fixed rdbms mutation of properties
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
603 |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
604 return 1 |
| 1165 | 605 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
606 def determine_all_columns(self, spec): |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
607 """Figure out the columns from the spec and also add internal columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
608 |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
609 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
610 cols, mls = self.determine_columns(list(spec.properties.iteritems())) |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
611 |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
612 # add on our special columns |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
613 cols.append(('id', 'INTEGER PRIMARY KEY')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
614 cols.append(('__retired__', 'INTEGER DEFAULT 0')) |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
615 return cols, mls |
|
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
616 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
617 def create_class_table(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
618 """Create the class table for the given Class "spec". Creates the |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
619 indexes too.""" |
|
3016
224c7c0b9708
First checkin of tsearch2 "backend". Miscellaneous notes:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
2985
diff
changeset
|
620 cols, mls = self.determine_all_columns(spec) |
| 1165 | 621 |
| 622 # create the base table | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
623 scols = ','.join(['%s %s'%x for x in cols]) |
| 1165 | 624 sql = 'create table _%s (%s)'%(spec.classname, scols) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
625 self.sql(sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
626 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
627 self.create_class_table_indexes(spec) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
628 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
629 return cols, mls |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
630 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
631 def create_class_table_indexes(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
632 """ create the class table for the given spec |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
633 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
634 # create __retired__ index |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
635 index_sql2 = 'create index _%s_retired_idx on _%s(__retired__)'%( |
|
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
636 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
637 self.sql(index_sql2) |
| 1165 | 638 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
639 # create index for key property |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
640 if spec.key: |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
641 index_sql3 = 'create index _%s_%s_idx on _%s(_%s)'%( |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
642 spec.classname, spec.key, |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
643 spec.classname, spec.key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
644 self.sql(index_sql3) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
645 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
646 # and the unique index for key / retired(id) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
647 self.add_class_key_required_unique_constraint(spec.classname, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
648 spec.key) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
649 |
|
2228
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
650 # TODO: create indexes on (selected?) Link property columns, as |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
651 # they're more likely to be used for lookup |
|
1d1362c54c94
Some doc / comment fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
2217
diff
changeset
|
652 |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
653 def add_class_key_required_unique_constraint(self, cn, key): |
|
3969
905faf52a51f
fix mysql breakage in 1.4.2
Richard Jones <richard@users.sourceforge.net>
parents:
3967
diff
changeset
|
654 sql = '''create unique index _%s_key_retired_idx |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
655 on _%s(__retired__, _%s)'''%(cn, cn, key) |
|
4774
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
656 try: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
657 self.sql(sql) |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
658 except StandardError: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
659 # XXX catch e.g.: |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
660 # _sqlite.DatabaseError: index _status_key_retired_idx already exists |
|
3adff0fb0207
Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4652
diff
changeset
|
661 pass |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
662 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
663 def drop_class_table_indexes(self, cn, key): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
664 # drop the old table indexes first |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
665 l = ['_%s_id_idx'%cn, '_%s_retired_idx'%cn] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
666 if key: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
667 l.append('_%s_%s_idx'%(cn, key)) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
668 |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
669 table_name = '_%s'%cn |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
670 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
671 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
672 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
673 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
674 self.sql(index_sql) |
| 1165 | 675 |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
676 def create_class_table_key_index(self, cn, key): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
677 """ create the class table for the given spec |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
678 """ |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
679 sql = 'create index _%s_%s_idx on _%s(_%s)'%(cn, key, cn, key) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
680 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
681 |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
682 def drop_class_table_key_index(self, cn, key): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
683 table_name = '_%s'%cn |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
684 index_name = '_%s_%s_idx'%(cn, key) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
685 if self.sql_index_exists(table_name, index_name): |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
686 sql = 'drop index '+index_name |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
687 self.sql(sql) |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
688 |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
689 # and now the retired unique index too |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
690 index_name = '_%s_key_retired_idx'%cn |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
691 if self.sql_index_exists(table_name, index_name): |
|
3967
b1e81ad3fa6a
don't recompute the index name
Richard Jones <richard@users.sourceforge.net>
parents:
3963
diff
changeset
|
692 sql = 'drop index '+index_name |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
693 self.sql(sql) |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
694 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
695 def create_journal_table(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
696 """ create the journal table for a class given the spec and |
| 1165 | 697 already-determined cols |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
698 """ |
| 1165 | 699 # journal table |
| 700 cols = ','.join(['%s varchar'%x | |
| 701 for x in 'nodeid date tag action params'.split()]) | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
702 sql = """create table %s__journal ( |
|
3224
1fd11a9803bb
use backend datatype for journal timestamps.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3155
diff
changeset
|
703 nodeid integer, date %s, tag varchar(255), |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
704 action varchar(255), params text)""" % (spec.classname, |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
705 self.hyperdb_to_sql_datatype(hyperdb.Date)) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
706 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
707 self.create_journal_table_indexes(spec) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
708 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
709 def create_journal_table_indexes(self, spec): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
710 # index on nodeid |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
711 sql = 'create index %s_journ_idx on %s__journal(nodeid)'%( |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
712 spec.classname, spec.classname) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
713 self.sql(sql) |
| 1165 | 714 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
715 def drop_journal_table_indexes(self, classname): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
716 index_name = '%s_journ_idx'%classname |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
717 if not self.sql_index_exists('%s__journal'%classname, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
718 return |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
719 index_sql = 'drop index '+index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
720 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
721 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
722 def create_multilink_table(self, spec, ml): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
723 """ Create a multilink table for the "ml" property of the class |
| 1165 | 724 given by the spec |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
725 """ |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
726 # create the table |
|
2828
3a2f138fcb28
use integer IDs in multilink tables
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2755
diff
changeset
|
727 sql = 'create table %s_%s (linkid INTEGER, nodeid INTEGER)'%( |
| 1165 | 728 spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
729 self.sql(sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
730 self.create_multilink_table_indexes(spec, ml) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
731 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
732 def create_multilink_table_indexes(self, spec, ml): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
733 # create index on linkid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
734 index_sql = 'create index %s_%s_l_idx on %s_%s(linkid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
735 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
736 self.sql(index_sql) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
737 |
|
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
738 # create index on nodeid |
|
1836
94e430ad4fdb
make the RDBMS common backend and the SQLite and MYsql backend create...
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1800
diff
changeset
|
739 index_sql = 'create index %s_%s_n_idx on %s_%s(nodeid)'%( |
|
2100
62ed6505cbec
MySQL migration of old backend database to new, typed database complete.
Richard Jones <richard@users.sourceforge.net>
parents:
2098
diff
changeset
|
740 spec.classname, ml, spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
741 self.sql(index_sql) |
| 1165 | 742 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
743 def drop_multilink_table_indexes(self, classname, ml): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
744 l = [ |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
745 '%s_%s_l_idx'%(classname, ml), |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
746 '%s_%s_n_idx'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
747 ] |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
748 table_name = '%s_%s'%(classname, ml) |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
749 for index_name in l: |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
750 if not self.sql_index_exists(table_name, index_name): |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
751 continue |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
752 index_sql = 'drop index %s'%index_name |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
753 self.sql(index_sql) |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
754 |
| 1165 | 755 def create_class(self, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
756 """ Create a database table according to the given spec. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
757 """ |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
758 |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
759 if not self.config.RDBMS_ALLOW_CREATE: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
760 raise DatabaseError(_('CREATE operation disallowed: "%s".'%spec.classname)) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
761 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
762 cols, mls = self.create_class_table(spec) |
|
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
763 self.create_journal_table(spec) |
| 1165 | 764 |
| 765 # now create the multilink tables | |
| 766 for ml in mls: | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
767 self.create_multilink_table(spec, ml) |
| 1165 | 768 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
769 def drop_class(self, cn, spec): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
770 """ Drop the given table from the database. |
| 1165 | 771 |
| 772 Drop the journal and multilink tables too. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
773 """ |
|
4474
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
774 |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
775 if not self.config.RDBMS_ALLOW_DROP: |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
776 raise DatabaseError(_('DROP operation disallowed: "%s".'%cn)) |
|
9b4cf6c96ee2
Add flags to allow to restrict DB modifications.
Stefan Seefeld <stefan@seefeld.name>
parents:
4473
diff
changeset
|
777 |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
778 properties = spec[1] |
| 1165 | 779 # figure the multilinks |
| 780 mls = [] | |
|
3924
21d3d7eeea8c
assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3906
diff
changeset
|
781 for propname, prop in properties: |
| 1165 | 782 if isinstance(prop, Multilink): |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
783 mls.append(propname) |
| 1165 | 784 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
785 # drop class table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
786 self.drop_class_table_indexes(cn, spec[0]) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
787 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
788 self.drop_class_table(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
789 |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
790 # drop journal table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
791 self.drop_journal_table_indexes(cn) |
|
1873
f63aa57386b0
Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
792 sql = 'drop table %s__journal'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
793 self.sql(sql) |
| 1165 | 794 |
| 795 for ml in mls: | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
796 # drop multilink table and indexes |
|
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
797 self.drop_multilink_table_indexes(cn, ml) |
| 1165 | 798 sql = 'drop table %s_%s'%(spec.classname, ml) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
799 self.sql(sql) |
| 1165 | 800 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
801 def drop_class_table(self, cn): |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
802 sql = 'drop table _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
803 self.sql(sql) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
804 |
| 1165 | 805 # |
| 806 # Classes | |
| 807 # | |
| 808 def __getattr__(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
809 """ A convenient way of calling self.getclass(classname). |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
810 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
811 if classname in self.classes: |
| 1165 | 812 return self.classes[classname] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
813 raise AttributeError(classname) |
| 1165 | 814 |
| 815 def addclass(self, cl): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
816 """ Add a Class to the hyperdatabase. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
817 """ |
| 1165 | 818 cn = cl.classname |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
819 if cn in self.classes: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
820 raise ValueError(cn) |
| 1165 | 821 self.classes[cn] = cl |
| 822 | |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
823 # add default Edit and View permissions |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
824 self.security.addPermission(name="Create", klass=cn, |
|
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
825 description="User is allowed to create "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
826 self.security.addPermission(name="Edit", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
827 description="User is allowed to edit "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
828 self.security.addPermission(name="View", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
829 description="User is allowed to access "+cn) |
|
4517
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4490
diff
changeset
|
830 self.security.addPermission(name="Retire", klass=cn, |
|
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4490
diff
changeset
|
831 description="User is allowed to retire "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2075
diff
changeset
|
832 |
| 1165 | 833 def getclasses(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
834 """ Return a list of the names of all existing classes. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
835 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
836 return sorted(self.classes) |
| 1165 | 837 |
| 838 def getclass(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
839 """Get the Class object representing a particular class. |
| 1165 | 840 |
| 841 If 'classname' is not a valid class name, a KeyError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
842 """ |
| 1165 | 843 try: |
| 844 return self.classes[classname] | |
| 845 except KeyError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
846 raise KeyError('There is no class called "%s"'%classname) |
| 1165 | 847 |
| 848 def clear(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
849 """Delete all database contents. |
| 1165 | 850 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
851 Note: I don't commit here, which is different behaviour to the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
852 "nuke from orbit" behaviour in the dbs. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
853 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
854 logging.getLogger('roundup.hyperdb').info('clear') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
855 for cn in self.classes: |
| 1165 | 856 sql = 'delete from _%s'%cn |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
857 self.sql(sql) |
| 1165 | 858 |
| 859 # | |
| 860 # Nodes | |
| 861 # | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
862 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
863 hyperdb_to_sql_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
864 hyperdb.String : str, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
865 # fractional seconds by default |
|
3612
afda59d5d546
prevent generation of new single-digit seconds dates [SF#1429390]
Richard Jones <richard@users.sourceforge.net>
parents:
3601
diff
changeset
|
866 hyperdb.Date : lambda x: x.formal(sep=' ', sec='%06.3f'), |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
867 hyperdb.Link : int, |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
868 hyperdb.Interval : str, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
869 hyperdb.Password : str, |
|
2166
cd42c3c7173a
MySQL and Postgresql use BOOL/BOOLEAN for Boolean types
Richard Jones <richard@users.sourceforge.net>
parents:
2102
diff
changeset
|
870 hyperdb.Boolean : lambda x: x and 'TRUE' or 'FALSE', |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
871 hyperdb.Number : lambda x: x, |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
872 hyperdb.Integer : lambda x: x, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
873 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
874 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
875 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
876 def to_sql_value(self, propklass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
877 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
878 fn = self.hyperdb_to_sql_value.get(propklass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
879 if fn: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
880 return fn |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
881 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
882 for k, v in self.hyperdb_to_sql_value.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
883 if issubclass(propklass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
884 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
885 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
886 raise ValueError('%r is not a hyperdb property class' % propklass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
887 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
888 def _cache_del(self, key): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
889 del self.cache[key] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
890 self.cache_lru.remove(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
891 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
892 def _cache_refresh(self, key): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
893 self.cache_lru.remove(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
894 self.cache_lru.insert(0, key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
895 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
896 def _cache_save(self, key, node): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
897 self.cache[key] = node |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
898 # update the LRU |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
899 self.cache_lru.insert(0, key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
900 if len(self.cache_lru) > self.cache_size: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
901 del self.cache[self.cache_lru.pop()] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
902 |
| 1165 | 903 def addnode(self, classname, nodeid, node): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
904 """ Add the specified node to its class's db. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
905 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
906 self.log_debug('addnode %s%s %r'%(classname, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
907 nodeid, node)) |
|
1528
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
908 |
|
96cd422532ef
bye bye gadfly - you served your purpose well [SF#701127]
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
909 # determine the column definitions and multilink tables |
| 1165 | 910 cl = self.classes[classname] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
911 cols, mls = self.determine_columns(list(cl.properties.iteritems())) |
| 1165 | 912 |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
913 # we'll be supplied these props if we're doing an import |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
914 values = node.copy() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
915 if 'creator' not in values: |
|
1189
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
916 # add in the "calculated" properties (dupe so we don't affect |
|
23b8d1e87fe3
import fixes
Richard Jones <richard@users.sourceforge.net>
parents:
1187
diff
changeset
|
917 # calling code's node assumptions) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
918 values['creation'] = values['activity'] = date.Date() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
919 values['actor'] = values['creator'] = self.getuid() |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
920 |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
921 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
922 props = cl.getprops(protected=1) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
923 del props['id'] |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
924 |
| 1165 | 925 # default the non-multilink columns |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
926 for col, prop in props.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
927 if col not in values: |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
928 if isinstance(prop, Multilink): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
929 values[col] = [] |
|
1496
e6ac4e074acb
relaxed CVS importing (feature [SF#693277])
Richard Jones <richard@users.sourceforge.net>
parents:
1492
diff
changeset
|
930 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
931 values[col] = None |
| 1165 | 932 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
933 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
934 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
935 if key in self.cache: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
936 self._cache_del(key) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
937 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
938 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
939 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
940 for col,dt in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
941 # this is somewhat dodgy.... |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
942 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
943 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
944 value = values[col[2:-6]] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
945 # this is an Interval special "int" column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
946 if value is not None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
947 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
948 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
949 vals.append(value) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
950 continue |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
951 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
952 prop = props[col[1:]] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
953 value = values[col[1:]] |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
954 if value is not None: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
955 value = self.to_sql_value(prop.__class__)(value) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
956 vals.append(value) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
957 vals.append(nodeid) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
958 vals = tuple(vals) |
| 1165 | 959 |
| 960 # make sure the ordering is correct for column name -> column value | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
961 s = ','.join([self.arg for x in cols]) + ',%s'%self.arg |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
962 cols = ','.join([col for col,dt in cols]) + ',id' |
| 1165 | 963 |
| 964 # perform the inserts | |
| 965 sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
966 self.sql(sql, vals) |
| 1165 | 967 |
| 968 # insert the multilink rows | |
| 969 for col in mls: | |
| 970 t = '%s_%s'%(classname, col) | |
| 971 for entry in node[col]: | |
| 972 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, | |
| 973 self.arg, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
974 self.sql(sql, (entry, nodeid)) |
| 1165 | 975 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
976 def setnode(self, classname, nodeid, values, multilink_changes={}): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
977 """ Change the specified node. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
978 """ |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
979 self.log_debug('setnode %s%s %r' |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
980 % (classname, nodeid, values)) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
981 |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
982 # clear this node out of the cache if it's in there |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
983 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
984 if key in self.cache: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
985 self._cache_del(key) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
986 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
987 cl = self.classes[classname] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
988 props = cl.getprops() |
| 1165 | 989 |
| 990 cols = [] | |
| 991 mls = [] | |
| 992 # add the multilinks separately | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
993 for col in values: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
994 prop = props[col] |
| 1165 | 995 if isinstance(prop, Multilink): |
| 996 mls.append(col) | |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
997 elif isinstance(prop, Interval): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
998 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
999 cols.append(col) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1000 # extra leading '_' added by code below |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1001 cols.append('_' +col + '_int__') |
| 1165 | 1002 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1003 cols.append(col) |
| 1165 | 1004 cols.sort() |
| 1005 | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1006 # figure the values to insert |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1007 vals = [] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1008 for col in cols: |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1009 if col.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1010 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1011 # Intervals store the seconds value too |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1012 col = col[1:-6] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1013 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1014 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1015 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1016 vals.append(None) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1017 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1018 vals.append(value.as_seconds()) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1019 else: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1020 prop = props[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1021 value = values[col] |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1022 if value is None: |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1023 e = None |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1024 else: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1025 e = self.to_sql_value(prop.__class__)(value) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1026 vals.append(e) |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1027 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1028 vals.append(int(nodeid)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1029 vals = tuple(vals) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1030 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1031 # if there's any updates to regular columns, do them |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1032 if cols: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1033 # make sure the ordering is correct for column name -> column value |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1034 s = ','.join(['_%s=%s'%(x, self.arg) for x in cols]) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1035 cols = ','.join(cols) |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1036 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1037 # perform the update |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1038 sql = 'update _%s set %s where id=%s'%(classname, s, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1039 self.sql(sql, vals) |
| 1165 | 1040 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1041 # we're probably coming from an import, not a change |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1042 if not multilink_changes: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1043 for name in mls: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1044 prop = props[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1045 value = values[name] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1046 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1047 t = '%s_%s'%(classname, name) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1048 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1049 # clear out previous values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1050 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1051 self.sql('delete from %s where nodeid=%s'%(t, self.arg), |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1052 (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1053 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1054 # insert the values for this node |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1055 for entry in values[name]: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1056 sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1057 self.arg, self.arg) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1058 # XXX numeric ids |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1059 self.sql(sql, (entry, nodeid)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1060 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1061 # we have multilink changes to apply |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1062 for col, (add, remove) in multilink_changes.iteritems(): |
| 1165 | 1063 tn = '%s_%s'%(classname, col) |
| 1064 if add: | |
| 1065 sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn, | |
| 1066 self.arg, self.arg) | |
| 1067 for addid in add: | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1068 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1069 self.sql(sql, (int(nodeid), int(addid))) |
| 1165 | 1070 if remove: |
|
4039
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1071 s = ','.join([self.arg]*len(remove)) |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1072 sql = 'delete from %s where nodeid=%s and linkid in (%s)'%(tn, |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1073 self.arg, s) |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1074 # XXX numeric ids |
|
f44ec45cedb4
Fuse multiple DELETE calls into one for multilinks.
Stefan Seefeld <stefan@seefeld.name>
parents:
4038
diff
changeset
|
1075 self.sql(sql, [int(nodeid)] + remove) |
| 1165 | 1076 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1077 sql_to_hyperdb_value = { |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1078 hyperdb.String : str, |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
1079 hyperdb.Date : date_to_hyperdb_value, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1080 # hyperdb.Link : int, # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1081 hyperdb.Link : str, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1082 hyperdb.Interval : date.Interval, |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1083 hyperdb.Password : lambda x: password.Password(encrypted=x), |
|
2472
f41539b3c486
fixed Boolean values in postgresql (bugs [SF#972546] and [SF#972600])
Richard Jones <richard@users.sourceforge.net>
parents:
2456
diff
changeset
|
1084 hyperdb.Boolean : _bool_cvt, |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1085 hyperdb.Number : _num_cvt, |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1086 hyperdb.Integer : int, |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1087 hyperdb.Multilink : lambda x: x, # used in journal marshalling |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1088 } |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1089 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1090 def to_hyperdb_value(self, propklass): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1091 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1092 fn = self.sql_to_hyperdb_value.get(propklass) |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1093 if fn: |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1094 return fn |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1095 |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1096 for k, v in self.sql_to_hyperdb_value.iteritems(): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1097 if issubclass(propklass, k): |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1098 return v |
|
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1099 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1100 raise ValueError('%r is not a hyperdb property class' % propklass) |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1101 |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1102 def _materialize_multilink(self, classname, nodeid, node, propname): |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1103 """ evaluation of single Multilink (lazy eval may have skipped this) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1104 """ |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1105 if propname not in node: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1106 sql = 'select linkid from %s_%s where nodeid=%s'%(classname, |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1107 propname, self.arg) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1108 self.sql(sql, (nodeid,)) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1109 # extract the first column from the result |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1110 # XXX numeric ids |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1111 items = [int(x[0]) for x in self.cursor.fetchall()] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1112 items.sort () |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1113 node[propname] = [str(x) for x in items] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1114 |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1115 def _materialize_multilinks(self, classname, nodeid, node, props=None): |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1116 """ get all Multilinks of a node (lazy eval may have skipped this) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1117 """ |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1118 cl = self.classes[classname] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1119 props = props or [pn for (pn, p) in cl.properties.iteritems() |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1120 if isinstance(p, Multilink)] |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1121 for propname in props: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1122 if propname not in node: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1123 self._materialize_multilink(classname, nodeid, node, propname) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1124 |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1125 def getnode(self, classname, nodeid, fetch_multilinks=True): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1126 """ Get a node from the database. |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1127 For optimisation optionally we don't fetch multilinks |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1128 (lazy Multilinks). |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1129 But for internal database operations we need them. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1130 """ |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1131 # see if we have this node cached |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1132 key = (classname, nodeid) |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1133 if key in self.cache: |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1134 # push us back to the top of the LRU |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1135 self._cache_refresh(key) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1136 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1137 self.stats['cache_hits'] += 1 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1138 # return the cached information |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1139 if fetch_multilinks: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1140 self._materialize_multilinks(classname, nodeid, self.cache[key]) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1141 return self.cache[key] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1142 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1143 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1144 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1145 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1146 |
| 1165 | 1147 # figure the columns we're fetching |
| 1148 cl = self.classes[classname] | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1149 cols, mls = self.determine_columns(list(cl.properties.iteritems())) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1150 scols = ','.join([col for col,dt in cols]) |
| 1165 | 1151 |
| 1152 # perform the basic property fetch | |
| 1153 sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1154 self.sql(sql, (nodeid,)) |
| 1165 | 1155 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1156 values = self.sql_fetchone() |
| 1165 | 1157 if values is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1158 raise IndexError('no such %s node %s'%(classname, nodeid)) |
| 1165 | 1159 |
| 1160 # make up the node | |
| 1161 node = {} | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1162 props = cl.getprops(protected=1) |
| 1165 | 1163 for col in range(len(cols)): |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1164 name = cols[col][0][1:] |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1165 if name.endswith('_int__'): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1166 # XXX eugh, this test suxxors |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1167 # ignore the special Interval-as-seconds column |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
1168 continue |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1169 value = values[col] |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1170 if value is not None: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1171 value = self.to_hyperdb_value(props[name].__class__)(value) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1172 node[name] = value |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1173 |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1174 if fetch_multilinks and mls: |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1175 self._materialize_multilinks(classname, nodeid, node, mls) |
|
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1176 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1177 # save off in the cache |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1178 key = (classname, nodeid) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1179 self._cache_save(key, node) |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1180 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1181 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1182 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
1183 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1184 return node |
| 1165 | 1185 |
| 1186 def destroynode(self, classname, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1187 """Remove a node from the database. Called exclusively by the |
| 1165 | 1188 destroy() method on Class. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1189 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1190 logging.getLogger('roundup.hyperdb').info('destroynode %s%s'%( |
|
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1191 classname, nodeid)) |
| 1165 | 1192 |
| 1193 # make sure the node exists | |
| 1194 if not self.hasnode(classname, nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1195 raise IndexError('%s has no node %s'%(classname, nodeid)) |
| 1165 | 1196 |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1197 # see if we have this node cached |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1198 if (classname, nodeid) in self.cache: |
|
1173
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1199 del self.cache[(classname, nodeid)] |
|
58f1a2c174ed
simple LRU cache for SQL databases
Richard Jones <richard@users.sourceforge.net>
parents:
1172
diff
changeset
|
1200 |
| 1165 | 1201 # see if there's any obvious commit actions that we should get rid of |
| 1202 for entry in self.transactions[:]: | |
| 1203 if entry[1][:2] == (classname, nodeid): | |
| 1204 self.transactions.remove(entry) | |
| 1205 | |
| 1206 # now do the SQL | |
| 1207 sql = 'delete from _%s where id=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1208 self.sql(sql, (nodeid,)) |
| 1165 | 1209 |
| 1210 # remove from multilnks | |
| 1211 cl = self.getclass(classname) | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1212 x, mls = self.determine_columns(list(cl.properties.iteritems())) |
| 1165 | 1213 for col in mls: |
| 1214 # get the link ids | |
| 1215 sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg) | |
|
1906
f255363e6d97
PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents:
1873
diff
changeset
|
1216 self.sql(sql, (nodeid,)) |
| 1165 | 1217 |
| 1218 # remove journal entries | |
| 1219 sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg) | |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1220 self.sql(sql, (nodeid,)) |
| 1165 | 1221 |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1222 # cleanup any blob filestorage when we commit |
|
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1223 self.transactions.append((FileStorage.destroy, (self, classname, nodeid))) |
|
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1224 |
| 1165 | 1225 def hasnode(self, classname, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1226 """ Determine if the database has a given node. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1227 """ |
|
5086
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1228 # nodeid (aka link) is sql type INTEGER. max positive value |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1229 # for INTEGER is 2^31-1 for Postgres and MySQL. The max |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1230 # positive value for SqLite is 2^63 -1, so arguably this check |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1231 # needs to adapt for the type of the RDBMS. For right now, |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1232 # choose lowest common denominator. |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1233 if int(nodeid) >= 2**31: |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1234 # value out of range return false |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1235 return 0 |
|
2b551b6b0830
issue2550549 Postgres error on message templating
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
1236 |
| 4031 | 1237 # If this node is in the cache, then we do not need to go to |
| 1238 # the database. (We don't consider this an LRU hit, though.) | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1239 if (classname, nodeid) in self.cache: |
| 4031 | 1240 # Return 1, not True, to match the type of the result of |
| 1241 # the SQL operation below. | |
| 1242 return 1 | |
| 1165 | 1243 sql = 'select count(*) from _%s where id=%s'%(classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1244 self.sql(sql, (nodeid,)) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1245 return int(self.cursor.fetchone()[0]) |
| 1165 | 1246 |
| 1247 def countnodes(self, classname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1248 """ Count the number of nodes that exist for a particular Class. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1249 """ |
| 1165 | 1250 sql = 'select count(*) from _%s'%classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1251 self.sql(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1252 return self.cursor.fetchone()[0] |
| 1165 | 1253 |
| 1254 def addjournal(self, classname, nodeid, action, params, creator=None, | |
| 1255 creation=None): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1256 """ Journal the Action |
| 1165 | 1257 'action' may be: |
| 1258 | |
| 1259 'create' or 'set' -- 'params' is a dictionary of property values | |
| 1260 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) | |
| 1261 'retire' -- 'params' is None | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1262 """ |
| 1165 | 1263 # handle supply of the special journalling parameters (usually |
| 1264 # supplied on importing an existing database) | |
| 1265 if creator: | |
| 1266 journaltag = creator | |
| 1267 else: | |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1268 journaltag = self.getuid() |
| 1165 | 1269 if creation: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1270 journaldate = creation |
| 1165 | 1271 else: |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1272 journaldate = date.Date() |
| 1165 | 1273 |
| 1274 # create the journal entry | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1275 cols = 'nodeid,date,tag,action,params' |
| 1165 | 1276 |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1277 self.log_debug('addjournal %s%s %r %s %s %r'%(classname, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1278 nodeid, journaldate, journaltag, action, params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1279 |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1280 # make the journalled data marshallable |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1281 if isinstance(params, type({})): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1282 self._journal_marshal(params, classname) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1283 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1284 params = repr(params) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1285 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1286 dc = self.to_sql_value(hyperdb.Date) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1287 journaldate = dc(journaldate) |
| 1165 | 1288 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1289 self.save_journal(classname, cols, nodeid, journaldate, |
| 1165 | 1290 journaltag, action, params) |
| 1291 | |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1292 def setjournal(self, classname, nodeid, journal): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1293 """Set the journal to the "journal" list.""" |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1294 # clear out any existing entries |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1295 self.sql('delete from %s__journal where nodeid=%s'%(classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1296 self.arg), (nodeid,)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1297 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1298 # create the journal entry |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1299 cols = 'nodeid,date,tag,action,params' |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1300 |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1301 dc = self.to_sql_value(hyperdb.Date) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1302 for nodeid, journaldate, journaltag, action, params in journal: |
|
4059
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1303 self.log_debug('addjournal %s%s %r %s %s %r'%( |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1304 classname, nodeid, journaldate, journaltag, action, |
|
ef0b4396888a
Enhance and simplify logging.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1305 params)) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1306 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1307 # make the journalled data marshallable |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1308 if isinstance(params, type({})): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1309 self._journal_marshal(params, classname) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1310 params = repr(params) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1311 |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1312 self.save_journal(classname, cols, nodeid, dc(journaldate), |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1313 journaltag, action, params) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
1314 |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1315 def _journal_marshal(self, params, classname): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1316 """Convert the journal params values into safely repr'able and |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1317 eval'able values.""" |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1318 properties = self.getclass(classname).getprops() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1319 for param, value in params.iteritems(): |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1320 if not value: |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1321 continue |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1322 property = properties[param] |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1323 cvt = self.to_sql_value(property.__class__) |
|
2275
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1324 if isinstance(property, Password): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1325 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1326 elif isinstance(property, Date): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1327 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1328 elif isinstance(property, Interval): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1329 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1330 elif isinstance(property, Boolean): |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1331 params[param] = cvt(value) |
|
3197e37346de
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2256
diff
changeset
|
1332 |
| 1165 | 1333 def getjournal(self, classname, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1334 """ get the journal for id |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1335 """ |
| 1165 | 1336 # make sure the node exists |
| 1337 if not self.hasnode(classname, nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1338 raise IndexError('%s has no node %s'%(classname, nodeid)) |
| 1165 | 1339 |
| 1340 cols = ','.join('nodeid date tag action params'.split()) | |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1341 journal = self.load_journal(classname, cols, nodeid) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1342 |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1343 # now unmarshal the data |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1344 dc = self.to_hyperdb_value(hyperdb.Date) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1345 res = [] |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1346 properties = self.getclass(classname).getprops() |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1347 for nodeid, date_stamp, user, action, params in journal: |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1348 params = eval(params) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1349 if isinstance(params, type({})): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1350 for param, value in params.iteritems(): |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1351 if not value: |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1352 continue |
|
2731
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1353 property = properties.get(param, None) |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1354 if property is None: |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1355 # deleted property |
|
7c472ed6babf
handle deleted properties in RDBMS history
Richard Jones <richard@users.sourceforge.net>
parents:
2727
diff
changeset
|
1356 continue |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1357 cvt = self.to_hyperdb_value(property.__class__) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1358 if isinstance(property, Password): |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1359 params[param] = password.JournalPassword(value) |
|
2248
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1360 elif isinstance(property, Date): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1361 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1362 elif isinstance(property, Interval): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1363 params[param] = cvt(value) |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1364 elif isinstance(property, Boolean): |
|
cd7e6d6288c6
fixed rego from email address [SF#947414]
Richard Jones <richard@users.sourceforge.net>
parents:
2244
diff
changeset
|
1365 params[param] = cvt(value) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1366 # XXX numeric ids |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1367 res.append((str(nodeid), dc(date_stamp), user, action, params)) |
|
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1368 return res |
| 1165 | 1369 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1370 def save_journal(self, classname, cols, nodeid, journaldate, |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1371 journaltag, action, params): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1372 """ Save the journal entry to the database |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1373 """ |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1374 entry = (nodeid, journaldate, journaltag, action, params) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1375 |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1376 # do the insert |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1377 a = self.arg |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1378 sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%( |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1379 classname, cols, a, a, a, a, a) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1380 self.sql(sql, entry) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1381 |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
1382 def load_journal(self, classname, cols, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1383 """ Load the journal from the database |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1384 """ |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1385 # now get the journal entries |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1386 sql = 'select %s from %s__journal where nodeid=%s order by date'%( |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1387 cols, classname, self.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1388 self.sql(sql, (nodeid,)) |
|
2244
ac4f295499a4
fixed journal marshalling in RDBMS backends [SF#943627]
Richard Jones <richard@users.sourceforge.net>
parents:
2240
diff
changeset
|
1389 return self.cursor.fetchall() |
| 1165 | 1390 |
| 1391 def pack(self, pack_before): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1392 """ Delete all journal entries except "create" before 'pack_before'. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1393 """ |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
1394 date_stamp = self.to_sql_value(Date)(pack_before) |
| 1165 | 1395 |
| 1396 # do the delete | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1397 for classname in self.classes: |
| 1165 | 1398 sql = "delete from %s__journal where date<%s and "\ |
| 1399 "action<>'create'"%(classname, self.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1400 self.sql(sql, (date_stamp,)) |
| 1165 | 1401 |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1402 def sql_commit(self, fail_ok=False): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1403 """ Actually commit to the database. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1404 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1405 logging.getLogger('roundup.hyperdb').info('commit') |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1406 |
| 1165 | 1407 self.conn.commit() |
| 1408 | |
|
2374
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1409 # open a new cursor for subsequent work |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1410 self.cursor = self.conn.cursor() |
|
31cb1014300c
Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents:
2362
diff
changeset
|
1411 |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1412 def commit(self, fail_ok=False): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1413 """ Commit the current transactions. |
| 1165 | 1414 |
| 1415 Save all data changed since the database was opened or since the | |
| 1416 last commit() or rollback(). | |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1417 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1418 fail_ok indicates that the commit is allowed to fail. This is used |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1419 in the web interface when committing cleaning of the session |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1420 database. We don't care if there's a concurrency issue there. |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1421 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1422 The only backend this seems to affect is postgres. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1423 """ |
| 1165 | 1424 # commit the database |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3685
diff
changeset
|
1425 self.sql_commit(fail_ok) |
| 1165 | 1426 |
| 1427 # now, do all the other transaction stuff | |
| 1428 for method, args in self.transactions: | |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1429 method(*args) |
| 1165 | 1430 |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1431 # save the indexer |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1432 self.indexer.save_index() |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
1433 |
| 1165 | 1434 # clear out the transactions |
| 1435 self.transactions = [] | |
| 1436 | |
|
4448
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1437 # clear the cache: Don't carry over cached values from one |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1438 # transaction to the next (there may be other changes from other |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1439 # transactions) |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1440 self.clearCache() |
|
2784c239e6c8
clear the cache on commit for rdbms backends:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4430
diff
changeset
|
1441 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1442 def sql_rollback(self): |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1443 self.conn.rollback() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1444 |
| 1165 | 1445 def rollback(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1446 """ Reverse all actions from the current transaction. |
| 1165 | 1447 |
| 1448 Undo all the changes made since the database was opened or the last | |
| 1449 commit() or rollback() was performed. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1450 """ |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1451 logging.getLogger('roundup.hyperdb').info('rollback') |
| 1165 | 1452 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1453 self.sql_rollback() |
| 1165 | 1454 |
| 1455 # roll back "other" transaction stuff | |
| 1456 for method, args in self.transactions: | |
| 1457 # delete temporary files | |
| 1458 if method == self.doStoreFile: | |
| 1459 self.rollbackStoreFile(*args) | |
| 1460 self.transactions = [] | |
| 1461 | |
|
1492
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1462 # clear the cache |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1463 self.clearCache() |
|
2fc7d4a8c9e7
fixed sqlite rollback/caching bug [SF#689383]
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1464 |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1465 def sql_close(self): |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
1466 logging.getLogger('roundup.hyperdb').info('close') |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1467 self.conn.close() |
|
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1468 |
| 1165 | 1469 def close(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1470 """ Close off the connection. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1471 """ |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
1472 self.indexer.close() |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
1473 self.sql_close() |
| 1165 | 1474 |
| 1475 # | |
| 1476 # The base Class class | |
| 1477 # | |
| 1478 class Class(hyperdb.Class): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1479 """ The handle to a particular class of nodes in a hyperdatabase. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1480 |
| 1165 | 1481 All methods except __repr__ and getnode must be implemented by a |
| 1482 concrete backend Class. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1483 """ |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1484 # For many databases the LIKE operator ignores case. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1485 # Postgres and Oracle have an ILIKE operator to support this. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1486 # We define the default here, can be changed in derivative class |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
1487 case_insensitive_like = 'LIKE' |
| 1165 | 1488 |
| 1489 def schema(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1490 """ A dumpable version of the schema that we can store in the |
| 1165 | 1491 database |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1492 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1493 return (self.key, [(x, repr(y)) for x,y in self.properties.iteritems()]) |
| 1165 | 1494 |
| 1495 def enableJournalling(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1496 """Turn journalling on for this class |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1497 """ |
| 1165 | 1498 self.do_journal = 1 |
| 1499 | |
| 1500 def disableJournalling(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1501 """Turn journalling off for this class |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1502 """ |
| 1165 | 1503 self.do_journal = 0 |
| 1504 | |
| 1505 # Editing nodes: | |
| 1506 def create(self, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1507 """ Create a new node of this class and return its id. |
| 1165 | 1508 |
| 1509 The keyword arguments in 'propvalues' map property names to values. | |
| 1510 | |
| 1511 The values of arguments must be acceptable for the types of their | |
| 1512 corresponding properties or a TypeError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1513 |
| 1165 | 1514 If this class has a key property, it must be present and its value |
| 1515 must not collide with other key strings or a ValueError is raised. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1516 |
| 1165 | 1517 Any other properties on this class that are missing from the |
| 1518 'propvalues' dictionary are set to None. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1519 |
| 1165 | 1520 If an id in a link or multilink property does not refer to a valid |
| 1521 node, an IndexError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1522 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1523 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1524 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1525 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1526 return newid |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1527 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
1528 def create_inner(self, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1529 """ Called by create, in-between the audit and react calls. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1530 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1531 if 'id' in propvalues: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1532 raise KeyError('"id" is reserved') |
| 1165 | 1533 |
| 1534 if self.db.journaltag is None: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1535 raise DatabaseError(_('Database open read-only')) |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1536 |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1537 if ('creator' in propvalues or 'actor' in propvalues or |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1538 'creation' in propvalues or 'activity' in propvalues): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1539 raise KeyError('"creator", "actor", "creation" and ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1540 '"activity" are reserved') |
| 1165 | 1541 |
| 1542 # new node's id | |
| 1543 newid = self.db.newid(self.classname) | |
| 1544 | |
| 1545 # validate propvalues | |
| 1546 num_re = re.compile('^\d+$') | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1547 for key, value in propvalues.iteritems(): |
| 1165 | 1548 if key == self.key: |
| 1549 try: | |
| 1550 self.lookup(value) | |
| 1551 except KeyError: | |
| 1552 pass | |
| 1553 else: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1554 raise ValueError('node with key "%s" exists'%value) |
| 1165 | 1555 |
| 1556 # try to handle this property | |
| 1557 try: | |
| 1558 prop = self.properties[key] | |
| 1559 except KeyError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1560 raise KeyError('"%s" has no property "%s"'%(self.classname, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1561 key)) |
| 1165 | 1562 |
| 1563 if value is not None and isinstance(prop, Link): | |
| 1564 if type(value) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1565 raise ValueError('link value must be String') |
| 1165 | 1566 link_class = self.properties[key].classname |
| 1567 # if it isn't a number, it's a key | |
| 1568 if not num_re.match(value): | |
| 1569 try: | |
| 1570 value = self.db.classes[link_class].lookup(value) | |
| 1571 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1572 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1573 key, value, link_class)) |
| 1165 | 1574 elif not self.db.getclass(link_class).hasnode(value): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1575 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1576 value)) |
| 1165 | 1577 |
| 1578 # save off the value | |
| 1579 propvalues[key] = value | |
| 1580 | |
| 1581 # register the link with the newly linked node | |
| 1582 if self.do_journal and self.properties[key].do_journal: | |
| 1583 self.db.addjournal(link_class, value, 'link', | |
| 1584 (self.classname, newid, key)) | |
| 1585 | |
| 1586 elif isinstance(prop, Multilink): | |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1587 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1588 value = [] |
|
3869
16faac822fe5
Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
1589 if not hasattr(value, '__iter__'): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1590 raise TypeError('new property "%s" not an iterable of ids'%key) |
| 1165 | 1591 # clean up and validate the list of links |
| 1592 link_class = self.properties[key].classname | |
| 1593 l = [] | |
| 1594 for entry in value: | |
| 1595 if type(entry) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1596 raise ValueError('"%s" multilink value (%r) ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1597 'must contain Strings'%(key, value)) |
| 1165 | 1598 # if it isn't a number, it's a key |
| 1599 if not num_re.match(entry): | |
| 1600 try: | |
| 1601 entry = self.db.classes[link_class].lookup(entry) | |
| 1602 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1603 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1604 key, entry, self.properties[key].classname)) |
| 1165 | 1605 l.append(entry) |
| 1606 value = l | |
| 1607 propvalues[key] = value | |
| 1608 | |
| 1609 # handle additions | |
| 1610 for nodeid in value: | |
| 1611 if not self.db.getclass(link_class).hasnode(nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1612 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1613 nodeid)) |
| 1165 | 1614 # register the link with the newly linked node |
| 1615 if self.do_journal and self.properties[key].do_journal: | |
| 1616 self.db.addjournal(link_class, nodeid, 'link', | |
| 1617 (self.classname, newid, key)) | |
| 1618 | |
| 1619 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1620 if type(value) != type('') and type(value) != type(u''): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1621 raise TypeError('new property "%s" not a string'%key) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1622 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1623 self.db.indexer.add_text((self.classname, newid, key), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1624 value) |
| 1165 | 1625 |
| 1626 elif isinstance(prop, Password): | |
| 1627 if not isinstance(value, password.Password): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1628 raise TypeError('new property "%s" not a Password'%key) |
| 1165 | 1629 |
| 1630 elif isinstance(prop, Date): | |
| 1631 if value is not None and not isinstance(value, date.Date): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1632 raise TypeError('new property "%s" not a Date'%key) |
| 1165 | 1633 |
| 1634 elif isinstance(prop, Interval): | |
| 1635 if value is not None and not isinstance(value, date.Interval): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1636 raise TypeError('new property "%s" not an Interval'%key) |
| 1165 | 1637 |
| 1638 elif value is not None and isinstance(prop, Number): | |
| 1639 try: | |
| 1640 float(value) | |
| 1641 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1642 raise TypeError('new property "%s" not numeric'%key) |
| 1165 | 1643 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1644 elif value is not None and isinstance(prop, Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1645 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1646 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1647 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1648 raise TypeError('new property "%s" not integer'%key) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1649 |
| 1165 | 1650 elif value is not None and isinstance(prop, Boolean): |
| 1651 try: | |
| 1652 int(value) | |
| 1653 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1654 raise TypeError('new property "%s" not boolean'%key) |
| 1165 | 1655 |
| 1656 # make sure there's data where there needs to be | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1657 for key, prop in self.properties.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1658 if key in propvalues: |
| 1165 | 1659 continue |
| 1660 if key == self.key: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1661 raise ValueError('key property "%s" is required'%key) |
| 1165 | 1662 if isinstance(prop, Multilink): |
| 1663 propvalues[key] = [] | |
| 1664 else: | |
| 1665 propvalues[key] = None | |
| 1666 | |
| 1667 # done | |
| 1668 self.db.addnode(self.classname, newid, propvalues) | |
| 1669 if self.do_journal: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1670 self.db.addjournal(self.classname, newid, ''"create", {}) |
| 1165 | 1671 |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1672 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
1673 return str(newid) |
| 1165 | 1674 |
| 1675 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1676 """Get the value of a property on an existing node of this class. |
| 1165 | 1677 |
| 1678 'nodeid' must be the id of an existing node of this class or an | |
| 1679 IndexError is raised. 'propname' must be the name of a property | |
| 1680 of this class or a KeyError is raised. | |
| 1681 | |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1682 'cache' exists for backwards compatibility, and is not used. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1683 """ |
| 1165 | 1684 if propname == 'id': |
| 1685 return nodeid | |
| 1686 | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1687 # get the node's dict |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1688 d = self.db.getnode(self.classname, nodeid, fetch_multilinks=False) |
|
4476
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1689 # handle common case -- that property is in dict -- first |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1690 # if None and one of creator/creation actor/activity return None |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1691 if propname in d: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1692 r = d [propname] |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1693 # return copy of our list |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1694 if isinstance (r, list): |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1695 return r[:] |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1696 if r is not None: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1697 return r |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1698 elif propname in ('creation', 'activity', 'creator', 'actor'): |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1699 return r |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1700 |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1701 # propname not in d: |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1702 if propname == 'creation' or propname == 'activity': |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1703 return date.Date() |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1704 if propname == 'creator' or propname == 'actor': |
|
143f52d48e60
- small performance optimisation for 'get': do common case first
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4474
diff
changeset
|
1705 return self.db.getuid() |
| 1165 | 1706 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1707 # get the property (raises KeyError if invalid) |
| 1165 | 1708 prop = self.properties[propname] |
| 1709 | |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1710 # lazy evaluation of Multilink |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1711 if propname not in d and isinstance(prop, Multilink): |
|
4490
559d9a2a0191
Fixed bug in filter_iter refactoring (lazy multilinks)...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4483
diff
changeset
|
1712 self.db._materialize_multilink(self.classname, nodeid, d, propname) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1713 |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1714 # handle there being no value in the table for the property |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1715 if propname not in d or d[propname] is None: |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
1716 if default is _marker: |
| 1165 | 1717 if isinstance(prop, Multilink): |
| 1718 return [] | |
| 1719 else: | |
| 1720 return None | |
| 1721 else: | |
| 1722 return default | |
| 1723 | |
| 1724 # don't pass our list to other code | |
| 1725 if isinstance(prop, Multilink): | |
| 1726 return d[propname][:] | |
| 1727 | |
| 1728 return d[propname] | |
| 1729 | |
| 1730 def set(self, nodeid, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1731 """Modify a property on an existing node of this class. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1732 |
| 1165 | 1733 'nodeid' must be the id of an existing node of this class or an |
| 1734 IndexError is raised. | |
| 1735 | |
| 1736 Each key in 'propvalues' must be the name of a property of this | |
| 1737 class or a KeyError is raised. | |
| 1738 | |
| 1739 All values in 'propvalues' must be acceptable types for their | |
| 1740 corresponding properties or a TypeError is raised. | |
| 1741 | |
| 1742 If the value of the key property is set, it must not collide with | |
| 1743 other key strings or a ValueError is raised. | |
| 1744 | |
| 1745 If the value of a Link or Multilink property contains an invalid | |
| 1746 node id, a ValueError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1747 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1748 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1749 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1750 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1751 self.fireReactors('set', nodeid, oldvalues) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1752 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1753 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1754 def set_inner(self, nodeid, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1755 """ Called by set, in-between the audit and react calls. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1756 """ |
| 1165 | 1757 if not propvalues: |
| 1758 return propvalues | |
| 1759 | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1760 if ('creator' in propvalues or 'actor' in propvalues or |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1761 'creation' in propvalues or 'activity' in propvalues): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1762 raise KeyError('"creator", "actor", "creation" and ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1763 '"activity" are reserved') |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1764 |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1765 if 'id' in propvalues: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1766 raise KeyError('"id" is reserved') |
| 1165 | 1767 |
| 1768 if self.db.journaltag is None: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1769 raise DatabaseError(_('Database open read-only')) |
| 1165 | 1770 |
| 1771 node = self.db.getnode(self.classname, nodeid) | |
| 1772 if self.is_retired(nodeid): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1773 raise IndexError('Requested item is retired') |
| 1165 | 1774 num_re = re.compile('^\d+$') |
| 1775 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1776 # make a copy of the values dictionary - we'll modify the contents |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1777 propvalues = propvalues.copy() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1778 |
| 1165 | 1779 # if the journal value is to be different, store it in here |
| 1780 journalvalues = {} | |
| 1781 | |
| 1782 # remember the add/remove stuff for multilinks, making it easier | |
| 1783 # for the Database layer to do its stuff | |
| 1784 multilink_changes = {} | |
| 1785 | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1786 for propname, value in list(propvalues.items()): |
| 1165 | 1787 # check to make sure we're not duplicating an existing key |
| 1788 if propname == self.key and node[propname] != value: | |
| 1789 try: | |
| 1790 self.lookup(value) | |
| 1791 except KeyError: | |
| 1792 pass | |
| 1793 else: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1794 raise ValueError('node with key "%s" exists'%value) |
| 1165 | 1795 |
| 1796 # this will raise the KeyError if the property isn't valid | |
| 1797 # ... we don't use getprops() here because we only care about | |
| 1798 # the writeable properties. | |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1799 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1800 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1801 except KeyError: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1802 raise KeyError('"%s" has no property named "%s"'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1803 self.classname, propname)) |
| 1165 | 1804 |
| 1805 # if the value's the same as the existing value, no sense in | |
| 1806 # doing anything | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1807 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1808 if value == current: |
| 1165 | 1809 del propvalues[propname] |
| 1810 continue | |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1295
diff
changeset
|
1811 journalvalues[propname] = current |
| 1165 | 1812 |
| 1813 # do stuff based on the prop type | |
| 1814 if isinstance(prop, Link): | |
| 1815 link_class = prop.classname | |
| 1816 # if it isn't a number, it's a key | |
| 1817 if value is not None and not isinstance(value, type('')): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1818 raise ValueError('property "%s" link value be a string'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1819 propname)) |
| 1165 | 1820 if isinstance(value, type('')) and not num_re.match(value): |
| 1821 try: | |
| 1822 value = self.db.classes[link_class].lookup(value) | |
| 1823 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1824 raise IndexError('new property "%s": %s not a %s'%( |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1825 propname, value, prop.classname)) |
| 1165 | 1826 |
| 1827 if (value is not None and | |
| 1828 not self.db.getclass(link_class).hasnode(value)): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1829 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1830 value)) |
| 1165 | 1831 |
| 1832 if self.do_journal and prop.do_journal: | |
| 1833 # register the unlink with the old linked node | |
| 1834 if node[propname] is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1835 self.db.addjournal(link_class, node[propname], |
|
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1836 ''"unlink", (self.classname, nodeid, propname)) |
| 1165 | 1837 |
| 1838 # register the link with the newly linked node | |
| 1839 if value is not None: | |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1840 self.db.addjournal(link_class, value, ''"link", |
| 1165 | 1841 (self.classname, nodeid, propname)) |
| 1842 | |
| 1843 elif isinstance(prop, Multilink): | |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1844 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1845 value = [] |
|
3869
16faac822fe5
Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3858
diff
changeset
|
1846 if not hasattr(value, '__iter__'): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1847 raise TypeError('new property "%s" not an iterable of' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1848 ' ids'%propname) |
| 1165 | 1849 link_class = self.properties[propname].classname |
| 1850 l = [] | |
| 1851 for entry in value: | |
| 1852 # if it isn't a number, it's a key | |
| 1853 if type(entry) != type(''): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1854 raise ValueError('new property "%s" link value ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1855 'must be a string'%propname) |
| 1165 | 1856 if not num_re.match(entry): |
| 1857 try: | |
| 1858 entry = self.db.classes[link_class].lookup(entry) | |
| 1859 except (TypeError, KeyError): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1860 raise IndexError('new property "%s": %s not a %s'%( |
| 1165 | 1861 propname, entry, |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1862 self.properties[propname].classname)) |
| 1165 | 1863 l.append(entry) |
| 1864 value = l | |
| 1865 propvalues[propname] = value | |
| 1866 | |
| 1867 # figure the journal entry for this property | |
| 1868 add = [] | |
| 1869 remove = [] | |
| 1870 | |
| 1871 # handle removals | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1872 if propname in node: |
| 1165 | 1873 l = node[propname] |
| 1874 else: | |
| 1875 l = [] | |
| 1876 for id in l[:]: | |
| 1877 if id in value: | |
| 1878 continue | |
| 1879 # register the unlink with the old linked node | |
| 1880 if self.do_journal and self.properties[propname].do_journal: | |
| 1881 self.db.addjournal(link_class, id, 'unlink', | |
| 1882 (self.classname, nodeid, propname)) | |
| 1883 l.remove(id) | |
| 1884 remove.append(id) | |
| 1885 | |
| 1886 # handle additions | |
| 1887 for id in value: | |
|
4095
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1888 if id in l: |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1889 continue |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1890 # We can safely check this condition after |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1891 # checking that this is an addition to the |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1892 # multilink since the condition was checked for |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1893 # existing entries at the point they were added to |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1894 # the multilink. Since the hasnode call will |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1895 # result in a SQL query, it is more efficient to |
|
01eb89b07c13
Don't check for node's existence if we know it exists.
Stefan Seefeld <stefan@seefeld.name>
parents:
4085
diff
changeset
|
1896 # avoid the check if possible. |
| 1165 | 1897 if not self.db.getclass(link_class).hasnode(id): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1898 raise IndexError('%s has no node %s'%(link_class, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1899 id)) |
| 1165 | 1900 # register the link with the newly linked node |
| 1901 if self.do_journal and self.properties[propname].do_journal: | |
| 1902 self.db.addjournal(link_class, id, 'link', | |
| 1903 (self.classname, nodeid, propname)) | |
| 1904 l.append(id) | |
| 1905 add.append(id) | |
| 1906 | |
| 1907 # figure the journal entry | |
| 1908 l = [] | |
| 1909 if add: | |
| 1910 l.append(('+', add)) | |
| 1911 if remove: | |
| 1912 l.append(('-', remove)) | |
| 1913 multilink_changes[propname] = (add, remove) | |
| 1914 if l: | |
| 1915 journalvalues[propname] = tuple(l) | |
| 1916 | |
| 1917 elif isinstance(prop, String): | |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1365
diff
changeset
|
1918 if value is not None and type(value) != type('') and type(value) != type(u''): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1919 raise TypeError('new property "%s" not a string'%propname) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1920 if prop.indexme: |
|
3399
3a0d4e4a0f34
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
1921 if value is None: value = '' |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1922 self.db.indexer.add_text((self.classname, nodeid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
1923 value) |
| 1165 | 1924 |
| 1925 elif isinstance(prop, Password): | |
| 1926 if not isinstance(value, password.Password): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1927 raise TypeError('new property "%s" not a Password'%propname) |
| 1165 | 1928 propvalues[propname] = value |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1929 journalvalues[propname] = \ |
|
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1930 current and password.JournalPassword(current) |
| 1165 | 1931 |
| 1932 elif value is not None and isinstance(prop, Date): | |
| 1933 if not isinstance(value, date.Date): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1934 raise TypeError('new property "%s" not a Date'% propname) |
| 1165 | 1935 propvalues[propname] = value |
| 1936 | |
| 1937 elif value is not None and isinstance(prop, Interval): | |
| 1938 if not isinstance(value, date.Interval): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1939 raise TypeError('new property "%s" not an ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1940 'Interval'%propname) |
| 1165 | 1941 propvalues[propname] = value |
| 1942 | |
| 1943 elif value is not None and isinstance(prop, Number): | |
| 1944 try: | |
| 1945 float(value) | |
| 1946 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1947 raise TypeError('new property "%s" not numeric'%propname) |
| 1165 | 1948 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1949 elif value is not None and isinstance(prop, Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1950 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1951 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1952 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1953 raise TypeError('new property "%s" not integer'%propname) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1954 |
| 1165 | 1955 elif value is not None and isinstance(prop, Boolean): |
| 1956 try: | |
| 1957 int(value) | |
| 1958 except ValueError: | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1959 raise TypeError('new property "%s" not boolean'%propname) |
| 1165 | 1960 |
| 1961 # nothing to do? | |
| 1962 if not propvalues: | |
| 1963 return propvalues | |
| 1964 | |
|
3082
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1965 # update the activity time |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1966 propvalues['activity'] = date.Date() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1967 propvalues['actor'] = self.db.getuid() |
|
29bd9f06160e
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3057
diff
changeset
|
1968 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1969 # do the set |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
1970 self.db.setnode(self.classname, nodeid, propvalues, multilink_changes) |
| 1165 | 1971 |
|
3328
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1972 # remove the activity props now they're handled |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1973 del propvalues['activity'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1974 del propvalues['actor'] |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1975 |
|
475c8560ef9b
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3310
diff
changeset
|
1976 # journal the set |
| 1165 | 1977 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
1978 self.db.addjournal(self.classname, nodeid, ''"set", journalvalues) |
| 1165 | 1979 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
1980 return propvalues |
| 1165 | 1981 |
| 1982 def retire(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1983 """Retire a node. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1984 |
| 1165 | 1985 The properties on the node remain available from the get() method, |
| 1986 and the node's id is never reused. | |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
1987 |
| 1165 | 1988 Retired nodes are not returned by the find(), list(), or lookup() |
| 1989 methods, and other nodes may reuse the values of their key properties. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
1990 """ |
| 1165 | 1991 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
1992 raise DatabaseError(_('Database open read-only')) |
| 1165 | 1993 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1994 self.fireAuditors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
1995 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1996 # use the arg for __retired__ to cope with any odd database type |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1997 # conversion (hello, sqlite) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1998 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
1999 self.db.arg, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2000 self.db.sql(sql, (nodeid, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2001 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
2002 self.db.addjournal(self.classname, nodeid, ''"retired", None) |
| 1165 | 2003 |
|
1345
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2004 self.fireReactors('retire', nodeid, None) |
|
618aa9c37d65
fire auditors and reactors in rdbms retire (thanks Sheila King)
Richard Jones <richard@users.sourceforge.net>
parents:
1333
diff
changeset
|
2005 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2006 def restore(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2007 """Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2008 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2009 Make node available for all operations like it was before retirement. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2010 """ |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2011 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2012 raise DatabaseError(_('Database open read-only')) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2013 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2014 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2015 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2016 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2017 try: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2018 id = self.lookup(node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2019 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2020 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2021 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2022 raise KeyError("Key property (%s) of retired node clashes " |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2023 "with existing one (%s)" % (key, node[key])) |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
2024 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2025 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2026 # use the arg for __retired__ to cope with any odd database type |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2027 # conversion (hello, sqlite) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2028 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2029 self.db.arg, self.db.arg) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2030 self.db.sql(sql, (0, nodeid)) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2031 if self.do_journal: |
|
2546
1b07c9740bba
mark names of journal actions for translation.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2539
diff
changeset
|
2032 self.db.addjournal(self.classname, nodeid, ''"restored", None) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2033 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1515
diff
changeset
|
2034 self.fireReactors('restore', nodeid, None) |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2035 |
| 1165 | 2036 def is_retired(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2037 """Return true if the node is rerired |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2038 """ |
| 1165 | 2039 sql = 'select __retired__ from _%s where id=%s'%(self.classname, |
| 2040 self.db.arg) | |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2041 self.db.sql(sql, (nodeid,)) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2042 return int(self.db.sql_fetchone()[0]) > 0 |
| 1165 | 2043 |
| 2044 def destroy(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2045 """Destroy a node. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2046 |
| 1165 | 2047 WARNING: this method should never be used except in extremely rare |
| 2048 situations where there could never be links to the node being | |
| 2049 deleted | |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2050 |
| 1165 | 2051 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2052 |
| 1165 | 2053 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1988
diff
changeset
|
2054 |
| 1165 | 2055 WARNING: really, use retire() instead |
| 2056 | |
| 2057 Well, I think that's enough warnings. This method exists mostly to | |
| 2058 support the session storage of the cgi interface. | |
| 2059 | |
| 2060 The node is completely removed from the hyperdb, including all journal | |
| 2061 entries. It will no longer be available, and will generally break code | |
| 2062 if there are any references to the node. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2063 """ |
| 1165 | 2064 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2065 raise DatabaseError(_('Database open read-only')) |
| 1165 | 2066 self.db.destroynode(self.classname, nodeid) |
| 2067 | |
| 2068 # Locating nodes: | |
| 2069 def hasnode(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2070 """Determine if the given nodeid actually exists |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2071 """ |
| 1165 | 2072 return self.db.hasnode(self.classname, nodeid) |
| 2073 | |
| 2074 def setkey(self, propname): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2075 """Select a String property of this class to be the key property. |
| 1165 | 2076 |
| 2077 'propname' must be the name of a String property of this class or | |
| 2078 None, or a TypeError is raised. The values of the key property on | |
| 2079 all existing nodes must be unique or a ValueError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2080 """ |
| 1165 | 2081 prop = self.getprops()[propname] |
| 2082 if not isinstance(prop, String): | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2083 raise TypeError('key properties must be String') |
| 1165 | 2084 self.key = propname |
| 2085 | |
| 2086 def getkey(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2087 """Return the name of the key property for this class or None.""" |
| 1165 | 2088 return self.key |
| 2089 | |
| 2090 def lookup(self, keyvalue): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2091 """Locate a particular node by its key property and return its id. |
| 1165 | 2092 |
| 2093 If this class has no key property, a TypeError is raised. If the | |
| 2094 'keyvalue' matches one of the values for the key property among | |
| 2095 the nodes in this class, the matching node's id is returned; | |
| 2096 otherwise a KeyError is raised. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2097 """ |
| 1165 | 2098 if not self.key: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2099 raise TypeError('No key property set for class %s'%self.classname) |
| 1165 | 2100 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2101 # use the arg to handle any odd database type conversion (hello, |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2102 # sqlite) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2103 sql = "select id from _%s where _%s=%s and __retired__=%s"%( |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2104 self.classname, self.key, self.db.arg, self.db.arg) |
|
3977
732a37da3a10
Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479])
Richard Jones <richard@users.sourceforge.net>
parents:
3969
diff
changeset
|
2105 self.db.sql(sql, (str(keyvalue), 0)) |
| 1165 | 2106 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1173
diff
changeset
|
2107 # see if there was a result that's not retired |
|
1203
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
2108 row = self.db.sql_fetchone() |
|
735adcbfc665
fix to SQL lookup() and retirement
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
2109 if not row: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2110 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key, |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2111 keyvalue, self.classname)) |
| 1165 | 2112 |
| 2113 # return the id | |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2114 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2115 return str(row[0]) |
| 1165 | 2116 |
| 2117 def find(self, **propspec): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2118 """Get the ids of nodes in this class which link to the given nodes. |
| 1165 | 2119 |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2120 'propspec' consists of keyword args propname=nodeid or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2121 propname={nodeid:1, } |
| 1165 | 2122 'propname' must be the name of a property in this class, or a |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2123 KeyError is raised. That property must be a Link or |
|
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2124 Multilink property, or a TypeError is raised. |
| 1165 | 2125 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2126 Any node in this class whose 'propname' property links to any of |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2127 the nodeids will be returned. Examples:: |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2128 |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3224
diff
changeset
|
2129 db.issue.find(messages='1') |
| 1165 | 2130 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2131 """ |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2132 # shortcut |
| 1165 | 2133 if not propspec: |
| 2134 return [] | |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2135 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2136 # validate the args |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2137 props = self.getprops() |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2138 for propname, nodeids in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2139 # check the prop is OK |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2140 prop = props[propname] |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2141 if not isinstance(prop, Link) and not isinstance(prop, Multilink): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2142 raise TypeError("'%s' not a Link/Multilink property"%propname) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2143 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2144 # first, links |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2145 a = self.db.arg |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2146 allvalues = () |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2147 sql = [] |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2148 where = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2149 for prop, values in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2150 if not isinstance(props[prop], hyperdb.Link): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2151 continue |
|
1912
2b0ab61db194
fixes for [SF#818339]
Richard Jones <richard@users.sourceforge.net>
parents:
1911
diff
changeset
|
2152 if type(values) is type({}) and len(values) == 1: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2153 values = list(values)[0] |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2154 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2155 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2156 where.append('_%s = %s'%(prop, a)) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
2157 elif values is None: |
|
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1557
diff
changeset
|
2158 where.append('_%s is NULL'%prop) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2159 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2160 values = list(values) |
|
2494
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2161 s = '' |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2162 if None in values: |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2163 values.remove(None) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2164 s = '_%s is NULL or '%prop |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2165 allvalues += tuple(values) |
|
ea7fb2f416db
fixed RDBMS Class.find() to handle None value in multiple find...
Richard Jones <richard@users.sourceforge.net>
parents:
2472
diff
changeset
|
2166 s += '_%s in (%s)'%(prop, ','.join([a]*len(values))) |
|
2530
a182c536b72d
fix filtering by Link property:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2516
diff
changeset
|
2167 where.append('(' + s +')') |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2168 if where: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2169 allvalues = (0, ) + allvalues |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2170 sql.append("""select id from _%s where __retired__=%s |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2171 and %s"""%(self.classname, a, ' and '.join(where))) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2172 |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2173 # now multilinks |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2174 for prop, values in propspec.iteritems(): |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2175 if not isinstance(props[prop], hyperdb.Multilink): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2176 continue |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2177 if not values: |
|
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2178 continue |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2179 allvalues += (0, ) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2180 if type(values) is type(''): |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2181 allvalues += (values,) |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2182 s = a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2183 else: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2184 allvalues += tuple(values) |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1212
diff
changeset
|
2185 s = ','.join([a]*len(values)) |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2186 tn = '%s_%s'%(self.classname, prop) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2187 sql.append("""select id from _%s, %s where __retired__=%s |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2188 and id = %s.nodeid and %s.linkid in (%s)"""%(self.classname, |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2189 tn, a, tn, tn, s)) |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2190 |
|
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2191 if not sql: |
|
1988
5660b89f8903
more compliance testing, this time for find()
Richard Jones <richard@users.sourceforge.net>
parents:
1986
diff
changeset
|
2192 return [] |
|
2733
ef396596a24e
more efficient find() in RDBMS [SF#1012781]
Richard Jones <richard@users.sourceforge.net>
parents:
2731
diff
changeset
|
2193 sql = ' union '.join(sql) |
|
1183
08a13a84ed43
Some speedups - both of the SQL backends can handle using only one cursor.
Richard Jones <richard@users.sourceforge.net>
parents:
1181
diff
changeset
|
2194 self.db.sql(sql, allvalues) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2195 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2196 l = [str(x[0]) for x in self.db.sql_fetchall()] |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2197 return l |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2198 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2199 def stringFind(self, **requirements): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2200 """Locate a particular node by matching a set of its String |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2201 properties in a caseless search. |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2202 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2203 If the property is not a String property, a TypeError is raised. |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2204 |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2205 The return is a list of the id of all nodes that match. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2206 """ |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2207 where = [] |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2208 args = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2209 for propname in requirements: |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2210 prop = self.properties[propname] |
|
1951
767ff2a03eee
more unit tests to improve coverage (up from 85% to 88% for anydbm! :)
Richard Jones <richard@users.sourceforge.net>
parents:
1926
diff
changeset
|
2211 if not isinstance(prop, String): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2212 raise TypeError("'%s' not a String property"%propname) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2213 where.append(propname) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2214 args.append(requirements[propname].lower()) |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2215 |
|
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2216 # generate the where clause |
|
1549
a53a7e197360
fixed rdbms email address lookup (case insensitivity)
Richard Jones <richard@users.sourceforge.net>
parents:
1539
diff
changeset
|
2217 s = ' and '.join(['lower(_%s)=%s'%(col, self.db.arg) for col in where]) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2218 sql = 'select id from _%s where %s and __retired__=%s'%( |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2219 self.classname, s, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2220 args.append(0) |
|
1195
e0032f4ab334
added missing stringFind to sql backends
Richard Jones <richard@users.sourceforge.net>
parents:
1189
diff
changeset
|
2221 self.db.sql(sql, tuple(args)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2222 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2223 l = [str(x[0]) for x in self.db.sql_fetchall()] |
| 1165 | 2224 return l |
| 2225 | |
| 2226 def list(self): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2227 """ Return a list of the ids of the active nodes in this class. |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2228 """ |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2229 return self.getnodeids(retired=0) |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2230 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2231 def getnodeids(self, retired=None): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2232 """ Retrieve all the ids of the nodes for a particular Class. |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2233 |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2234 Set retired=None to get all nodes. Otherwise it'll get all the |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2235 retired or non-retired nodes, depending on the flag. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2236 """ |
|
1707
3d627e34f18e
sqlite backend now passes all tests under 2.3.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1599
diff
changeset
|
2237 # flip the sense of the 'retired' flag if we don't want all of them |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1479
diff
changeset
|
2238 if retired is not None: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2239 args = (0, ) |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2240 if retired: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2241 compare = '>' |
|
1719
eeb167fb8faf
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
1707
diff
changeset
|
2242 else: |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2243 compare = '=' |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2244 sql = 'select id from _%s where __retired__%s%s'%(self.classname, |
|
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2245 compare, self.db.arg) |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2246 else: |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2247 args = () |
|
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2248 sql = 'select id from _%s'%self.classname |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2249 self.db.sql(sql, args) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2250 # XXX numeric ids |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2251 ids = [str(x[0]) for x in self.db.cursor.fetchall()] |
|
1539
800b5896e14a
fixed rdbms export - getnodeids in particular with NULL values
Richard Jones <richard@users.sourceforge.net>
parents:
1528
diff
changeset
|
2252 return ids |
| 1165 | 2253 |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2254 def _subselect(self, classname, multilink_table): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2255 """Create a subselect. This is factored out because some |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2256 databases (hmm only one, so far) doesn't support subselects |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2257 look for "I can't believe it's not a toy RDBMS" in the mysql |
|
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2258 backend. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2259 """ |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2260 return '_%s.id not in (select nodeid from %s)'%(classname, |
|
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2261 multilink_table) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2262 |
|
3685
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2263 # Some DBs order NULL values last. Set this variable in the backend |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2264 # for prepending an order by clause for each attribute that causes |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2265 # correct sort order for NULLs. Examples: |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2266 # order_by_null_values = '(%s is not NULL)' |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2267 # order_by_null_values = 'notnull(%s)' |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2268 # The format parameter is replaced with the attribute. |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2269 order_by_null_values = None |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2270 |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2271 def supports_subselects(self): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2272 '''Assuming DBs can do subselects, overwrite if they cannot. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2273 ''' |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2274 return True |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2275 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2276 def _filter_multilink_expression_fallback( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2277 self, classname, multilink_table, expr): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2278 '''This is a fallback for database that do not support |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2279 subselects.''' |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2280 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2281 is_valid = expr.evaluate |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2282 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2283 last_id, kws = None, [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2284 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2285 ids = IdListOptimizer() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2286 append = ids.append |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2287 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2288 # This join and the evaluation in program space |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2289 # can be expensive for larger databases! |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2290 # TODO: Find a faster way to collect the data needed |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2291 # to evalute the expression. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2292 # Moving the expression evaluation into the database |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2293 # would be nice but this tricky: Think about the cases |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2294 # where the multilink table does not have join values |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2295 # needed in evaluation. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2296 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2297 stmnt = "SELECT c.id, m.linkid FROM _%s c " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2298 "LEFT OUTER JOIN %s m " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2299 "ON c.id = m.nodeid ORDER BY c.id" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2300 classname, multilink_table) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2301 self.db.sql(stmnt) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2302 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2303 # collect all multilink items for a class item |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2304 for nid, kw in self.db.sql_fetchiter(): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2305 if nid != last_id: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2306 if last_id is None: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2307 last_id = nid |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2308 else: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2309 # we have all multilink items -> evaluate! |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2310 if is_valid(kws): append(last_id) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2311 last_id, kws = nid, [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2312 if kw is not None: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2313 kws.append(kw) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2314 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2315 if last_id is not None and is_valid(kws): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2316 append(last_id) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2317 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2318 # we have ids of the classname table |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2319 return ids.where("_%s.id" % classname, self.db.arg) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2320 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2321 def _filter_multilink_expression(self, classname, multilink_table, v): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2322 """ Filters out elements of the classname table that do not |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2323 match the given expression. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2324 Returns tuple of 'WHERE' introns for the overall filter. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2325 """ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2326 try: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2327 opcodes = [int(x) for x in v] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2328 if min(opcodes) >= -1: raise ValueError() |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2329 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2330 expr = compile_expression(opcodes) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2331 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2332 if not self.supports_subselects(): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2333 # We heavily rely on subselects. If there is |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2334 # no decent support fall back to slower variant. |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2335 return self._filter_multilink_expression_fallback( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2336 classname, multilink_table, expr) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2337 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2338 atom = \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2339 "%s IN(SELECT linkid FROM %s WHERE nodeid=a.id)" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2340 self.db.arg, |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2341 multilink_table) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2342 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2343 intron = \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2344 "_%(classname)s.id in (SELECT id " \ |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2345 "FROM _%(classname)s AS a WHERE %(condition)s) " % { |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2346 'classname' : classname, |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2347 'condition' : expr.generate(lambda n: atom) } |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2348 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2349 values = [] |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2350 def collect_values(n): values.append(n.x) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2351 expr.visit(collect_values) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2352 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2353 return intron, values |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2354 except: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2355 # original behavior |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2356 where = "%s.linkid in (%s)" % ( |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2357 multilink_table, ','.join([self.db.arg] * len(v))) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2358 return where, v, True # True to indicate original |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2359 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2360 def _filter_sql (self, search_matches, filterspec, srt=[], grp=[], retr=0): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2361 """ Compute the proptree and the SQL/ARGS for a filter. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2362 For argument description see filter below. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2363 We return a 3-tuple, the proptree, the sql and the sql-args |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2364 or None if no SQL is necessary. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2365 The flag retr serves to retrieve *all* non-Multilink properties |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2366 (for filling the cache during a filter_iter) |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2367 """ |
|
2578
7f25486ff85e
fixed RDBMS filter() for no matches from full-text search [SF#990778]
Richard Jones <richard@users.sourceforge.net>
parents:
2546
diff
changeset
|
2368 # we can't match anything if search_matches is empty |
| 4044 | 2369 if not search_matches and search_matches is not None: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2370 return None |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2371 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2372 icn = self.classname |
| 1165 | 2373 |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2374 # vars to hold the components of the SQL statement |
|
2586
3c67f4bfa225
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
2578
diff
changeset
|
2375 frum = [] # FROM clauses |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2376 loj = [] # LEFT OUTER JOIN clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2377 where = [] # WHERE clauses |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2378 args = [] # *any* positional arguments |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2379 a = self.db.arg |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2380 |
| 1165 | 2381 # figure the WHERE clause from the filterspec |
|
2256
0b198ed096af
fixes for py2.1 (booleans, sigh)
Richard Jones <richard@users.sourceforge.net>
parents:
2248
diff
changeset
|
2382 mlfilt = 0 # are we joining with Multilink tables? |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2383 sortattr = self._sortattr (group = grp, sort = srt) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2384 proptree = self._proptree(filterspec, sortattr, retr) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2385 mlseen = 0 |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2386 for pt in reversed(proptree.sortattr): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2387 p = pt |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2388 while p.parent: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2389 if isinstance (p.propclass, Multilink): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2390 mlseen = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2391 if mlseen: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2392 p.sort_ids_needed = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2393 p.tree_sort_done = False |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2394 p = p.parent |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2395 if not mlseen: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2396 pt.attr_sort_done = pt.tree_sort_done = True |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2397 proptree.compute_sort_done() |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2398 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2399 cols = ['_%s.id'%icn] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2400 mlsort = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2401 rhsnum = 0 |
|
3635
53987aa153d2
Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2402 for p in proptree: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2403 rc = ac = oc = None |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2404 cn = p.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2405 ln = p.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2406 pln = p.parent.uniqname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2407 pcn = p.parent.classname |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2408 k = p.name |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2409 v = p.val |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2410 propclass = p.propclass |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2411 if p.parent == proptree and p.name == 'id' \ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2412 and 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2413 p.sql_idx = 0 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2414 if 'sort' in p.need_for or 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2415 rc = oc = ac = '_%s._%s'%(pln, k) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2416 if isinstance(propclass, Multilink): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2417 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2418 mlfilt = 1 |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2419 tn = '%s_%s'%(pcn, k) |
|
4061
52431f2ad76b
Correct handling of a Multilink filterspec with an empty list of linked-to items.
Stefan Seefeld <stefan@seefeld.name>
parents:
4060
diff
changeset
|
2420 if v in ('-1', ['-1'], []): |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2421 # only match rows that have count(linkid)=0 in the |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2422 # corresponding multilink table) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2423 where.append(self._subselect(pcn, tn)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2424 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2425 frum.append(tn) |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2426 gen_join = True |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2427 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2428 if p.has_values and isinstance(v, type([])): |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2429 result = self._filter_multilink_expression(pln, tn, v) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2430 # XXX: We dont need an id join if we used the filter |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2431 gen_join = len(result) == 3 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2432 |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2433 if gen_join: |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2434 where.append('_%s.id=%s.nodeid'%(pln,tn)) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2435 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2436 if p.children: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2437 frum.append('_%s as _%s' % (cn, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2438 where.append('%s.linkid=_%s.id'%(tn, ln)) |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2439 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2440 if p.has_values: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2441 if isinstance(v, type([])): |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2442 where.append(result[0]) |
|
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4462
diff
changeset
|
2443 args += result[1] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2444 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2445 where.append('%s.linkid=%s'%(tn, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2446 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2447 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2448 assert not p.attr_sort_done and not p.sort_ids_needed |
|
1365
4884fb0860f9
fixed rdbms searching by ID [SF#666615]
Richard Jones <richard@users.sourceforge.net>
parents:
1351
diff
changeset
|
2449 elif k == 'id': |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2450 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2451 if isinstance(v, type([])): |
|
4462
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2452 # If there are no permitted values, then the |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2453 # where clause will always be false, and we |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2454 # can optimize the query away. |
|
0f4a8fcb9a1d
fix SQL generation for empty WHERE clause.
Stefan Seefeld <stefan@seefeld.name>
parents:
4448
diff
changeset
|
2455 if not v: |
|
4803
5e679e364f9a
Bug-fix: need to return None for empty query
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4787
diff
changeset
|
2456 return None |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2457 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2458 where.append('_%s.%s in (%s)'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2459 args = args + v |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2460 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2461 where.append('_%s.%s=%s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2462 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2463 if 'sort' in p.need_for or 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2464 rc = oc = ac = '_%s.id'%pln |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2465 elif isinstance(propclass, String): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2466 if 'search' in p.need_for: |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2467 if not isinstance(v, type([])): |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2468 v = [v] |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2469 |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2470 # Quote special search characters '%' and '_' for |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2471 # correct matching with LIKE/ILIKE |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2472 # Note that we now pass the elements of v as query |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2473 # arguments and don't interpolate the quoted string |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2474 # into the sql statement. Should be safer. |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2475 v = [self.db.search_stringquote(s) for s in v] |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2476 |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2477 # now add to the where clause |
|
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2478 where.append('(' |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2479 +' and '.join(["_%s._%s %s %s ESCAPE %s"%( |
|
4787
4a017661e414
Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents:
4774
diff
changeset
|
2480 pln, |
|
4a017661e414
Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
Bernhard Reiter <bernhard@intevation.de>
parents:
4774
diff
changeset
|
2481 k, |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2482 self.case_insensitive_like, |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2483 a, |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2484 a) for s in v]) |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2485 +')') |
|
4849
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2486 for vv in v: |
|
e68920390aad
Fix SQL wildcards in string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4803
diff
changeset
|
2487 args.extend((vv, '\\')) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2488 if 'sort' in p.need_for: |
|
3683
ad5e1d75e028
Fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3682
diff
changeset
|
2489 oc = ac = 'lower(_%s._%s)'%(pln, k) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2490 elif isinstance(propclass, Link): |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2491 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2492 if p.children: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2493 if 'sort' not in p.need_for: |
|
3694
595041d78104
Fix a peculiarity in sql generation.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3693
diff
changeset
|
2494 frum.append('_%s as _%s' % (cn, ln)) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2495 where.append('_%s._%s=_%s.id'%(pln, k, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2496 if p.has_values: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2497 if isinstance(v, type([])): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2498 d = {} |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2499 for entry in v: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2500 if entry == '-1': |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2501 entry = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2502 d[entry] = entry |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2503 l = [] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2504 if None in d or not d: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2505 if None in d: del d[None] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2506 l.append('_%s._%s is NULL'%(pln, k)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2507 if d: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2508 v = list(d) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2509 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2510 l.append('(_%s._%s in (%s))'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2511 args = args + v |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2512 if l: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2513 where.append('(' + ' or '.join(l) +')') |
|
3636
fa7becc62534
Of course for Links there is the same bug as I fixed before for multilinks.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3635
diff
changeset
|
2514 else: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2515 if v in ('-1', None): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2516 v = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2517 where.append('_%s._%s is NULL'%(pln, k)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2518 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2519 where.append('_%s._%s=%s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2520 args.append(v) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2521 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2522 lp = p.cls.labelprop() |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2523 oc = ac = '_%s._%s'%(pln, k) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2524 if lp != 'id': |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2525 if p.tree_sort_done: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2526 loj.append( |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2527 'LEFT OUTER JOIN _%s as _%s on _%s._%s=_%s.id'%( |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2528 cn, ln, pln, k, ln)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2529 oc = '_%s._%s'%(ln, lp) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2530 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2531 rc = '_%s._%s'%(pln, k) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2532 elif isinstance(propclass, Date) and 'search' in p.need_for: |
|
4060
2a68d7494bbc
Robustify SQL<->HyperDB data type conversion.
Stefan Seefeld <stefan@seefeld.name>
parents:
4059
diff
changeset
|
2533 dc = self.db.to_sql_value(hyperdb.Date) |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2534 if isinstance(v, type([])): |
|
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2535 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2536 where.append('_%s._%s in (%s)'%(pln, k, s)) |
|
3633
a292054b4393
The whole filter method was replicated in back_mysql.py from rdbms_common.py
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3612
diff
changeset
|
2537 args = args + [dc(date.Date(x)) for x in v] |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2538 else: |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2539 try: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2540 # Try to filter on range of dates |
|
3586
f47bddab5a49
date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents:
3554
diff
changeset
|
2541 date_rng = propclass.range_from_raw(v, self.db) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2542 if date_rng.from_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2543 where.append('_%s._%s >= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2544 args.append(dc(date_rng.from_value)) |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2545 if date_rng.to_value: |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2546 where.append('_%s._%s <= %s'%(pln, k, a)) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2547 args.append(dc(date_rng.to_value)) |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2548 except ValueError: |
|
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
2549 # If range creation fails - ignore that search parameter |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2550 pass |
|
1351
d1bfb479e527
fixed searching on date / interval fields [SF#658157]
Richard Jones <richard@users.sourceforge.net>
parents:
1345
diff
changeset
|
2551 elif isinstance(propclass, Interval): |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2552 # filter/sort using the __<prop>_int__ column |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2553 if 'search' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2554 if isinstance(v, type([])): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2555 s = ','.join([a for x in v]) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2556 where.append('_%s.__%s_int__ in (%s)'%(pln, k, s)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2557 args = args + [date.Interval(x).as_seconds() for x in v] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2558 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2559 try: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2560 # Try to filter on range of intervals |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2561 date_rng = Range(v, date.Interval) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2562 if date_rng.from_value: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2563 where.append('_%s.__%s_int__ >= %s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2564 args.append(date_rng.from_value.as_seconds()) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2565 if date_rng.to_value: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2566 where.append('_%s.__%s_int__ <= %s'%(pln, k, a)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2567 args.append(date_rng.to_value.as_seconds()) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2568 except ValueError: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2569 # If range creation fails - ignore search parameter |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2570 pass |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2571 if 'sort' in p.need_for: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2572 oc = ac = '_%s.__%s_int__'%(pln,k) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2573 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2574 rc = '_%s._%s'%(pln,k) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2575 elif isinstance(propclass, Boolean) and 'search' in p.need_for: |
|
4365
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2576 if type(v) == type(""): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2577 v = v.split(',') |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2578 if type(v) != type([]): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2579 v = [v] |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2580 bv = [] |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2581 for val in v: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2582 if type(val) is type(''): |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2583 bv.append(propclass.from_raw (val)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2584 else: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2585 bv.append(bool(val)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2586 if len(bv) == 1: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2587 where.append('_%s._%s=%s'%(pln, k, a)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2588 args = args + bv |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2589 else: |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2590 s = ','.join([a for x in v]) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2591 where.append('_%s._%s in (%s)'%(pln, k, s)) |
|
667c818f6a22
- unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4359
diff
changeset
|
2592 args = args + bv |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2593 elif 'search' in p.need_for: |
| 1165 | 2594 if isinstance(v, type([])): |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2595 s = ','.join([a for x in v]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2596 where.append('_%s._%s in (%s)'%(pln, k, s)) |
| 1165 | 2597 args = args + v |
| 2598 else: | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2599 where.append('_%s._%s=%s'%(pln, k, a)) |
| 1165 | 2600 args.append(v) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2601 if oc: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2602 if p.sort_ids_needed: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2603 if rc == ac: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2604 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2605 p.auxcol = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2606 cols.append(ac) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2607 if p.tree_sort_done and p.sort_direction: |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2608 # Don't select top-level id or multilink twice |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2609 if (not p.sort_ids_needed or ac != oc) and (p.name != 'id' |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2610 or p.parent != proptree): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2611 if rc == oc: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2612 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2613 cols.append(oc) |
|
3685
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2614 desc = ['', ' desc'][p.sort_direction == '-'] |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2615 # Some SQL dbs sort NULL values last -- we want them first. |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2616 if (self.order_by_null_values and p.name != 'id'): |
|
4d9adb8bc3b1
Null-value sorting fixes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3683
diff
changeset
|
2617 nv = self.order_by_null_values % oc |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2618 cols.append(nv) |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2619 p.orderby.append(nv + desc) |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2620 p.orderby.append(oc + desc) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2621 if 'retrieve' in p.need_for and p.sql_idx is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2622 assert(rc) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2623 p.sql_idx = len(cols) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2624 cols.append (rc) |
| 1165 | 2625 |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2626 props = self.getprops() |
|
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2627 |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2628 # don't match retired nodes |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2629 where.append('_%s.__retired__=0'%icn) |
|
1740
5ca448ff8052
don't have RDBMS backends list retired nodes [SF#767319]
Richard Jones <richard@users.sourceforge.net>
parents:
1719
diff
changeset
|
2630 |
| 1165 | 2631 # add results of full text search |
| 2632 if search_matches is not None: | |
| 4044 | 2633 s = ','.join([a for x in search_matches]) |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2634 where.append('_%s.id in (%s)'%(icn, s)) |
| 4113 | 2635 args = args + [x for x in search_matches] |
| 1165 | 2636 |
| 2637 # construct the SQL | |
|
3634
57c66056ffe4
Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3633
diff
changeset
|
2638 frum.append('_'+icn) |
| 1165 | 2639 frum = ','.join(frum) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2640 if where: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2641 where = ' where ' + (' and '.join(where)) |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2642 else: |
|
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2643 where = '' |
|
2240
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2644 if mlfilt: |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2645 # we're joining tables on the id, so we will get dupes if we |
|
ad4b717e12b9
Small optimisation to only use "select distinct(id) ..."...
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2646 # don't distinct() |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2647 cols[0] = 'distinct(_%s.id)'%icn |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2648 |
|
3693
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2649 order = [] |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2650 # keep correct sequence of order attributes. |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2651 for sa in proptree.sortattr: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2652 if not sa.attr_sort_done: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2653 continue |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2654 order.extend(sa.orderby) |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2655 if order: |
|
a775afeeee8b
Database interface fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3688
diff
changeset
|
2656 order = ' order by %s'%(','.join(order)) |
| 1165 | 2657 else: |
| 2658 order = '' | |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2659 |
| 1165 | 2660 cols = ','.join(cols) |
|
2379
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2661 loj = ' '.join(loj) |
|
a2025bdd1491
fix grouping by NULL linked values
Richard Jones <richard@users.sourceforge.net>
parents:
2374
diff
changeset
|
2662 sql = 'select %s from %s %s %s%s'%(cols, frum, loj, where, order) |
| 1165 | 2663 args = tuple(args) |
|
2319
7cf7e3bd1b31
more column uniqueness fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2318
diff
changeset
|
2664 __traceback_info__ = (sql, args) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2665 return proptree, sql, args |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2666 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2667 def filter(self, search_matches, filterspec, sort=[], group=[]): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2668 """Return a list of the ids of the active nodes in this class that |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2669 match the 'filter' spec, sorted by the group spec and then the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2670 sort spec |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2671 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2672 "filterspec" is {propname: value(s)} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2673 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2674 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-' |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2675 or None and prop is a prop name or None. Note that for |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2676 backward-compatibility reasons a single (dir, prop) tuple is |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2677 also allowed. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2678 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2679 "search_matches" is a container type or None |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2680 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2681 The filter must match all properties specificed. If the property |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2682 value to match is a list: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2683 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2684 1. String properties must match all elements in the list, and |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2685 2. Other properties must match any of the elements in the list. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2686 """ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2687 if __debug__: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2688 start_t = time.time() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2689 |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2690 sq = self._filter_sql (search_matches, filterspec, sort, group) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2691 # nothing to match? |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2692 if sq is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2693 return [] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2694 proptree, sql, args = sq |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2695 |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
2696 self.db.sql(sql, args) |
|
1911
f5c804379c85
fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents:
1906
diff
changeset
|
2697 l = self.db.sql_fetchall() |
| 1165 | 2698 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2699 # Compute values needed for sorting in proptree.sort |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2700 for p in proptree: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2701 if hasattr(p, 'auxcol'): |
|
4886
a116de39e38c
Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4849
diff
changeset
|
2702 p.sort_ids = [row[p.auxcol] for row in l] |
|
a116de39e38c
Fix sort-representation for RDBMS backends
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4849
diff
changeset
|
2703 p.sort_result = p._sort_repr (p.propclass.sort_repr, p.sort_ids) |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1168
diff
changeset
|
2704 # return the IDs (the first column) |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
2705 # XXX numeric ids |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2706 l = [str(row[0]) for row in l] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2707 l = proptree.sort (l) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2708 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2709 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2234
diff
changeset
|
2710 self.db.stats['filtering'] += (time.time() - start_t) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3636
diff
changeset
|
2711 return l |
|
1168
94620e088e3a
fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents:
1165
diff
changeset
|
2712 |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2713 def filter_iter(self, search_matches, filterspec, sort=[], group=[]): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2714 """Iterator similar to filter above with same args. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2715 Limitation: We don't sort on multilinks. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2716 This uses an optimisation: We put all nodes that are in the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2717 current row into the node cache. Then we return the node id. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2718 That way a fetch of a node won't create another sql-fetch (with |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2719 a join) from the database because the nodes are already in the |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2720 cache. We're using our own temporary cursor. |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2721 """ |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2722 sq = self._filter_sql(search_matches, filterspec, sort, group, retr=1) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2723 # nothing to match? |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2724 if sq is None: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2725 return |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2726 proptree, sql, args = sq |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2727 cursor = self.db.conn.cursor() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2728 self.db.sql(sql, args, cursor) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2729 classes = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2730 for p in proptree: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2731 if 'retrieve' in p.need_for: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2732 cn = p.parent.classname |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2733 ptid = p.parent.id # not the nodeid! |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2734 key = (cn, ptid) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2735 if key not in classes: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2736 classes[key] = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2737 name = p.name |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2738 assert (name) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2739 classes[key][name] = p |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
2740 p.to_hyperdb = self.db.to_hyperdb_value(p.propclass.__class__) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2741 while True: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2742 row = cursor.fetchone() |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2743 if not row: break |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2744 # populate cache with current items |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2745 for (classname, ptid), pt in classes.iteritems(): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2746 nodeid = str(row[pt['id'].sql_idx]) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2747 key = (classname, nodeid) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2748 if key in self.db.cache: |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2749 self.db._cache_refresh(key) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2750 continue |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2751 node = {} |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2752 for propname, p in pt.iteritems(): |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2753 value = row[p.sql_idx] |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2754 if value is not None: |
|
4473
fccf7e09af0c
- optimisation for date:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4472
diff
changeset
|
2755 value = p.to_hyperdb(value) |
|
4472
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2756 node[propname] = value |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2757 self.db._cache_save(key, node) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2758 yield str(row[0]) |
|
34dce76bb202
Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2759 |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2760 def filter_sql(self, sql): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2761 """Return a list of the ids of the items in this class that match |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2762 the SQL provided. The SQL is a complete "select" statement. |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2763 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2764 The SQL select must include the item id as the first column. |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2765 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2766 This function DOES NOT filter out retired items, add on a where |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2767 clause "__retired__=0" if you don't want retired nodes. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2768 """ |
|
3688
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2769 if __debug__: |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2770 start_t = time.time() |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2771 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2772 self.db.sql(sql) |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2773 l = self.db.sql_fetchall() |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2774 |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2775 if __debug__: |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2776 self.db.stats['filtering'] += (time.time() - start_t) |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2777 return l |
|
722ab52d47fc
added filter_sql to SQL backends which takes an arbitrary SQL statement...
Richard Jones <richard@users.sourceforge.net>
parents:
3687
diff
changeset
|
2778 |
| 1165 | 2779 def count(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2780 """Get the number of nodes in this class. |
| 1165 | 2781 |
| 2782 If the returned integer is 'numnodes', the ids of all the nodes | |
| 2783 in this class run from 1 to numnodes, and numnodes+1 will be the | |
| 2784 id of the next node to be created in this class. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2785 """ |
| 1165 | 2786 return self.db.countnodes(self.classname) |
| 2787 | |
| 2788 # Manipulating properties: | |
| 2789 def getprops(self, protected=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2790 """Return a dictionary mapping property names to property objects. |
| 1165 | 2791 If the "protected" flag is true, we include protected properties - |
| 2792 those which may not be modified. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2793 """ |
| 1165 | 2794 d = self.properties.copy() |
| 2795 if protected: | |
| 2796 d['id'] = String() | |
| 2797 d['creation'] = hyperdb.Date() | |
| 2798 d['activity'] = hyperdb.Date() | |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1175
diff
changeset
|
2799 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2800 d['actor'] = hyperdb.Link('user') |
| 1165 | 2801 return d |
| 2802 | |
| 2803 def addprop(self, **properties): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2804 """Add properties to this class. |
| 1165 | 2805 |
| 2806 The keyword arguments in 'properties' must map names to property | |
| 2807 objects, or a TypeError is raised. None of the keys in 'properties' | |
| 2808 may collide with the names of existing properties, or a ValueError | |
| 2809 is raised before any properties have been added. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2810 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2811 for key in properties: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2812 if key in self.properties: |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2813 raise ValueError(key) |
| 1165 | 2814 self.properties.update(properties) |
| 2815 | |
| 2816 def index(self, nodeid): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2817 """Add (or refresh) the node to search indexes |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2818 """ |
| 1165 | 2819 # find all the String properties that have indexme |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2820 for prop, propclass in self.getprops().iteritems(): |
| 1165 | 2821 if isinstance(propclass, String) and propclass.indexme: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2822 self.db.indexer.add_text((self.classname, nodeid, prop), |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2823 str(self.get(nodeid, prop))) |
| 1165 | 2824 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2825 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2826 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2827 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2828 def export_list(self, propnames, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2829 """ Export a node - generate a list of CSV-able data in the order |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2830 specified by propnames for the given node. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2831 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2832 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2833 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2834 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2835 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2836 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2837 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2838 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2839 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2840 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2841 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2842 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2843 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2844 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2845 value = str(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2846 l.append(repr(value)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2847 l.append(repr(self.is_retired(nodeid))) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2848 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2849 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2850 def import_list(self, propnames, proplist): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2851 """ Import a node - all information including "id" is present and |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2852 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2853 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2854 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2855 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2856 Return the nodeid of the node imported. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2857 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2858 if self.db.journaltag is None: |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2859 raise DatabaseError(_('Database open read-only')) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2860 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2861 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2862 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2863 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2864 retire = 0 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2865 if not "id" in propnames: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2866 newid = self.db.newid(self.classname) |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2867 else: |
|
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2868 newid = eval(proplist[propnames.index("id")]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2869 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2870 # Use eval to reverse the repr() used to output the CSV |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2871 value = eval(proplist[i]) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2872 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2873 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2874 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2875 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2876 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2877 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2878 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2879 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2880 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2881 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2882 retire = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2883 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2884 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2885 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2886 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2887 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2888 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2889 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2890 # don't set Nones |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2891 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2892 elif isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2893 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2894 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2895 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2896 elif isinstance(prop, hyperdb.Password): |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4476
diff
changeset
|
2897 value = password.Password(encrypted=value) |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2898 elif isinstance(prop, String): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2899 if isinstance(value, unicode): |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2900 value = value.encode('utf8') |
|
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2901 if not isinstance(value, str): |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2902 raise TypeError('new property "%(propname)s" not a ' |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2903 'string: %(value)r'%locals()) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2904 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2905 self.db.indexer.add_text((self.classname, newid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2887
diff
changeset
|
2906 value) |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3525
diff
changeset
|
2907 d[propname] = value |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2908 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2909 # get a new id if necessary |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2910 if newid is None: |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2911 newid = self.db.newid(self.classname) |
|
2217
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2912 |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2913 # insert new node or update existing? |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2914 if not self.hasnode(newid): |
|
98d3bf8ffb19
store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents:
2196
diff
changeset
|
2915 self.db.addnode(self.classname, newid, d) # insert |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2916 else: |
|
2516
125311efe783
fix invalid sql produced for multilink condition with empty value list;
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2514
diff
changeset
|
2917 self.db.setnode(self.classname, newid, d) # update |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2918 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2919 # retire? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2920 if retire: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2921 # use the arg for __retired__ to cope with any odd database type |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2922 # conversion (hello, sqlite) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2923 sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2924 self.db.arg, self.db.arg) |
|
3963
3230f9c88086
Fix race condition for key properties in rdbms backends [SF#1876683]
Richard Jones <richard@users.sourceforge.net>
parents:
3933
diff
changeset
|
2925 self.db.sql(sql, (newid, newid)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2926 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2927 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2928 def export_journals(self): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2929 """Export a class's journal - generate a list of lists of |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2930 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2931 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2932 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2933 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2934 No heading here - the columns are fixed. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2935 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2936 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2937 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2938 for nodeid in self.getnodeids(): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2939 for nodeid, date, user, action, params in self.history(nodeid): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2940 date = date.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2941 if action == 'set': |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2942 export_data = {} |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2943 for propname, value in params.iteritems(): |
|
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2944 if propname not in properties: |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2945 # property no longer in the schema |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2946 continue |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2947 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2948 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2949 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2950 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2951 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2952 elif isinstance(prop, Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2953 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2954 elif isinstance(prop, Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2955 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2956 elif isinstance(prop, Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2957 value = str(value) |
|
3365
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2958 export_data[propname] = value |
|
e2d65f6c8d83
handle dropped properies in rdbms/metakit journal export [SF#1203569]
Richard Jones <richard@users.sourceforge.net>
parents:
3328
diff
changeset
|
2959 params = export_data |
|
3933
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2960 elif action == 'create' and params: |
|
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2961 # old tracker with data stored in the create! |
|
9095a4da67f9
Handle export and import of old trackers that have data attached to...
Richard Jones <richard@users.sourceforge.net>
parents:
3925
diff
changeset
|
2962 params = {} |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2963 l = [nodeid, date, user, action, params] |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2964 r.append(list(map(repr, l))) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2965 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2166
diff
changeset
|
2966 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2586
diff
changeset
|
2967 class FileClass(hyperdb.FileClass, Class): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2968 """This class defines a large chunk of data. To support this, it has a |
| 1165 | 2969 mandatory String property "content" which is typically saved off |
| 2970 externally to the hyperdb. | |
| 2971 | |
| 2972 The default MIME type of this data is defined by the | |
| 2973 "default_mime_type" class attribute, which may be overridden by each | |
| 2974 node if the class defines a "type" String property. | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2975 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2976 def __init__(self, db, classname, **properties): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2977 """The newly-created class automatically includes the "content" |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2978 and "type" properties. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2979 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2980 if 'content' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2981 properties['content'] = hyperdb.String(indexme='yes') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
2982 if 'type' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2983 properties['type'] = hyperdb.String() |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
2984 Class.__init__(self, db, classname, **properties) |
| 1165 | 2985 |
| 2986 def create(self, **propvalues): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2987 """ snaffle the file propvalue and store in a file |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
2988 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2989 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2990 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2991 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2992 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2993 # now remove the content property so it's not stored in the db |
| 1165 | 2994 content = propvalues['content'] |
| 2995 del propvalues['content'] | |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2996 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
2997 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2998 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2999 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3000 # figure the mime type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3001 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3002 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3003 # and index! |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3004 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3005 self.db.indexer.add_text((self.classname, newid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3006 content, mime_type) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3007 |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3008 # store off the content as a file |
|
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3009 self.db.storefile(self.classname, newid, None, content) |
|
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3977
diff
changeset
|
3010 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3011 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3012 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1417
diff
changeset
|
3013 |
| 1165 | 3014 return newid |
| 3015 | |
| 3016 def get(self, nodeid, propname, default=_marker, cache=1): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3017 """ Trap the content propname and get it from the file |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3018 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3019 'cache' exists for backwards compatibility, and is not used. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3020 """ |
| 1165 | 3021 poss_msg = 'Possibly a access right configuration problem.' |
| 3022 if propname == 'content': | |
| 3023 try: | |
| 3024 return self.db.getfile(self.classname, nodeid, None) | |
|
4410
de00a238a9ad
remove unnecessary brackets
Richard Jones <richard@users.sourceforge.net>
parents:
4365
diff
changeset
|
3025 except IOError, strerror: |
| 1165 | 3026 # BUG: by catching this we donot see an error in the log. |
| 3027 return 'ERROR reading file: %s%s\n%s\n%s'%( | |
| 3028 self.classname, nodeid, poss_msg, strerror) | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
3029 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3030 return Class.get(self, nodeid, propname, default) |
| 1165 | 3031 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
3032 return Class.get(self, nodeid, propname) |
| 1165 | 3033 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3034 def set(self, itemid, **propvalues): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3035 """ Snarf the "content" propvalue and update it in a file |
|
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3036 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3037 self.fireAuditors('set', itemid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3038 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
| 1165 | 3039 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3040 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3041 content = None |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3042 if 'content' in propvalues: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3043 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3044 del propvalues['content'] |
| 1165 | 3045 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3046 # do the database create |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3047 propvalues = self.set_inner(itemid, **propvalues) |
| 1165 | 3048 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3049 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3050 if content: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3051 # store and possibly index |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3052 self.db.storefile(self.classname, itemid, None, content) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3053 if self.properties['content'].indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3054 mime_type = self.get(itemid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3055 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3056 content, mime_type) |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
3057 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2634
diff
changeset
|
3058 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3059 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3060 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
3061 return propvalues |
| 1165 | 3062 |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
3063 def index(self, nodeid): |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3064 """ Add (or refresh) the node to search indexes. |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3065 |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3066 Use the content-type property for the content property. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3067 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3068 # find all the String properties that have indexme |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3069 for prop, propclass in self.getprops().iteritems(): |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3070 if prop == 'content' and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3071 mime_type = self.get(nodeid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3072 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3073 str(self.get(nodeid, 'content')), mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3074 elif isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3075 # index them under (classname, nodeid, property) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3076 try: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3077 value = str(self.get(nodeid, prop)) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3078 except IndexError: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3079 # node has been destroyed |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3080 continue |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
3081 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
3082 |
| 1165 | 3083 # XXX deviation from spec - was called ItemClass |
| 3084 class IssueClass(Class, roundupdb.IssueClass): | |
| 3085 # Overridden methods: | |
| 3086 def __init__(self, db, classname, **properties): | |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3087 """The newly-created class automatically includes the "messages", |
| 1165 | 3088 "files", "nosy", and "superseder" properties. If the 'properties' |
| 3089 dictionary attempts to specify any of these properties or a | |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
3090 "creation", "creator", "activity" or "actor" property, a ValueError |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
3091 is raised. |
|
3834
8068eb6c704e
Use """ instead of ''' for multi-line strings...
Erik Forsberg <forsberg@users.sourceforge.net>
parents:
3821
diff
changeset
|
3092 """ |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3093 if 'title' not in properties: |
| 1165 | 3094 properties['title'] = hyperdb.String(indexme='yes') |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3095 if 'messages' not in properties: |
| 1165 | 3096 properties['messages'] = hyperdb.Multilink("msg") |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3097 if 'files' not in properties: |
| 1165 | 3098 properties['files'] = hyperdb.Multilink("file") |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3099 if 'nosy' not in properties: |
| 1165 | 3100 # note: journalling is turned off as it really just wastes |
| 3101 # space. this behaviour may be overridden in an instance | |
| 3102 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
|
4359
b9abbdd15259
another module modernised
Richard Jones <richard@users.sourceforge.net>
parents:
4113
diff
changeset
|
3103 if 'superseder' not in properties: |
| 1165 | 3104 properties['superseder'] = hyperdb.Multilink(classname) |
| 3105 Class.__init__(self, db, classname, **properties) | |
| 3106 | |
|
2692
f1c9873496f0
fix Class.get(): it was relying on self._marker...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2691
diff
changeset
|
3107 # vim: set et sts=4 sw=4 : |