Mercurial > p > roundup > code

annotate roundup/hyperdb.py @ 6238:6834bb5473da

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Summary: Constrain format of classname and document Define the format of a valid classname. Updated design doc, function doc and glossary to document format. Error message for case where we have a redefinition of a classname now says why ValueError is raised since we are raising ValueError for incorrect classname format as well. Tests for all cases including verification of cause for ValueError exceptions.
author John Rouillard <rouilj@ieee.org>
date Tue, 28 Jul 2020 06:24:39 -0400
parents f40c6b5de370
children 95183d73ac64
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
1 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
4 # under the same terms as Python, so long as this copyright message and
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
5 # disclaimer are retained in their original form.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
6 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
11 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 188
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
17 #
406
bdc2ea127ae9 Added module docstrings to all modules.
Jürgen Hermann <jhermann@users.sourceforge.net>
parents: 398
diff changeset
18
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
19 """Hyperdatabase implementation, especially field types.
1244
8dd4f736370b merge from maintenance branch
Richard Jones <richard@users.sourceforge.net>
parents: 1085
diff changeset
20 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
21 __docformat__ = 'restructuredtext'
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 # standard python modules
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
24 import os, re, shutil, sys, weakref
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
25 import traceback
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
26 import logging
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 # roundup modules
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
29 from . import date, password
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
30 from .support import ensureParentsExist, PrioList
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
31 from roundup.i18n import _
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
32 from roundup.cgi.exceptions import DetectorError
5481
9a09719b0d8e helper to allow comparing dicts and None values in Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5466
diff changeset
33 from roundup.anypy.cmp_ import NoneAndDictComparable
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5481
diff changeset
34 from roundup.anypy.strings import eval_import
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
35
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
36 logger = logging.getLogger('roundup.hyperdb')
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
37
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
38
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 #
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 # Types
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 #
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
42 class _Type(object):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
43 """A roundup property type."""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
44 def __init__(self, required=False, default_value=None, quiet=False):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
45 self.required = required
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
46 self.__default_value = default_value
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5101
diff changeset
47 self.quiet = quiet
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
48 # We do not allow updates if self.computed is True
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
49 # For now only Multilinks (using the rev_multilink) can be computed
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
50 self.computed = False
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
51
224
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
52 def __repr__(self):
657
e0a1cc7538e9 more pychecker warnings removed
Richard Jones <richard@users.sourceforge.net>
parents: 648
diff changeset
53 ' more useful for dumps '
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
54 return '<%s.%s>' % (self.__class__.__module__, self.__class__.__name__)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
55
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
56 def get_default_value(self):
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
57 """The default value when creating a new instance of this property."""
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
58 return self.__default_value
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
59
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
60 def register (self, cls, propname):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
61 """Register myself to the class of which we are a property
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
62 the given propname is the name we have in our class.
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
63 """
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
64 assert not getattr(self, 'cls', None)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
65 self.name = propname
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
66 self.cls = cls
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
67
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
68 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
69 """Representation used for sorting. This should be a python
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
70 built-in type, otherwise sorting will take ages. Note that
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
71 individual backends may chose to use something different for
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
72 sorting as long as the outcome is the same.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
73 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
74 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
75
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
76
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
77 class String(_Type):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
78 """An object designating a String property."""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
79 def __init__(self, indexme='no', required=False, default_value="",
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
80 quiet=False):
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5101
diff changeset
81 super(String, self).__init__(required, default_value, quiet)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
82 self.indexme = indexme == 'yes'
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
83
3992
fe2af84a5ca5 allow binary data for "content" props through rawToHyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
84 def from_raw(self, value, propname='', **kw):
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
85 """fix the CRLF/CR -> LF stuff"""
3992
fe2af84a5ca5 allow binary data for "content" props through rawToHyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
86 if propname == 'content':
fe2af84a5ca5 allow binary data for "content" props through rawToHyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
87 # Why oh why wasn't the FileClass content property a File
fe2af84a5ca5 allow binary data for "content" props through rawToHyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
88 # type from the beginning?
fe2af84a5ca5 allow binary data for "content" props through rawToHyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
89 return value
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
90 return fixNewlines(value)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
91
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
92 def sort_repr(self, cls, val, name):
3684
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
93 if not val:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
94 return val
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
95 if name == 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
96 return int(val)
3684
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3682
diff changeset
97 return val.lower()
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
99
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
100 class Password(_Type):
270
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 264
diff changeset
101 """An object designating a Password property."""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
102 def __init__(self, scheme=None, required=False, default_value=None,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
103 quiet=False):
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5101
diff changeset
104 super(Password, self).__init__(required, default_value, quiet)
5053
9792b18e0b19 issue 2550880: Ability to choose password store scheme and SSHA support.
John Rouillard <rouilj@ieee.org>
parents: 4995
diff changeset
105 self.scheme = scheme
9792b18e0b19 issue 2550880: Ability to choose password store scheme and SSHA support.
John Rouillard <rouilj@ieee.org>
parents: 4995
diff changeset
106
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
107 def from_raw(self, value, **kw):
3668
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
108 if not value:
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
109 return None
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
110 try:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
111 return password.Password(encrypted=value, scheme=self.scheme,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
112 strict=True)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
113 except password.PasswordValueError as message:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
114 raise HyperdbValueError(_('property %s: %s') %
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
115 (kw['propname'], message))
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
116
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
117 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
118 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
119 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
120 return str(val)
270
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 264
diff changeset
121
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
122
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
123 class Date(_Type):
224
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
124 """An object designating a Date property."""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
125 def __init__(self, offset=None, required=False, default_value=None,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
126 quiet=False):
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
127 super(Date, self).__init__(required=required,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
128 default_value=default_value,
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5101
diff changeset
129 quiet=quiet)
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
130 self._offset = offset
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
131
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
132 def offset(self, db):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
133 if self._offset is not None:
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
134 return self._offset
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
135 return db.getUserTimezone()
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
136
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
137 def from_raw(self, value, db, **kw):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
138 try:
3621
77ed6c517793 timezone support (patch [SF#1465296])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3601
diff changeset
139 value = date.Date(value, self.offset(db))
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
140 except ValueError as message:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
141 raise HyperdbValueError(_('property %s: %r is an invalid '
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
142 'date (%s)') % (kw['propname'],
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
143 value, message))
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
144 return value
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
145
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3545
diff changeset
146 def range_from_raw(self, value, db):
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
147 """return Range value from given raw value with offset correction"""
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3545
diff changeset
148 return date.Range(value, date.Date, offset=self.offset(db))
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
149
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
150 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
151 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
152 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
153 return str(val)
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
155
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
156 class Interval(_Type):
224
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
157 """An object designating an Interval property."""
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
158 def from_raw(self, value, **kw):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
159 try:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
160 value = date.Interval(value)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
161 except ValueError as message:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
162 raise HyperdbValueError(_('property %s: %r is an invalid '
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
163 'date interval (%s)') %
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
164 (kw['propname'], value, message))
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
165 return value
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
166
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
167 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
168 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
169 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
170 return val.as_seconds()
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
172
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
173 class _Pointer(_Type):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
174 """An object designating a Pointer property that links or multilinks
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
175 to a node in a specified class."""
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
176 def __init__(self, classname, do_journal='yes', try_id_parsing='yes',
5063
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
177 required=False, default_value=None,
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
178 msg_header_property=None, quiet=False, rev_multilink=None):
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
179 """ Default is to journal link and unlink events.
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
180 When try_id_parsing is false, we don't allow IDs in input
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
181 fields (the key of the Link or Multilink property must be
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
182 given instead). This is useful when the name of a property
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
183 can be numeric. It will only work if the linked item has a
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
184 key property and is a questionable feature for multilinks.
5063
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
185 The msg_header_property is used in the mail gateway when
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
186 sending out messages: By default roundup creates headers of
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
187 the form: 'X-Roundup-issue-prop: value' for all properties
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
188 prop of issue that have a 'name' property. This definition
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
189 allows to override the 'name' property. A common use-case is
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
190 adding a mail-header with the assigned_to property to allow
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
191 user mail-filtering of issue-emails for which they're
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
192 responsible. In that case setting
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
193 'msg_header_property="username"' for the assigned_to
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
194 property will generated message headers of the form:
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
195 'X-Roundup-issue-assigned_to: joe_user'.
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
196 The rev_multilink is used to inject a reverse multilink into
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
197 the Class linked by a Link or Multilink property. Note that
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
198 the result is always a Multilink. The name given with
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
199 rev_multilink is the name in the class where it is injected.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
200 """
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5101
diff changeset
201 super(_Pointer, self).__init__(required, default_value, quiet)
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202 self.classname = classname
555
5fd94347c6f2 Journal entries for link and multilink properties can now be switched on or off.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 543
diff changeset
203 self.do_journal = do_journal == 'yes'
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
204 self.try_id_parsing = try_id_parsing == 'yes'
5063
2840a9e86ef2 New Link/Multilink attribute 'msg_header_property'
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5053
diff changeset
205 self.msg_header_property = msg_header_property
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
206 self.rev_multilink = rev_multilink
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
207
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 def __repr__(self):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
209 """more useful for dumps. But beware: This is also used in schema
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
210 storage in SQL backends!
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
211 """
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
212 return '<%s.%s to "%s">' % (self.__class__.__module__,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
213 self.__class__.__name__, self.classname)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
214
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
215
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
216 class Link(_Pointer):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
217 """An object designating a Link property that links to a
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
218 node in a specified class."""
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
219 def from_raw(self, value, db, propname, **kw):
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
220 if (self.try_id_parsing and value == '-1') or not value:
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
221 value = None
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
222 else:
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
223 if self.try_id_parsing:
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
224 value = convertLinkValue(db, propname, self, value)
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
225 else:
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
226 value = convertLinkValue(db, propname, self, value, None)
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
227 return value
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
228
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
229 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
230 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
231 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
232 op = cls.labelprop()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
233 if op == 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
234 return int(cls.get(val, op))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
235 return cls.get(val, op)
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
237
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
238 class Multilink(_Pointer):
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 """An object designating a Multilink property that links
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 to nodes in a specified class.
557
dfac856502d1 brief docco on the do_journal argument
Richard Jones <richard@users.sourceforge.net>
parents: 555
diff changeset
241
dfac856502d1 brief docco on the do_journal argument
Richard Jones <richard@users.sourceforge.net>
parents: 555
diff changeset
242 "classname" indicates the class to link to
dfac856502d1 brief docco on the do_journal argument
Richard Jones <richard@users.sourceforge.net>
parents: 555
diff changeset
243
dfac856502d1 brief docco on the do_journal argument
Richard Jones <richard@users.sourceforge.net>
parents: 555
diff changeset
244 "do_journal" indicates whether the linked-to nodes should have
dfac856502d1 brief docco on the do_journal argument
Richard Jones <richard@users.sourceforge.net>
parents: 555
diff changeset
245 'link' and 'unlink' events placed in their journal
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
246 "rev_property" is used when injecting reverse multilinks. By
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
247 default (for a normal multilink) the table name is
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
248 <name_of_linking_class>_<name_of_link_property>
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
249 e.g. for the messages multilink in issue in the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
250 classic schema it would be "issue_messages". The
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
251 multilink table in that case has two columns, the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
252 nodeid contains the ID of the linking class while
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
253 the linkid contains the ID of the linked-to class.
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
254 When injecting backlinks, for a backlink resulting
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
255 from a Link or Multilink the table_name,
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
256 linkid_name, and nodeid_name must be explicitly
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
257 specified. So when specifying a rev_multilink
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
258 property for the messages attribute in the example
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
259 above, we would get 'issue_messages' for the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
260 table_name, 'nodeid' for the linkid_name and
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
261 'linkid' for the nodeid_name (note the reversal).
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
262 For a rev_multilink resulting, e.g. from the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
263 standard 'status' Link in the Class 'issue' in the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
264 classic template we would set table_name to '_issue'
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
265 (table names in the database get a leading
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
266 underscore), the nodeid_name to 'status' and the
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
267 linkid_name to 'id'. With these settings we can use
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
268 the standard query engine (with minor modifications
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
269 for the computed names) to resolve reverse
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
270 multilinks.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 """
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
272
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
273 def __init__(self, classname, do_journal='yes', required=False,
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
274 quiet=False, try_id_parsing='yes', rev_multilink=None,
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
275 rev_property=None):
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
276
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
277 super(Multilink, self).__init__(classname,
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
278 do_journal,
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
279 required=required,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
280 default_value=[], quiet=quiet,
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
281 try_id_parsing=try_id_parsing,
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
282 rev_multilink=rev_multilink)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
283 self.rev_property = rev_property
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
284 self.rev_classname = None
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
285 self.rev_propname = None
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
286 self.table_name = None # computed in 'register' below
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
287 self.linkid_name = 'linkid'
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
288 self.nodeid_name = 'nodeid'
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
289 if self.rev_property:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
290 # Do not allow updates if this is a reverse multilink
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
291 self.computed = True
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
292 self.rev_classname = rev_property.cls.classname
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
293 self.rev_propname = rev_property.name
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
294 if isinstance(self.rev_property, Link):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
295 self.table_name = '_' + self.rev_classname
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
296 self.linkid_name = 'id'
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
297 self.nodeid_name = '_' + self.rev_propname
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
298 else:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
299 self.table_name = self.rev_classname + '_' + self.rev_propname
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
300 self.linkid_name = 'nodeid'
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
301 self.nodeid_name = 'linkid'
4481
9bbf3758c16a Allow default value declaration in DB schema.
Stefan Seefeld <stefan@seefeld.name>
parents: 4480
diff changeset
302
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
303 def from_raw(self, value, db, klass, propname, itemid, **kw):
3668
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
304 if not value:
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
305 return []
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
306
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
307 # get the current item value if it's not a new item
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
308 if itemid and not itemid.startswith('-'):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
309 curvalue = klass.get(itemid, propname)
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
310 else:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
311 curvalue = []
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
312
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
313 # if the value is a comma-separated string then split it now
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
314 if isinstance(value, type('')):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
315 value = value.split(',')
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
316
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
317 # handle each add/remove in turn
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
318 # keep an extra list for all items that are
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
319 # definitely in the new list (in case of e.g.
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
320 # <propname>=A,+B, which should replace the old
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
321 # list with A,B)
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4063
diff changeset
322 do_set = 1
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
323 newvalue = []
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
324 for item in value:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
325 item = item.strip()
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
326
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
327 # skip blanks
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
328 if not item: continue
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
329
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
330 # handle +/-
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
331 remove = 0
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
332 if item.startswith('-'):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
333 remove = 1
5069
2706b66675bf issue2550763 Strip whitespace from Multilink values after + or -.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
334 item = item[1:].strip()
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4063
diff changeset
335 do_set = 0
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
336 elif item.startswith('+'):
5069
2706b66675bf issue2550763 Strip whitespace from Multilink values after + or -.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
337 item = item[1:].strip()
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4063
diff changeset
338 do_set = 0
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
339
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
340 # look up the value
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
341 if self.try_id_parsing:
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
342 itemid = convertLinkValue(db, propname, self, item)
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
343 else:
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
344 itemid = convertLinkValue(db, propname, self, item, None)
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
345
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
346 # perform the add/remove
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
347 if remove:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
348 try:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
349 curvalue.remove(itemid)
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
350 except ValueError:
5548
fea11d05110e Avoid errors from selecting "no selection" on multilink (issue2550722).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5527
diff changeset
351 # This can occur if the edit adding the element
fea11d05110e Avoid errors from selecting "no selection" on multilink (issue2550722).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5527
diff changeset
352 # produced an error, so the form has it in the
fea11d05110e Avoid errors from selecting "no selection" on multilink (issue2550722).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5527
diff changeset
353 # "no selection" choice but it's not set in the
fea11d05110e Avoid errors from selecting "no selection" on multilink (issue2550722).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5527
diff changeset
354 # database.
fea11d05110e Avoid errors from selecting "no selection" on multilink (issue2550722).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5527
diff changeset
355 pass
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
356 else:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
357 newvalue.append(itemid)
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
358 if itemid not in curvalue:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
359 curvalue.append(itemid)
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
360
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
361 # that's it, set the new Multilink property value,
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
362 # or overwrite it completely
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4063
diff changeset
363 if do_set:
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
364 value = newvalue
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
365 else:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
366 value = curvalue
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
367
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
368 # TODO: one day, we'll switch to numeric ids and this will be
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
369 # unnecessary :(
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
370 value = [int(x) for x in value]
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
371 value.sort()
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
372 value = [str(x) for x in value]
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
373 return value
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
375 def register(self, cls, propname):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
376 super(Multilink, self).register(cls, propname)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
377 if self.table_name is None:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
378 self.table_name = self.cls.classname + '_' + self.name
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
379
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
380 def sort_repr(self, cls, val, name):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
381 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
382 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
383 op = cls.labelprop()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
384 if op == 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
385 return [int(cls.get(v, op)) for v in val]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
386 return [cls.get(v, op) for v in val]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
387
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
388
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
389 class Boolean(_Type):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 858
diff changeset
390 """An object designating a boolean property"""
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
391 def from_raw(self, value, **kw):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
392 value = value.strip()
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
393 # checked is a common HTML checkbox value
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
394 value = value.lower() in ('checked', 'yes', 'true', 'on', '1')
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
395 return value
2603
5ccd99777869 fix metakit handling of filter on Link==None; fix some unit tests
Richard Jones <richard@users.sourceforge.net>
parents: 2571
diff changeset
396
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
397
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
398 class Number(_Type):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 858
diff changeset
399 """An object designating a numeric property"""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
400 def __init__(self, use_double=False, **kw):
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
401 """ The value use_double tells the database backend to use a
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
402 floating-point format with more precision than the default.
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
403 Usually implemented by type 'double precision' in the sql
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
404 backend. The default is to use single-precision float (aka
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
405 'real') in the db. Note that sqlite already uses 8-byte for
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
406 floating point numbers.
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
407 """
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
408 self.use_double = use_double
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5169
diff changeset
409 super(Number, self).__init__(**kw)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
410
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
411 def from_raw(self, value, **kw):
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
412 value = value.strip()
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
413 try:
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
414 value = float(value)
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
415 except ValueError:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
416 raise HyperdbValueError(_('property %s: %r is not a number') %
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
417 (kw['propname'], value))
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
418 return value
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
419
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
420
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
421 class Integer(_Type):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
422 """An object designating an integer property"""
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
423 def from_raw(self, value, **kw):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
424 value = value.strip()
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
425 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
426 value = int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
427 except ValueError:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
428 raise HyperdbValueError(_('property %s: %r is not an integer') %
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
429 (kw['propname'], value))
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5063
diff changeset
430 return value
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
431
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
432
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
433 #
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
434 # Support for splitting designators
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
435 #
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
436 class DesignatorError(ValueError):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
437 pass
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
438
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
439
6238
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
440 def splitDesignator(designator,
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
441 dre=re.compile(r'^([A-Za-z](?:[A-Za-z_0-9]*[A-Za-z_]+)?)(\d+)$')):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
442 """ Take a foo123 and return ('foo', 123)
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
443 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
444 m = dre.match(designator)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
445 if m is None:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
446 raise DesignatorError(_('"%s" not a node designator') % designator)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
447 return m.group(1), m.group(2)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
448
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
449
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
450 class Exact_Match(object):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
451 """ Used to encapsulate exact match semantics search values
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
452 """
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
453 def __init__(self, value):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
454 self.value = value
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
455
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
456
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
457 class Proptree(object):
4935
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
458 """ Simple tree data structure for property lookup. Each node in
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
459 the tree is a roundup Class Property that has to be navigated to
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
460 find given property. The need_for attribute is used to mark nodes
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
461 that are used for sorting, searching or retrieval: The attribute
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
462 is a dictionary containing one or several of the values 'sort',
adb8b787e157 Attempt to clarify hyperdb.Proptree meaning
anatoly techtonik <techtonik@gmail.com>
parents: 4877
diff changeset
463 'search', 'retrieve'.
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
464
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
465 The Proptree is also used for transitively searching attributes for
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
466 backends that do not support transitive search (e.g. anydbm). The
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
467 _val attribute with set_val is used for this.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
468 """
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
469
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
470 def __init__(self, db, cls, name, props, parent=None, retr=False):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
471 self.db = db
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
472 self.name = name
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
473 self.props = props
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
474 self.parent = parent
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
475 self._val = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
476 self.has_values = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
477 self.cls = cls
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
478 self.classname = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
479 self.uniqname = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
480 self.children = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
481 self.sortattr = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
482 self.propdict = {}
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
483 self.need_for = {'search': True}
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
484 self.sort_direction = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
485 self.sort_ids = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
486 self.sort_ids_needed = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
487 self.sort_result = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
488 self.attr_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
489 self.tree_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
490 self.propclass = None
3695
01ea89743311 Add an attribute for sort-order fix in rdbms backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3687
diff changeset
491 self.orderby = []
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
492 self.sql_idx = None # index of retrieved column in sql result
6179
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
493 self.need_retired = False
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
494 self.need_child_retired = False
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
495 if parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
496 self.root = parent.root
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
497 self.depth = parent.depth + 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
498 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
499 self.root = self
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
500 self.seqno = 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
501 self.depth = 0
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
502 self.need_for['sort'] = True
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
503 self.id = self.root.seqno
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
504 self.root.seqno += 1
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
505 if self.cls:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
506 self.classname = self.cls.classname
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
507 self.uniqname = '%s%s' % (self.cls.classname, self.id)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
508 if not self.parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
509 self.uniqname = self.cls.classname
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
510 if retr:
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
511 self.append_retr_props()
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
512
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
513 def append(self, name, need_for='search', retr=False):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
514 """Append a property to self.children. Will create a new
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
515 propclass for the child.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
516 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
517 if name in self.propdict:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
518 pt = self.propdict[name]
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
519 pt.need_for[need_for] = True
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
520 if retr and isinstance(pt.propclass, Link):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
521 pt.append_retr_props()
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
522 return pt
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
523 propclass = self.props[name]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
524 cls = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
525 props = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
526 if isinstance(propclass, (Link, Multilink)):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
527 cls = self.db.getclass(propclass.classname)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
528 props = cls.getprops()
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
529 child = self.__class__(self.db, cls, name, props, parent=self)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
530 child.need_for = {need_for: True}
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
531 child.propclass = propclass
6179
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
532 if isinstance(propclass, Multilink) and self.props[name].computed:
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
533 if isinstance(self.props[name].rev_property, Link):
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
534 child.need_retired = True
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
535 else:
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
536 child.need_child_retired = True
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
537 self.children.append(child)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
538 self.propdict[name] = child
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
539 if retr and isinstance(child.propclass, Link):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
540 child.append_retr_props()
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
541 return child
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
542
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
543 def append_retr_props(self):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
544 """Append properties for retrieval."""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
545 for name, prop in self.cls.getprops(protected=1).items():
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
546 if isinstance(prop, Multilink):
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
547 continue
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
548 self.append(name, need_for='retrieve')
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
549
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
550 def compute_sort_done(self, mlseen=False):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
551 """ Recursively check if attribute is needed for sorting
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
552 ('sort' in self.need_for) or all children have tree_sort_done set and
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
553 sort_ids_needed unset: set self.tree_sort_done if one of the conditions
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
554 holds. Also remove sort_ids_needed recursively once having seen a
4850
6998ad77841e Fix bug in SQL generation
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4687
diff changeset
555 Multilink that is used for sorting.
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
556 """
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
557 if isinstance(self.propclass, Multilink) and 'sort' in self.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
558 mlseen = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
559 if mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
560 self.sort_ids_needed = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
561 self.tree_sort_done = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
562 for p in self.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
563 p.compute_sort_done(mlseen)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
564 if not p.tree_sort_done:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
565 self.tree_sort_done = False
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
566 if 'sort' not in self.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
567 self.tree_sort_done = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
568 if mlseen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
569 self.tree_sort_done = False
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
570
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
571 def ancestors(self):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
572 p = self
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
573 while p.parent:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
574 yield p
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
575 p = p.parent
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
576
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5315
diff changeset
577 def search(self, search_matches=None, sort=True, retired=False):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
578 """ Recursively search for the given properties in a proptree.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
579 Once all properties are non-transitive, the search generates a
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
580 simple _filter call which does the real work
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
581 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
582 filterspec = {}
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
583 exact_match_spec = {}
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
584 for p in self.children:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
585 if 'search' in p.need_for:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
586 if p.children:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
587 p.search(sort=False)
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
588 if getattr(p.propclass,'rev_property',None):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
589 pn = p.propclass.rev_property.name
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
590 cl = p.propclass.rev_property.cls
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
591 if not isinstance(p.val, type([])):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
592 p.val = [p.val]
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
593 if p.val == ['-1'] :
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
594 s1 = set(self.cls.getnodeids(retired=False))
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
595 s2 = set()
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
596 for id in cl.getnodeids(retired=False):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
597 node = cl.getnode(id)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
598 if node[pn]:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
599 if isinstance(node [pn], type([])):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
600 s2.update(node [pn])
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
601 else:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
602 s2.add(node [pn])
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
603 items = s1.difference(s2)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
604 elif isinstance(p.propclass.rev_property, Link):
6179
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
605 items = set(cl.get(x, pn) for x in p.val
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
606 if not cl.is_retired(x))
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
607 else:
6179
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
608 items = set().union(*(cl.get(x, pn) for x in p.val
a701c9c81597 Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6148
diff changeset
609 if not cl.is_retired(x)))
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
610 filterspec[p.name] = list(sorted(items))
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
611 elif isinstance(p.val, type([])):
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
612 exact = []
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
613 subst = []
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
614 for v in p.val:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
615 if isinstance(v, Exact_Match):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
616 exact.append(v.value)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
617 else:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
618 subst.append(v)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
619 if exact:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
620 exact_match_spec[p.name] = exact
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
621 if subst:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
622 filterspec[p.name] = subst
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
623 else:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
624 assert not isinstance(p.val, Exact_Match)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
625 filterspec[p.name] = p.val
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5315
diff changeset
626 self.val = self.cls._filter(search_matches, filterspec, sort and self,
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
627 retired=retired,
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
628 exact_match_spec=exact_match_spec)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
629 return self.val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
630
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
631 def sort(self, ids=None):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
632 """ Sort ids by the order information stored in self. With
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
633 optimisations: Some order attributes may be precomputed (by the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
634 backend) and some properties may already be sorted.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
635 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
636 if ids is None:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
637 ids = self.val
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
638 if self.sortattr and [s for s in self.sortattr
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
639 if not s.attr_sort_done]:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
640 return self._searchsort(ids, True, True)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
641 return ids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
642
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
643 def sortable_children(self, intermediate=False):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
644 """ All children needed for sorting. If intermediate is True,
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
645 intermediate nodes (not being a sort attribute) are returned,
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
646 too.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
647 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
648 return [p for p in self.children
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
649 if 'sort' in p.need_for and (intermediate or p.sort_direction)]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
650
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
651 def __iter__(self):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
652 """ Yield nodes in depth-first order -- visited nodes first """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
653 for p in self.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
654 yield p
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
655 for c in p:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
656 yield c
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
657
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
658 def _get(self, ids):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
659 """Lookup given ids -- possibly a list of list. We recurse until
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
660 we have a list of ids.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
661 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
662 if not ids:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
663 return ids
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
664 if isinstance(ids[0], list):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
665 cids = [self._get(i) for i in ids]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
666 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
667 cids = [i and self.parent.cls.get(i, self.name) for i in ids]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
668 if self.sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
669 cids = [self._searchsort(i, False, True) for i in cids]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
670 return cids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
671
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
672 def _searchsort(self, ids=None, update=True, dosort=True):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
673 """ Recursively compute the sort attributes. Note that ids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
674 may be a deeply nested list of lists of ids if several
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
675 multilinks are encountered on the way from the root to an
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
676 individual attribute. We make sure that everything is properly
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
677 sorted on the way up. Note that the individual backend may
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
678 already have precomputed self.result or self.sort_ids. In this
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
679 case we do nothing for existing sa.result and recurse further if
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
680 self.sort_ids is available.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
681
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
682 Yech, Multilinks: This gets especially complicated if somebody
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
683 sorts by different attributes of the same multilink (or
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
684 transitively across several multilinks). My use-case is sorting
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
685 by issue.messages.author and (reverse) by issue.messages.date.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
686 In this case we sort the messages by author and date and use
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
687 this sorted list twice for sorting issues. This means that
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
688 issues are sorted by author and then by the time of the messages
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
689 *of this author*. Probably what the user intends in that case,
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
690 so we do *not* use two sorted lists of messages, one sorted by
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
691 author and one sorted by date for sorting issues.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
692 """
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
693 for pt in self.sortable_children(intermediate=True):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
694 # ids can be an empty list
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
695 if pt.tree_sort_done or not ids:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
696 continue
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
697 if pt.sort_ids: # cached or computed by backend
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
698 cids = pt.sort_ids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
699 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
700 cids = pt._get(ids)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
701 if pt.sort_direction and not pt.sort_result:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
702 sortrep = pt.propclass.sort_repr
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
703 pt.sort_result = pt._sort_repr(sortrep, cids)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
704 pt.sort_ids = cids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
705 if pt.children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
706 pt._searchsort(cids, update, False)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
707 if self.sortattr and dosort:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
708 ids = self._sort(ids)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
709 if not update:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
710 for pt in self.sortable_children(intermediate=True):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
711 pt.sort_ids = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
712 for pt in self.sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
713 pt.sort_result = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
714 return ids
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
715
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
716 def _set_val(self, val):
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
717 """ Check if self._val is already defined. If yes, we compute the
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
718 intersection of the old and the new value(s)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
719 Note: If self is a Leaf node we need to compute a
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
720 union: Normally we intersect (logical and) different
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
721 subqueries into a Link or Multilink property. But for
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
722 leaves we might have a part of a query in a filterspec and
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
723 in an exact_match_spec. These have to be all there, the
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
724 generated search will ensure a logical and of all tests for
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
725 equality/substring search.
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
726 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
727 if self.has_values:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
728 v = self._val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
729 if not isinstance(self._val, type([])):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
730 v = [self._val]
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4063
diff changeset
731 vals = set(v)
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
732 if not isinstance(val, type([])):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
733 val = [val]
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
734 # if cls is None we're a leaf
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
735 if self.cls:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
736 vals.intersection_update(val)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
737 else:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
738 vals.update(val)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
739 self._val = [v for v in vals]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
740 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
741 self._val = val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
742 self.has_values = True
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3802
diff changeset
743
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
744 val = property(lambda self: self._val, _set_val)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
745
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
746 def _sort(self, val):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
747 """Finally sort by the given sortattr.sort_result. Note that we
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
748 do not sort by attrs having attr_sort_done set. The caller is
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
749 responsible for setting attr_sort_done only for trailing
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
750 attributes (otherwise the sort order is wrong). Since pythons
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
751 sort is stable, we can sort already sorted lists without
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
752 destroying the sort-order for items that compare equal with the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
753 current sort.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
754
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
755 Sorting-Strategy: We sort repeatedly by different sort-keys from
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
756 right to left. Since pythons sort is stable, we can safely do
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
757 that. An optimisation is a "run-length encoding" of the
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
758 sort-directions: If several sort attributes sort in the same
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
759 direction we can combine them into a single sort. Note that
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
760 repeated sorting is probably more efficient than using
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
761 compare-methods in python due to the overhead added by compare
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
762 methods.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
763 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
764 if not val:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
765 return val
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
766 sortattr = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
767 directions = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
768 dir_idx = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
769 idx = 0
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
770 curdir = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
771 for sa in self.sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
772 if sa.attr_sort_done:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
773 break
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
774 if sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
775 assert len(sortattr[0]) == len(sa.sort_result)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
776 sortattr.append(sa.sort_result)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
777 if curdir != sa.sort_direction:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
778 dir_idx.append(idx)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
779 directions.append(sa.sort_direction)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
780 curdir = sa.sort_direction
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
781 idx += 1
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
782 sortattr.append(val)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
783 sortattr = zip(*sortattr)
5397
f2c5e0f6506e Python 3 preparation: use list() around zip() as needed.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5395
diff changeset
784 for dir, i in reversed(list(zip(directions, dir_idx))):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
785 rev = dir == '-'
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
786 sortattr = sorted(sortattr,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
787 key=lambda x: NoneAndDictComparable(x[i:idx]),
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
788 reverse=rev)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
789 idx = i
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
790 return [x[-1] for x in sortattr]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
791
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
792 def _sort_repr(self, sortrep, ids):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
793 """Call sortrep for given ids -- possibly a list of list. We
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
794 recurse until we have a list of ids.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
795 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
796 if not ids:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
797 return ids
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
798 if isinstance(ids[0], list):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
799 res = [self._sort_repr(sortrep, i) for i in ids]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
800 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
801 res = [sortrep(self.cls, i, self.name) for i in ids]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
802 return res
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
803
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
804 def __repr__(self):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
805 r = ["proptree:" + self.name]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
806 for n in self:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
807 r.append("proptree:" + " " * n.depth + n.name)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
808 return '\n'.join(r)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
809 __str__ = __repr__
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
810
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
811
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
812 #
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
813 # the base Database class
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
814 #
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
815 class DatabaseError(ValueError):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
816 """Error to be raised when there is some problem in the database code
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
817 """
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
818 pass
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
819
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
820
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
821 class Database(object):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
822 """A database for storing records containing flexible data types.
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
823
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
824 This class defines a hyperdatabase storage layer, which the Classes use to
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
825 store their data.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
826
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
827
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
828 Transactions
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
829 ------------
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
830 The Database should support transactions through the commit() and
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
831 rollback() methods. All other Database methods should be transaction-aware,
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
832 using data from the current transaction before looking up the database.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
833
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
834 An implementation must provide an override for the get() method so that the
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
835 in-database value is returned in preference to the in-transaction value.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
836 This is necessary to determine if any values have changed during a
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
837 transaction.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
838
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
839
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
840 Implementation
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
841 --------------
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
842
1926
3bdd34547fa7 Remove implementations of Class.getnode from back_anydbm and rdbms_common...
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1905
diff changeset
843 All methods except __repr__ must be implemented by a concrete backend Database.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
844
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
845 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
846
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
847 # flag to set on retired entries
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
848 RETIRED_FLAG = '__hyperdb_retired'
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
849
3491
0e5f15520e70 fix detection of "missing" existing values in CGI form parser [SF#1414149]
Richard Jones <richard@users.sourceforge.net>
parents: 3488
diff changeset
850 BACKEND_MISSING_STRING = None
0e5f15520e70 fix detection of "missing" existing values in CGI form parser [SF#1414149]
Richard Jones <richard@users.sourceforge.net>
parents: 3488
diff changeset
851 BACKEND_MISSING_NUMBER = None
0e5f15520e70 fix detection of "missing" existing values in CGI form parser [SF#1414149]
Richard Jones <richard@users.sourceforge.net>
parents: 3488
diff changeset
852 BACKEND_MISSING_BOOLEAN = None
0e5f15520e70 fix detection of "missing" existing values in CGI form parser [SF#1414149]
Richard Jones <richard@users.sourceforge.net>
parents: 3488
diff changeset
853
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 490
diff changeset
854 def __init__(self, config, journaltag=None):
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
855 """Open a hyperdatabase given a specifier to some storage.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
856
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 490
diff changeset
857 The 'storagelocator' is obtained from config.DATABASE.
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
858 The meaning of 'storagelocator' depends on the particular
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
859 implementation of the hyperdatabase. It could be a file name,
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
860 a directory path, a socket descriptor for a connection to a
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
861 database over the network, etc.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
862
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
863 The 'journaltag' is a token that will be attached to the journal
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
864 entries for any edits done on the database. If 'journaltag' is
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
865 None, the database is opened in read-only mode: the Class.create(),
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
866 Class.set(), and Class.retire() methods are disabled.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
867 """
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
868 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
869
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 805
diff changeset
870 def post_init(self):
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
871 """Called once the schema initialisation has finished.
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
872 If 'refresh' is true, we want to rebuild the backend
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
873 structures. Note that post_init can be called multiple times,
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
874 at least during regression testing.
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
875 """
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
876 done = getattr(self, 'post_init_done', None)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
877 for cn in self.getclasses():
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
878 cl = self.getclass(cn)
6228
f40c6b5de370 Fix reverse multilink iteration
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6183
diff changeset
879 # This will change properties if a back-multilink happens to
f40c6b5de370 Fix reverse multilink iteration
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6183
diff changeset
880 # have the same class, so we need to iterate over .keys()
f40c6b5de370 Fix reverse multilink iteration
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6183
diff changeset
881 for p in cl.properties.keys():
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
882 prop = cl.properties[p]
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
883 if not isinstance (prop, (Link, Multilink)):
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
884 continue
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
885 if prop.rev_multilink:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
886 linkcls = self.getclass(prop.classname)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
887 if prop.rev_multilink in linkcls.properties:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
888 if not done:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
889 raise ValueError(
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
890 "%s already a property of class %s"%
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
891 (prop.rev_multilink, linkcls.classname))
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
892 else:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
893 linkcls.properties[prop.rev_multilink] = Multilink(
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
894 cl.classname, rev_property=prop)
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
895 self.post_init_done = True
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
896
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
897 def refresh_database(self):
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
898 """Called to indicate that the backend should rebuild all tables
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1780
diff changeset
899 and structures. Not called in normal usage."""
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 805
diff changeset
900 raise NotImplementedError
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 805
diff changeset
901
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
902 def __getattr__(self, classname):
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
903 """A convenient way of calling self.getclass(classname)."""
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
904 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
905
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
906 def addclass(self, cl):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
907 """Add a Class to the hyperdatabase.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
908 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
909 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
910
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
911 def getclasses(self):
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
912 """Return a list of the names of all existing classes."""
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
913 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
914
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
915 def getclass(self, classname):
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
916 """Get the Class object representing a particular class.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
917
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
918 If 'classname' is not a valid class name, a KeyError is raised.
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
919 """
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
920 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
921
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
922 def clear(self):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
923 """Delete all database contents.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
924 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
925 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
926
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
927 def getclassdb(self, classname, mode='r'):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
928 """Obtain a connection to the class db that will be used for
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
929 multiple actions.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
930 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
931 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
932
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
933 def addnode(self, classname, nodeid, node):
1928
7c1ddebe7589 Small readability improvements.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1927
diff changeset
934 """Add the specified node to its class's db.
7c1ddebe7589 Small readability improvements.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1927
diff changeset
935 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
936 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
937
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 657
diff changeset
938 def serialise(self, classname, node):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
939 """Copy the node contents, converting non-marshallable data into
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 657
diff changeset
940 marshallable data.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
941 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
942 return node
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 657
diff changeset
943
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
944 def setnode(self, classname, nodeid, node):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
945 """Change the specified node.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
946 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
947 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
948
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 657
diff changeset
949 def unserialise(self, classname, node):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
950 """Decode the marshalled node data
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
951 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
952 return node
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 657
diff changeset
953
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
954 def getnode(self, classname, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
955 """Get a node from the database.
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
956
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
957 'cache' exists for backwards compatibility, and is not used.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
958 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
959 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
960
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
961 def hasnode(self, classname, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
962 """Determine if the database has a given node.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
963 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
964 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
965
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
966 def countnodes(self, classname):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
967 """Count the number of nodes that exist for a particular Class.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
968 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
969 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
970
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
971 def storefile(self, classname, nodeid, property, content):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
972 """Store the content of the file in the database.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
973
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
974 The property may be None, in which case the filename does not
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
975 indicate which property is being saved.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
976 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
977 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
978
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
979 def getfile(self, classname, nodeid, property):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
980 """Get the content of the file in the database.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
981 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
982 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
983
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
984 def addjournal(self, classname, nodeid, action, params):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
985 """ Journal the Action
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
986 'action' may be:
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
987
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
988 'set' -- 'params' is a dictionary of property values
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
989 'create' -- 'params' is an empty dictionary as of
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
990 Wed Nov 06 11:38:43 2002 +0000
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
991 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
992 'retired' or 'restored'-- 'params' is None
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
993 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
994 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
995
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
996 def getjournal(self, classname, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
997 """ get the journal for id
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
998 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
999 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1000
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 557
diff changeset
1001 def pack(self, pack_before):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1002 """ pack the database
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1003 """
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 557
diff changeset
1004 raise NotImplementedError
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 557
diff changeset
1005
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1006 def commit(self):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1007 """ Commit the current transactions.
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1008
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1009 Save all data changed since the database was opened or since the
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1010 last commit() or rollback().
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1011 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1012 raise NotImplementedError
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1013
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1014 def rollback(self):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1015 """ Reverse all actions from the current transaction.
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1016
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1017 Undo all the changes made since the database was opened or the last
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1018 commit() or rollback() was performed.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1019 """
477
05a46da7293a hyperdb docstrings
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
1020 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1021
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1022 def close(self):
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1023 """Close the database.
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1024
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1025 This method must be called at the end of processing.
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1026
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1027 """
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1028
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1029
4306
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1030 def iter_roles(roles):
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1031 ''' handle the text processing of turning the roles list
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1032 into something python can use more easily
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1033 '''
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1034 if not roles or not roles.strip():
5807
0467d80eaeec Try to deal with this warning/error: DeprecationWarning: generator
John Rouillard <rouilj@ieee.org>
parents: 5697
diff changeset
1035 return
4306
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1036 for role in [x.lower().strip() for x in roles.split(',')]:
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1037 yield role
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1038
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1039
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1040 #
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1041 # The base Class class
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1042 #
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1043 class Class:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1044 """ The handle to a particular class of nodes in a hyperdatabase.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1045
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1046 All methods except __repr__ and getnode must be implemented by a
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1047 concrete backend Class.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1048 """
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1049
6238
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1050 class_re = r'^([A-Za-z](?:[A-Za-z_0-9]*[A-Za-z_]+)?)$'
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1051
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1052 def __init__(self, db, classname, **properties):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1053 """Create a new class with a given name and property specification.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1054
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1055 'classname' must not collide with the name of an existing class,
6238
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1056 or a ValueError is raised. 'classname' must start with an
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1057 alphabetic letter. It must end with an alphabetic letter or '_'.
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1058 Internal characters can be alphanumeric or '_'. ValueError is
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1059 raised if the classname is not correct.
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1060 The keyword arguments in 'properties' must map names to property
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1061 objects, or a TypeError is raised.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1062 """
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1063 for name in 'creation activity creator actor'.split():
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1064 if name in properties:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1065 raise ValueError('"creation", "activity", "creator" and '
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1066 '"actor" are reserved')
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1067
6238
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1068 if not re.match(self.class_re, classname):
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1069 raise ValueError('Class name %s is not valid. It must start '
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1070 'with a letter, end with a letter or "_", and '
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1071 'only have alphanumerics and "_" in the '
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1072 'middle.' % (classname,))
6834bb5473da Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents: 6228
diff changeset
1073
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1074 self.classname = classname
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1075 self.properties = properties
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
1076 # Make the class and property name known to the property
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
1077 for p in properties:
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6006
diff changeset
1078 properties[p].register(self, p)
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1079 self.db = weakref.proxy(db) # use a weak ref to avoid circularity
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1080 self.key = ''
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1081
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1082 # should we journal changes (default yes)
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1083 self.do_journal = 1
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1084
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1085 # do the db-related init stuff
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1086 db.addclass(self)
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1087
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1088 actions = "create set retire restore".split()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1089 self.auditors = dict([(a, PrioList()) for a in actions])
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1090 self.reactors = dict([(a, PrioList()) for a in actions])
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1091
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
1092 def __repr__(self):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1093 """Slightly more useful representation
6183
bb198596f85c Fix __repr__ of hyperdb.Class
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6179
diff changeset
1094 Note that an error message can be raised at a point
bb198596f85c Fix __repr__ of hyperdb.Class
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6179
diff changeset
1095 where self.classname isn't known yet if the error
bb198596f85c Fix __repr__ of hyperdb.Class
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6179
diff changeset
1096 occurs during schema parsing.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1097 """
6183
bb198596f85c Fix __repr__ of hyperdb.Class
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6179
diff changeset
1098 cn = getattr (self, 'classname', 'Unknown')
bb198596f85c Fix __repr__ of hyperdb.Class
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6179
diff changeset
1099 return '<hyperdb.Class "%s">' % cn
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
1100
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1101 # Editing nodes:
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1102
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1103 def create(self, **propvalues):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1104 """Create a new node of this class and return its id.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1105
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1106 The keyword arguments in 'propvalues' map property names to values.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1107
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1108 The values of arguments must be acceptable for the types of their
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1109 corresponding properties or a TypeError is raised.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1110
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1111 If this class has a key property, it must be present and its value
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1112 must not collide with other key strings or a ValueError is raised.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1113
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1114 Any other properties on this class that are missing from the
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1115 'propvalues' dictionary are set to None.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1116
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1117 If an id in a link or multilink property does not refer to a valid
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1118 node, an IndexError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1119 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1120 raise NotImplementedError
775
4409798dfa15 Can debug to stdout now
Richard Jones <richard@users.sourceforge.net>
parents: 764
diff changeset
1121
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1122 _marker = []
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1123
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1124 def get(self, nodeid, propname, default=_marker, cache=1):
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1125 """Get the value of a property on an existing node of this class.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1126
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1127 'nodeid' must be the id of an existing node of this class or an
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1128 IndexError is raised. 'propname' must be the name of a property
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1129 of this class or a KeyError is raised.
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1130
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1131 'cache' exists for backwards compatibility, and is not used.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1132 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1133 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1134
1926
3bdd34547fa7 Remove implementations of Class.getnode from back_anydbm and rdbms_common...
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1905
diff changeset
1135 # not in spec
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1136 def getnode(self, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1137 """ Return a convenience wrapper for the node.
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1138
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1139 'nodeid' must be the id of an existing node of this class or an
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1140 IndexError is raised.
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1141
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1142 'cache' exists for backwards compatibility, and is not used.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1143 """
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1144 return Node(self, nodeid)
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1145
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1146 def getnodeids(self, retired=None):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1147 """Retrieve all the ids of the nodes for a particular Class.
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1148 """
1864
969a14faf707 Fixed "documentation" of getnodeids in roundup.hyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
1149 raise NotImplementedError
969a14faf707 Fixed "documentation" of getnodeids in roundup.hyperdb
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
1150
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1151 def set(self, nodeid, **propvalues):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1152 """Modify a property on an existing node of this class.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1153
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1154 'nodeid' must be the id of an existing node of this class or an
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1155 IndexError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1156
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1157 Each key in 'propvalues' must be the name of a property of this
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1158 class or a KeyError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1159
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1160 All values in 'propvalues' must be acceptable types for their
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1161 corresponding properties or a TypeError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1162
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1163 If the value of the key property is set, it must not collide with
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1164 other key strings or a ValueError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1165
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1166 If the value of a Link or Multilink property contains an invalid
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1167 node id, a ValueError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1168 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1169 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1170
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1171 def retire(self, nodeid):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1172 """Retire a node.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1173
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1174 The properties on the node remain available from the get() method,
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1175 and the node's id is never reused.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1176
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1177 Retired nodes are not returned by the find(), list(), or lookup()
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1178 methods, and other nodes may reuse the values of their key properties.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1179 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1180 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1181
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1470
diff changeset
1182 def restore(self, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1183 """Restpre a retired node.
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1470
diff changeset
1184
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1470
diff changeset
1185 Make node available for all operations like it was before retirement.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1186 """
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1470
diff changeset
1187 raise NotImplementedError
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1188
940
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 910
diff changeset
1189 def is_retired(self, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1190 """Return true if the node is rerired
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1191 """
940
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 910
diff changeset
1192 raise NotImplementedError
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 910
diff changeset
1193
894
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1194 def destroy(self, nodeid):
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1195 """Destroy a node.
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1196
894
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1197 WARNING: this method should never be used except in extremely rare
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1198 situations where there could never be links to the node being
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1199 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1200
894
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1201 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1202
894
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1203 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1204
894
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1205 WARNING: really, use retire() instead
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1206
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1207 Well, I think that's enough warnings. This method exists mostly to
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1208 support the session storage of the cgi interface.
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1209
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1210 The node is completely removed from the hyperdb, including all journal
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1211 entries. It will no longer be available, and will generally break code
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1212 if there are any references to the node.
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1213 """
cbefecea6c74 Gordon, does this help?
Richard Jones <richard@users.sourceforge.net>
parents: 883
diff changeset
1214
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1215 def history(self, nodeid, enforceperm=True, skipquiet=True):
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1216 """Retrieve the journal of edits on a particular node.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1217
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1218 'nodeid' must be the id of an existing node of this class or an
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1219 IndexError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1220
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1221 The returned list contains tuples of the form
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1222
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1223 (date, tag, action, params)
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1224
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1225 'date' is a Timestamp object specifying the time of the change and
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1226 'tag' is the journaltag specified when the database was opened.
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1227
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1228 If the property to be displayed is a quiet property, it will
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1229 not be shown. This can be disabled by setting skipquiet=False.
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1230
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1231 If the user requesting the history does not have View access
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1232 to the property, the journal entry will not be shown. This can
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1233 be disabled by setting enforceperm=False.
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1234
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1235 Note that there is a check for obsolete properties and classes
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1236 resulting from history changes. These are also only checked if
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1237 enforceperm is True.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1238 """
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4481
diff changeset
1239 if not self.do_journal:
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4481
diff changeset
1240 raise ValueError('Journalling is disabled for this class')
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1241
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1242 perm = self.db.security.hasPermission
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1243 journal = []
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1244
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1245 uid = self.db.getuid() # id of the person requesting the history
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1246
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1247 # Roles of the user and the configured obsolete_history_roles
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1248 hr = set(iter_roles(self.db.config.OBSOLETE_HISTORY_ROLES))
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1249 ur = set(self.db.user.get_roles(uid))
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1250 allow_obsolete = bool(hr & ur)
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1251
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1252 for j in self.db.getjournal(self.classname, nodeid):
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1253 # hide/remove journal entry if:
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1254 # property is quiet
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1255 # property is not (viewable or editable)
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1256 # property is obsolete and not allow_obsolete
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1257 id, evt_date, user, action, args = j
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1258 if logger.isEnabledFor(logging.DEBUG):
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1259 j_repr = "%s" % (j,)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1260 else:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1261 j_repr = ''
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1262 if args and isinstance(args, type({})):
5460
87f22a5d65ca container modification while iterating over it
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5397
diff changeset
1263 for key in list(args.keys()):
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1264 if key not in self.properties:
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1265 if enforceperm and not allow_obsolete:
5460
87f22a5d65ca container modification while iterating over it
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5397
diff changeset
1266 del args[key]
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1267 continue
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1268 if skipquiet and self.properties[key].quiet:
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1269 logger.debug("skipping quiet property"
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1270 " %s::%s in %s",
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1271 self.classname, key, j_repr)
5460
87f22a5d65ca container modification while iterating over it
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5397
diff changeset
1272 del args[key]
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1273 continue
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1274 if enforceperm and not (perm("View",
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1275 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1276 self.classname,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1277 property=key) or
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1278 perm("Edit",
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1279 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1280 self.classname,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1281 property=key)):
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1282 logger.debug("skipping unaccessible property "
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1283 "%s::%s seen by user%s in %s",
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1284 self.classname, key, uid, j_repr)
5460
87f22a5d65ca container modification while iterating over it
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5397
diff changeset
1285 del args[key]
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1286 continue
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1287 if not args:
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1288 logger.debug("Omitting journal entry for %s%s"
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1289 " all props removed in: %s",
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1290 self.classname, nodeid, j_repr)
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1291 continue
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1292 journal.append(j)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1293 elif action in ['link', 'unlink'] and isinstance(args, type(())):
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1294 # definitions:
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1295 # myself - object whose history is being filtered
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1296 # linkee - object/class whose property is changing to
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1297 # include/remove myself
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1298 # link property - property of the linkee class that is changing
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1299 #
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1300 # Remove the history item if
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1301 # linkee.link property (key) is quiet
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1302 # linkee class.link property is not (viewable or editable)
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1303 # to user
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1304 # [ should linkee object.link property is not
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1305 # (viewable or editable) to user be included?? ]
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1306 # linkee object (linkcl, linkid) is not
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1307 # (viewable or editable) to user
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1308 if len(args) == 3:
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1309 # e.g. for issue3 blockedby adds link to issue5 with:
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1310 # j = id, evt_date, user, action, args
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1311 # 3|20170528045201.484|5|link|('issue', '5', 'blockedby')
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1312 linkcl, linkid, key = args
5315
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1313 cls = None
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1314 try:
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1315 cls = self.db.getclass(linkcl)
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1316 except KeyError:
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1317 pass
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1318 # obsolete property or class
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1319 if not cls or key not in cls.properties:
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1320 if not enforceperm or allow_obsolete:
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1321 journal.append(j)
5a014410f254 Fix issue2550954: History display breaks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5309
diff changeset
1322 continue
5527
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1323 # obsolete linked-to item
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1324 try:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1325 cls.get(linkid, key) # does linkid exist
5527
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1326 except IndexError:
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1327 if not enforceperm or allow_obsolete:
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1328 journal.append(j)
a7c3cd2edf51 No traceback for non-existent items in history
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5525
diff changeset
1329 continue
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1330 # is the updated property quiet?
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1331 if skipquiet and cls.properties[key].quiet:
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1332 logger.debug("skipping quiet property: "
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1333 "%s %sed %s%s",
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1334 j_repr, action, self.classname, nodeid)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1335 continue
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1336 # can user view the property in linkee class
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1337 if enforceperm and not (perm("View",
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1338 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1339 linkcl,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1340 property=key) or
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1341 perm("Edit",
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1342 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1343 linkcl,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1344 property=key)):
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1345 logger.debug("skipping unaccessible property: "
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1346 "%s with uid %s %sed %s%s",
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1347 j_repr, uid, action,
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1348 self.classname, nodeid)
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1349 continue
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1350 # check access to linkee object
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1351 if enforceperm and not (perm("View",
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1352 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1353 cls.classname,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1354 itemid=linkid) or
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1355 perm("Edit",
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1356 uid,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1357 cls.classname,
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1358 itemid=linkid)):
5261
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1359 logger.debug("skipping unaccessible object: "
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1360 "%s uid %s %sed %s%s",
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1361 j_repr, uid, action,
53a853c06e9a Fixed quiet history filter for linked items. Test for same fixed.
John Rouillard <rouilj@ieee.org>
parents: 5260
diff changeset
1362 self.classname, nodeid)
5257
928512faf565 - issue2550864: Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5248
diff changeset
1363 continue
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1364 journal.append(j)
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1365 else:
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1366 logger.error("Invalid %s journal entry for %s%s: %s",
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1367 action, self.classname, nodeid, j)
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1368 elif action in ['create', 'retired', 'restored']:
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1369 journal.append(j)
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1370 else:
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1371 logger.warning("Possibly malformed journal for %s%s %s",
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1372 self.classname, nodeid, j)
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
1373 return journal
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1374
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1375 # Locating nodes:
627
952679be9e2c Added simple editing for classes that don't define a templated interface.
Richard Jones <richard@users.sourceforge.net>
parents: 618
diff changeset
1376 def hasnode(self, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1377 """Determine if the given nodeid actually exists
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1378 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1379 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1380
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1381 def setkey(self, propname):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1382 """Select a String property of this class to be the key property.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1383
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1384 'propname' must be the name of a String property of this class or
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1385 None, or a TypeError is raised. The values of the key property on
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1386 all existing nodes must be unique or a ValueError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1387 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1388 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1389
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1390 def setlabelprop(self, labelprop):
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1391 """Set the label property. Used for override of labelprop
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1392 resolution order.
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1393 """
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1394 if labelprop not in self.getprops():
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5327
diff changeset
1395 raise ValueError(_("Not a property name: %s") % labelprop)
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1396 self._labelprop = labelprop
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1397
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1398 def setorderprop(self, orderprop):
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1399 """Set the order property. Used for override of orderprop
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1400 resolution order
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1401 """
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1402 if orderprop not in self.getprops():
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5327
diff changeset
1403 raise ValueError(_("Not a property name: %s") % orderprop)
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1404 self._orderprop = orderprop
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1405
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1406 def getkey(self):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1407 """Return the name of the key property for this class or None."""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1408 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1409
188
1536be43d2fa Roundupdb now appends "mailing list" information to its messages...
Richard Jones <richard@users.sourceforge.net>
parents: 172
diff changeset
1410 def labelprop(self, default_to_id=0):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1411 """Return the property name for a label for the given node.
123
51cce9671db0 Cleanup of the link label generation.
Richard Jones <richard@users.sourceforge.net>
parents: 118
diff changeset
1412
51cce9671db0 Cleanup of the link label generation.
Richard Jones <richard@users.sourceforge.net>
parents: 118
diff changeset
1413 This method attempts to generate a consistent label for the node.
51cce9671db0 Cleanup of the link label generation.
Richard Jones <richard@users.sourceforge.net>
parents: 118
diff changeset
1414 It tries the following in order:
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1415
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1416 0. self._labelprop if set
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1417 1. key property
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1418 2. "name" property
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1419 3. "title" property
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1420 4. first property from the sorted property name list
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1421 """
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1422 if hasattr(self, '_labelprop'):
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1423 return self._labelprop
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1424 k = self.getkey()
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1425 if k:
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1426 return k
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1427 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1428 if 'name' in props:
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1429 return 'name'
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1430 elif 'title' in props:
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1431 return 'title'
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1432 if default_to_id:
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1433 return 'id'
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1434 props = sorted(props.keys())
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1435 return props[0]
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1436
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1437 def orderprop(self):
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1438 """Return the property name to use for sorting for the given node.
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1439
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1440 This method computes the property for sorting.
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1441 It tries the following in order:
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1442
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1443 0. self._orderprop if set
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1444 1. "order" property
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1445 2. self.labelprop()
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1446 """
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1447
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1448 if hasattr(self, '_orderprop'):
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1449 return self._orderprop
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1450 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1451 if 'order' in props:
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1452 return 'order'
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1453 return self.labelprop()
123
51cce9671db0 Cleanup of the link label generation.
Richard Jones <richard@users.sourceforge.net>
parents: 118
diff changeset
1454
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1455 def lookup(self, keyvalue):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1456 """Locate a particular node by its key property and return its id.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1457
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1458 If this class has no key property, a TypeError is raised. If the
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1459 'keyvalue' matches one of the values for the key property among
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1460 the nodes in this class, the matching node's id is returned;
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1461 otherwise a KeyError is raised.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1462 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1463 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1464
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1465 def find(self, **propspec):
834
568eed5fb4fd Optimize Class.find so that the propspec can contain a set of ids to match.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 825
diff changeset
1466 """Get the ids of nodes in this class which link to the given nodes.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1467
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1468 'propspec' consists of keyword args propname={nodeid:1,}
883
Richard Jones <richard@users.sourceforge.net>
parents: 880
diff changeset
1469 'propname' must be the name of a property in this class, or a
Richard Jones <richard@users.sourceforge.net>
parents: 880
diff changeset
1470 KeyError is raised. That property must be a Link or Multilink
Richard Jones <richard@users.sourceforge.net>
parents: 880
diff changeset
1471 property, or a TypeError is raised.
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1472
834
568eed5fb4fd Optimize Class.find so that the propspec can contain a set of ids to match.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 825
diff changeset
1473 Any node in this class whose 'propname' property links to any of the
568eed5fb4fd Optimize Class.find so that the propspec can contain a set of ids to match.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 825
diff changeset
1474 nodeids will be returned. Used by the full text indexing, which knows
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1475 that "foo" occurs in msg1, msg3 and file7, so we have hits on these
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1476 issues:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1477
834
568eed5fb4fd Optimize Class.find so that the propspec can contain a set of ids to match.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 825
diff changeset
1478 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1479 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1480 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1481
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1482 def _filter(self, search_matches, filterspec, sort=(None, None),
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1483 group=(None, None), retired=False, exact_match_spec={}):
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1484 """For some backends this implements the non-transitive
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1485 search, for more information see the filter method.
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1486 """
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1487 raise NotImplementedError
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1488
5871
acc4a128ab9b Backwards-compatible signature of _proptree method
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5869
diff changeset
1489 def _proptree(self, filterspec, exact_match_spec={}, sortattr=[],
acc4a128ab9b Backwards-compatible signature of _proptree method
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5869
diff changeset
1490 retr=False):
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1491 """Build a tree of all transitive properties in the given
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1492 exact_match_spec/filterspec.
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1493 If we retrieve (retr is True) linked items we don't follow
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1494 across multilinks. We also don't follow if the searched value
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1495 can contain NULL values.
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1496 """
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1497 proptree = Proptree(self.db, self, '', self.getprops(), retr=retr)
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1498 for exact, spec in enumerate((filterspec, exact_match_spec)):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1499 for key, v in spec.items():
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1500 keys = key.split('.')
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1501 p = proptree
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1502 mlseen = False
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1503 for k in keys:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1504 if isinstance(p.propclass, Multilink):
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1505 mlseen = True
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1506 isnull = v == '-1' or v is None
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1507 islist = isinstance(v, type([]))
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1508 nullin = islist and ('-1' in v or None in v)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1509 r = retr and not mlseen and not isnull and not nullin
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1510 p = p.append(k, retr=r)
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1511 if exact:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1512 if isinstance(v, type([])):
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1513 vv = []
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1514 for x in v:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1515 vv.append(Exact_Match(x))
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1516 p.val = vv
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1517 else:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1518 p.val = [Exact_Match(v)]
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1519 else:
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1520 p.val = v
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1521 multilinks = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1522 for s in sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1523 keys = s[1].split('.')
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1524 p = proptree
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1525 mlseen = False
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1526 for k in keys:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1527 if isinstance(p.propclass, Multilink):
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1528 mlseen = True
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1529 r = retr and not mlseen
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1530 p = p.append(k, need_for='sort', retr=r)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1531 if isinstance(p.propclass, Multilink):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1532 multilinks[p] = True
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1533 if p.cls:
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1534 p = p.append(p.cls.orderprop(), need_for='sort')
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1535 if p.sort_direction: # if orderprop is also specified explicitly
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1536 continue
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1537 p.sort_direction = s[0]
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1538 proptree.sortattr.append(p)
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1539 for p in multilinks.keys():
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1540 sattr = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1541 for c in p:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1542 if c.sort_direction:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1543 sattr[c] = True
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1544 for sa in proptree.sortattr:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1545 if sa in sattr:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1546 p.sortattr.append(sa)
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1547 return proptree
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1548
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1549 def get_transitive_prop(self, propname_path, default=None):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1550 """Expand a transitive property (individual property names
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1551 separated by '.' into a new property at the end of the path. If
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1552 one of the names does not refer to a valid property, we return
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1553 None.
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1554 Example propname_path (for class issue): "messages.author"
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1555 """
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1556 props = self.db.getclass(self.classname).getprops()
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1557 for k in propname_path.split('.'):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1558 try:
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1559 prop = props[k]
4408
06af6d5bedbe fix for incorrect except: syntax, issue2550661
Richard Jones <richard@users.sourceforge.net>
parents: 4306
diff changeset
1560 except (KeyError, TypeError):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1561 return default
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1562 cl = getattr(prop, 'classname', None)
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1563 props = None
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1564 if cl:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1565 props = self.db.getclass(cl).getprops()
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1566 return prop
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1567
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1568 def _sortattr(self, sort=[], group=[]):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1569 """Build a single list of sort attributes in the correct order
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1570 with sanity checks (no duplicate properties) included. Always
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1571 sort last by id -- if id is not already in sortattr.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1572 """
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1573 seen = {}
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1574 sortattr = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1575 for srt in group, sort:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1576 if not isinstance(srt, list):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1577 srt = [srt]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1578 for s in srt:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1579 if s[1] and s[1] not in seen:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1580 sortattr.append((s[0] or '+', s[1]))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1581 seen[s[1]] = True
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1582 if 'id' not in seen:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1583 sortattr.append(('+', 'id'))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1584 return sortattr
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1585
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5315
diff changeset
1586 def filter(self, search_matches, filterspec, sort=[], group=[],
5869
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1587 retired=False, exact_match_spec={}, limit=None, offset=None):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1588 """Return a list of the ids of the active nodes in this class that
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1589 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1590 sort spec.
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1591
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1592 "search_matches" is a container type which by default is None
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1593 and optionally contains IDs of items to match. If non-empty only
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1594 IDs of the initial set are returned.
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1595
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1596 "filterspec" is {propname: value(s)}
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1597 "exact_match_spec" is the same format as "filterspec" but
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1598 specifies exact match for the given propnames. This only makes a
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1599 difference for String properties, these specify case insensitive
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1600 substring search when in "filterspec" and exact match when in
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1601 exact_match_spec.
1249
6c24a86a12ae Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents: 1244
diff changeset
1602
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1603 "sort" and "group" are [(dir, prop), ...] where dir is '+', '-'
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1604 or None and prop is a prop name or None. Note that for
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1605 backward-compatibility reasons a single (dir, prop) tuple is
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1606 also allowed.
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1607
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1608 The parameter retired when set to False, returns only live
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1609 (un-retired) results. When setting it to True, only retired
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1610 items are returned. If None, both retired and unretired items
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1611 are returned. The default is False, i.e. only live items are
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1612 returned by default.
1249
6c24a86a12ae Fixes for SourceForge tracker bugs.
Richard Jones <richard@users.sourceforge.net>
parents: 1244
diff changeset
1613
5869
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1614 The "limit" and "offset" parameters define a limit on the number
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1615 of results returned and an offset before returning any results,
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1616 respectively. These can be used when displaying a number of
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1617 items in a pagination application or similar. A common use-case
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1618 is returning the first item of a sorted search by specifying
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1619 limit=1 (i.e. the maximum or minimum depending on sort order).
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1620
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3399
diff changeset
1621 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3399
diff changeset
1622 value to match is a list:
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3399
diff changeset
1623
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3399
diff changeset
1624 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3399
diff changeset
1625 2. Other properties must match any of the elements in the list.
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1626
5867
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1627 This also means that for strings in exact_match_spec it doesn't
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1628 make sense to specify multiple values because those cannot all
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1629 be matched exactly.
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1630
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1631 The propname in filterspec and prop in a sort/group spec may be
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1632 transitive, i.e., it may contain properties of the form
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1633 link.link.link.name, e.g. you can search for all issues where a
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1634 message was added by a certain user in the last week with a
ee2e8f8d6648 Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5809
diff changeset
1635 filterspec of
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1636 {'messages.author' : '42', 'messages.creation' : '.-1w;'}
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3802
diff changeset
1637
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1638 Implementation note:
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1639 This implements a non-optimized version of Transitive search
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1640 using _filter implemented in a backend class. A more efficient
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1641 version can be implemented in the individual backends -- e.g.,
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1642 an SQL backend will want to create a single SQL statement and
3634
57c66056ffe4 Implemented what I'll call for now "transitive searching"...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3621
diff changeset
1643 override the filter method instead of implementing _filter.
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1929
diff changeset
1644 """
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1645 sortattr = self._sortattr(sort=sort, group=group)
5871
acc4a128ab9b Backwards-compatible signature of _proptree method
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5869
diff changeset
1646 proptree = self._proptree(filterspec, exact_match_spec, sortattr)
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5315
diff changeset
1647 proptree.search(search_matches, retired=retired)
5869
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1648 if offset is not None or limit is not None:
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1649 items = proptree.sort()
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1650 if limit and offset:
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1651 return items[offset:offset+limit]
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1652 elif offset is not None:
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1653 return items[offset:]
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1654 else:
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
1655 return items[:limit]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1656 return proptree.sort()
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1657
4472
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1658 # non-optimized filter_iter, a backend may chose to implement a
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1659 # better version that provides a real iterator that pre-fills the
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1660 # cache for each id returned. Note that the filter_iter doesn't
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1661 # promise to correctly sort by multilink (which isn't sane to do
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1662 # anyway).
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1663 filter_iter = filter
34dce76bb202 Multilink fixes and optimizations:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4431
diff changeset
1664
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1665 def count(self):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1666 """Get the number of nodes in this class.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1667
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1668 If the returned integer is 'numnodes', the ids of all the nodes
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1669 in this class run from 1 to numnodes, and numnodes+1 will be the
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1670 id of the next node to be created in this class.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1671 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1672 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1673
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1674 # Manipulating properties:
262
ab921dc53ccd Added nicer command-line item adding:
Richard Jones <richard@users.sourceforge.net>
parents: 236
diff changeset
1675 def getprops(self, protected=1):
ab921dc53ccd Added nicer command-line item adding:
Richard Jones <richard@users.sourceforge.net>
parents: 236
diff changeset
1676 """Return a dictionary mapping property names to property objects.
ab921dc53ccd Added nicer command-line item adding:
Richard Jones <richard@users.sourceforge.net>
parents: 236
diff changeset
1677 If the "protected" flag is true, we include protected properties -
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1678 those which may not be modified.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1679 """
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1680 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1681
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1682 def get_required_props(self, propnames=[]):
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1683 """Return a dict of property names mapping to property objects.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1684 All properties that have the "required" flag set will be
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1685 returned in addition to all properties in the propnames
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1686 parameter.
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1687 """
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1688 props = self.getprops(protected=False)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1689 pdict = dict([(p, props[p]) for p in propnames])
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1690 pdict.update([(k, v) for k, v in props.items() if v.required])
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1691 return pdict
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3668
diff changeset
1692
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1693 def addprop(self, **properties):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1694 """Add properties to this class.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1695
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1696 The keyword arguments in 'properties' must map names to property
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1697 objects, or a TypeError is raised. None of the keys in 'properties'
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1698 may collide with the names of existing properties, or a ValueError
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1699 is raised before any properties have been added.
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1700 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1701 raise NotImplementedError
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1702
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 805
diff changeset
1703 def index(self, nodeid):
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1704 """Add (or refresh) the node to search indexes"""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 841
diff changeset
1705 raise NotImplementedError
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 805
diff changeset
1706
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1707 #
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1708 # Detector interface
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1709 #
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1710 def audit(self, event, detector, priority=100):
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1711 """Register an auditor detector"""
3743
e754cc14e76a fix unstable ordering of detectors [SF#1585378]
Richard Jones <richard@users.sourceforge.net>
parents: 3695
diff changeset
1712 self.auditors[event].append((priority, detector.__name__, detector))
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1713
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1714 def fireAuditors(self, event, nodeid, newvalues):
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1715 """Fire all registered auditors"""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1716 for _prio, _name, audit in self.auditors[event]:
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1717 try:
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1718 audit(self.db, self, nodeid, newvalues)
5084
675b3b3d88f0 Fix issue2550826 third try. This limits the capture of exceptions to
John Rouillard <rouilj@ieee.org>
parents: 5080
diff changeset
1719 except (EnvironmentError, ArithmeticError) as e:
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1720 tb = traceback.format_exc()
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1721 html = ("<h1>Traceback</h1>" + str(tb).replace('\n', '<br>').
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1722 replace(' ', '&nbsp;'))
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1723 txt = 'Caught exception %s: %s\n%s' % (str(type(e)), e, tb)
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1724 exc_info = sys.exc_info()
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1725 subject = "Error: %s" % exc_info[1]
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1726 raise DetectorError(subject, html, txt)
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1727
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1728 def react(self, event, detector, priority=100):
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1729 """Register a reactor detector"""
3743
e754cc14e76a fix unstable ordering of detectors [SF#1585378]
Richard Jones <richard@users.sourceforge.net>
parents: 3695
diff changeset
1730 self.reactors[event].append((priority, detector.__name__, detector))
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1731
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1732 def fireReactors(self, event, nodeid, oldvalues):
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1733 """Fire all registered reactors"""
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1734 for _prio, _name, react in self.reactors[event]:
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1735 try:
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1736 react(self.db, self, nodeid, oldvalues)
5084
675b3b3d88f0 Fix issue2550826 third try. This limits the capture of exceptions to
John Rouillard <rouilj@ieee.org>
parents: 5080
diff changeset
1737 except (EnvironmentError, ArithmeticError) as e:
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1738 tb = traceback.format_exc()
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1739 html = ("<h1>Traceback</h1>" + str(tb).replace('\n', '<br>').
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1740 replace(' ', '&nbsp;'))
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1741 txt = 'Caught exception %s: %s\n%s' % (str(type(e)), e, tb)
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1742 exc_info = sys.exc_info()
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1743 subject = "Error: %s" % exc_info[1]
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 5069
diff changeset
1744 raise DetectorError(subject, html, txt)
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1745
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1746 #
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1747 # import / export support
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1748 #
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1749 def export_propnames(self):
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1750 """List the property names for export from this Class"""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1751 propnames = sorted(self.getprops().keys())
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1752 return propnames
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1753
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1754 def import_journals(self, entries):
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1755 """Import a class's journal.
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1756
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1757 Uses setjournal() to set the journal for each item.
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1758 Strategy for import: Sort first by id, then import journals for
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1759 each id, this way the memory footprint is a lot smaller than the
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1760 initial implementation which stored everything in a big hash by
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1761 id and then proceeded to import journals for each id."""
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1762 properties = self.getprops()
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1763 a = []
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1764 for l in entries:
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1765 # first element in sorted list is the (numeric) id
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1766 # in python2.4 and up we would use sorted with a key...
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1767 a.append((int(l[0].strip("'")), l))
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1768 a.sort()
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1769
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1770 last = 0
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1771 r = []
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1772 for n, l in a:
5525
bb7865241f8a Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents: 5481
diff changeset
1773 nodeid, jdate, user, action, params = map(eval_import, l)
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1774 assert (str(n) == nodeid)
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1775 if n != last:
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1776 if r:
4431
9b619dcb030a - fix import (now passes for memorydb, too)
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
1777 self.db.setjournal(self.classname, str(last), r)
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1778 last = n
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1779 r = []
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1780
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1781 if action == 'set':
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1782 for propname, value in params.items():
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1783 prop = properties[propname]
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1784 if value is None:
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1785 pass
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1786 elif isinstance(prop, Date):
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1787 value = date.Date(value)
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1788 elif isinstance(prop, Interval):
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1789 value = date.Interval(value)
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1790 elif isinstance(prop, Password):
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4481
diff changeset
1791 value = password.JournalPassword(encrypted=value)
4430
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1792 params[propname] = value
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1793 elif action == 'create' and params:
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1794 # old tracker with data stored in the create!
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1795 params = {}
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1796 r.append((nodeid, date.Date(jdate), user, action, params))
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1797 if r:
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1798 self.db.setjournal(self.classname, nodeid, r)
f2f2904fe6ce - refactor: move import_journal to hyperdb
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4408
diff changeset
1799
4306
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1800 #
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1801 # convenience methods
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1802 #
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1803 def get_roles(self, nodeid):
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1804 """Return iterator for all roles for this nodeid.
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1805
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1806 Yields string-processed roles.
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1807 This method can be overridden to provide a hook where we can
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1808 insert other permission models (e.g. get roles from database)
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1809 In standard schemas only a user has a roles property but
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1810 this may be different in customized schemas.
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1811 Note that this is the *central place* where role
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1812 processing happens!
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1813 """
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1814 node = self.db.getnode(self.classname, nodeid)
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1815 return iter_roles(node['roles'])
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1816
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1817 def has_role(self, nodeid, *roles):
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1818 '''See if this node has any roles that appear in roles.
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4472
diff changeset
1819
4306
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1820 For convenience reasons we take a list.
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1821 In standard schemas only a user has a roles property but
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1822 this may be different in customized schemas.
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1823 '''
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1824 roles = dict.fromkeys([r.strip().lower() for r in roles])
4306
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1825 for role in self.get_roles(nodeid):
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1826 if role in roles:
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1827 return True
966592263fb8 Clean up all the places where role processing occurs.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4089
diff changeset
1828 return False
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1829
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1830
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1831 class HyperdbValueError(ValueError):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1832 """ Error converting a raw value into a Hyperdb value """
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1833 pass
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1834
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1835
5809
936275dfe1fa Try to fix:
John Rouillard <rouilj@ieee.org>
parents: 5807
diff changeset
1836 def convertLinkValue(db, propname, prop, value, idre=re.compile(r'^\d+$')):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1837 """ Convert the link value (may be id or key value) to an id value. """
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1838 linkcl = db.classes[prop.classname]
4877
2ba982dcdf2c New Link / Multilink option "try_id_parsing"
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4850
diff changeset
1839 if not idre or not idre.match(value):
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1840 if linkcl.getkey():
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1841 try:
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1842 value = linkcl.lookup(value)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1843 except KeyError:
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1844 raise HyperdbValueError(_('property %s: %r is not a %s.') % (
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5327
diff changeset
1845 propname, value, prop.classname))
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1846 else:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1847 raise HyperdbValueError(_('you may only enter ID values '
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1848 'for property %s') % propname)
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1849 return value
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1850
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1851
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1852 def fixNewlines(text):
1928
7c1ddebe7589 Small readability improvements.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1927
diff changeset
1853 """ Homogenise line endings.
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1854
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1855 Different web clients send different line ending values, but
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1856 other systems (eg. email) don't necessarily handle those line
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1857 endings. Our solution is to convert all line endings to LF.
1928
7c1ddebe7589 Small readability improvements.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1927
diff changeset
1858 """
5309
20084e2b48c3 Fix bug where unsetting a text value triggered an exception because
John Rouillard <rouilj@ieee.org>
parents: 5261
diff changeset
1859 if text is not None:
20084e2b48c3 Fix bug where unsetting a text value triggered an exception because
John Rouillard <rouilj@ieee.org>
parents: 5261
diff changeset
1860 text = text.replace('\r\n', '\n')
20084e2b48c3 Fix bug where unsetting a text value triggered an exception because
John Rouillard <rouilj@ieee.org>
parents: 5261
diff changeset
1861 return text.replace('\r', '\n')
20084e2b48c3 Fix bug where unsetting a text value triggered an exception because
John Rouillard <rouilj@ieee.org>
parents: 5261
diff changeset
1862 return text
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1863
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1864
3383
c1c67c62b699 applied patch [SF#236087]
Richard Jones <richard@users.sourceforge.net>
parents: 3019
diff changeset
1865 def rawToHyperdb(db, klass, itemid, propname, value, **kw):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1866 """ Convert the raw (user-input) value to a hyperdb-storable value. The
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1867 value is for the "propname" property on itemid (may be None for a
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1868 new item) of "klass" in "db".
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1869
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1870 The value is usually a string, but in the case of multilink inputs
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1871 it may be either a list of strings or a string with comma-separated
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1872 values.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1873 """
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1874 properties = klass.getprops()
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1875
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1876 # ensure it's a valid property name
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1877 propname = propname.strip()
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1878 try:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1879 proptype = properties[propname]
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1880 except KeyError:
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1881 raise HyperdbValueError(_('%r is not a property of %s') % (
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1882 propname, klass.classname))
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1883
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1884 # if we got a string, strip it now
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1885 if isinstance(value, type('')):
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1886 value = value.strip()
3668
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
1887
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1888 # convert the input value to a real property value
3668
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
1889 value = proptype.from_raw(value, db=db, klass=klass,
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1890 propname=propname, itemid=itemid, **kw)
3668
a15c15510e99 hyperdb handling of empty raw values for Multilink and Password [SF#1507814]
Richard Jones <richard@users.sourceforge.net>
parents: 3635
diff changeset
1891
1905
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1892 return value
dc43e339e607 Centralised conversion of user-input data to hyperdb values
Richard Jones <richard@users.sourceforge.net>
parents: 1864
diff changeset
1893
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1894
1442
b42fa71754c9 don't attempt to create FileClass items if no content is supplied
Richard Jones <richard@users.sourceforge.net>
parents: 1249
diff changeset
1895 class FileClass:
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1896 """ A class that requires the "content" property and stores it on
1442
b42fa71754c9 don't attempt to create FileClass items if no content is supplied
Richard Jones <richard@users.sourceforge.net>
parents: 1249
diff changeset
1897 disk.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1898 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
1899 default_mime_type = 'text/plain'
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
1900
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
1901 def __init__(self, db, classname, **properties):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1902 """The newly-created class automatically includes the "content"
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
1903 property.
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1904 """
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1905 if 'content' not in properties:
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3802
diff changeset
1906 properties['content'] = String(indexme='yes')
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
1907
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1908 def export_propnames(self):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1909 """ Don't export the "content" property
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1910 """
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1911 propnames = list(self.getprops().keys())
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1912 propnames.remove('content')
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1913 propnames.sort()
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1914 return propnames
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1915
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1916 def exportFilename(self, dirname, nodeid):
5101
d043a09952db Tried to explain the use of hyperdb's exportFilename
John Rouillard <rouilj@ieee.org>
parents: 5084
diff changeset
1917 """ Returns destination filename for a exported file
d043a09952db Tried to explain the use of hyperdb's exportFilename
John Rouillard <rouilj@ieee.org>
parents: 5084
diff changeset
1918
d043a09952db Tried to explain the use of hyperdb's exportFilename
John Rouillard <rouilj@ieee.org>
parents: 5084
diff changeset
1919 Called by export function in roundup admin to generate
d043a09952db Tried to explain the use of hyperdb's exportFilename
John Rouillard <rouilj@ieee.org>
parents: 5084
diff changeset
1920 the <class>-files subdirectory
d043a09952db Tried to explain the use of hyperdb's exportFilename
John Rouillard <rouilj@ieee.org>
parents: 5084
diff changeset
1921 """
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1922 subdir_filename = self.db.subdirFilename(self.classname, nodeid)
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1923 return os.path.join(dirname, self.classname+'-files', subdir_filename)
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1924
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1925 def export_files(self, dirname, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1926 """ Export the "content" property as a file, not csv column
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1927 """
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1928 source = self.db.filename(self.classname, nodeid)
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1929
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1930 dest = self.exportFilename(dirname, nodeid)
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1931 ensureParentsExist(dest)
3488
e4177cf4d30d common initialization code and detectors interface...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3476
diff changeset
1932 shutil.copyfile(source, dest)
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1933
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1934 def import_files(self, dirname, nodeid):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1935 """ Import the "content" property as a file
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1936 """
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1937 source = self.exportFilename(dirname, nodeid)
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1938
2962
4607f58a007b py2.1 compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 2909
diff changeset
1939 dest = self.db.filename(self.classname, nodeid, create=1)
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2962
diff changeset
1940 ensureParentsExist(dest)
2496
Richard Jones <richard@users.sourceforge.net>
parents: 2334
diff changeset
1941 shutil.copyfile(source, dest)
1442
b42fa71754c9 don't attempt to create FileClass items if no content is supplied
Richard Jones <richard@users.sourceforge.net>
parents: 1249
diff changeset
1942
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3491
diff changeset
1943 mime_type = None
3802
c3af8c6a6b5b Bug-fix: Only index the content property if it has the indexme attribute set
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3743
diff changeset
1944 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1945 if 'type' in props:
3545
507b2df02956 fixes to import_files
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
1946 mime_type = self.get(nodeid, 'type')
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3491
diff changeset
1947 if not mime_type:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3491
diff changeset
1948 mime_type = self.default_mime_type
3802
c3af8c6a6b5b Bug-fix: Only index the content property if it has the indexme attribute set
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3743
diff changeset
1949 if props['content'].indexme:
5648
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1950 index_content = self.get(nodeid, 'binary_content')
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1951 if bytes != str and isinstance(index_content, bytes):
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1952 index_content = index_content.decode('utf-8', errors='ignore')
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1953 # indexer will only index text mime type. It will skip
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1954 # other types. So if mime type of file is correct, we
a4514df51ded Python3: fix crash bug when importing binary file (like jpeg). The
John Rouillard <rouilj@ieee.org>
parents: 5548
diff changeset
1955 # call add_text on content.
3802
c3af8c6a6b5b Bug-fix: Only index the content property if it has the indexme attribute set
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3743
diff changeset
1956 self.db.indexer.add_text((self.classname, nodeid, 'content'),
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1957 index_content, mime_type)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1958
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3491
diff changeset
1959
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1960 class Node:
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1961 """ A convenience wrapper for the given node
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
1962 """
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 466
diff changeset
1963 def __init__(self, cl, nodeid, cache=1):
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1964 self.__dict__['cl'] = cl
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1965 self.__dict__['nodeid'] = nodeid
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1966
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1967 def keys(self, protected=1):
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5388
diff changeset
1968 return list(self.cl.getprops(protected=protected).keys())
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1969
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1970 def values(self, protected=1):
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1971 l = []
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1972 for name in self.cl.getprops(protected=protected).keys():
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1973 l.append(self.cl.get(self.nodeid, name))
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1974 return l
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1975
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1976 def items(self, protected=1):
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1977 l = []
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1978 for name in self.cl.getprops(protected=protected).keys():
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1979 l.append((name, self.cl.get(self.nodeid, name)))
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1980 return l
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1981
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1982 def has_key(self, name):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1983 return name in self.cl.getprops()
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1984
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
1985 def get(self, name, default=None):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1986 if name in self:
1470
9ccd69fbe33e applied patch [SF#688595]
Richard Jones <richard@users.sourceforge.net>
parents: 1442
diff changeset
1987 return self[name]
9ccd69fbe33e applied patch [SF#688595]
Richard Jones <richard@users.sourceforge.net>
parents: 1442
diff changeset
1988 else:
9ccd69fbe33e applied patch [SF#688595]
Richard Jones <richard@users.sourceforge.net>
parents: 1442
diff changeset
1989 return default
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
1990
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1991 def __getattr__(self, name):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1992 if name in self.__dict__:
278
a5dabf2430c5 Added database importing and exporting through CSV files.
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
1993 return self.__dict__[name]
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1994 try:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1995 return self.cl.get(self.nodeid, name)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
1996 except KeyError as value:
460
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 457
diff changeset
1997 # we trap this but re-raise it as AttributeError - all other
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 457
diff changeset
1998 # exceptions should pass through untrapped
5697
5a9159ad773f Properly handle mapping of KeyError to AttributeError in __getattr__
John Rouillard <rouilj@ieee.org>
parents: 5648
diff changeset
1999 raise AttributeError(str(value))
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2000
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2001 def __getitem__(self, name):
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
2002 return self.cl.get(self.nodeid, name)
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2003
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2004 def __setattr__(self, name, value):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2005 try:
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2006 return self.cl.set(self.nodeid, **{name: value})
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
2007 except KeyError as value:
5697
5a9159ad773f Properly handle mapping of KeyError to AttributeError in __getattr__
John Rouillard <rouilj@ieee.org>
parents: 5648
diff changeset
2008 # we trap this but re-raise it as AttributeError - all other
5a9159ad773f Properly handle mapping of KeyError to AttributeError in __getattr__
John Rouillard <rouilj@ieee.org>
parents: 5648
diff changeset
2009 # exceptions should pass through untrapped
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5327
diff changeset
2010 raise AttributeError(str(value))
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2011
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2012 def __setitem__(self, name, value):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2013 self.cl.set(self.nodeid, **{name: value})
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2014
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
2015 def history(self, enforceperm=True, skipquiet=True):
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
2016 return self.cl.history(self.nodeid,
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5175
diff changeset
2017 enforceperm=enforceperm,
6006
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2018 skipquiet=skipquiet)
cf800f1febe6 Flake8 fixes; use isinstance rather than type equality.
John Rouillard <rouilj@ieee.org>
parents: 5871
diff changeset
2019
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2020 def retire(self):
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2021 return self.cl.retire(self.nodeid)
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2022
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2023
648
07422fe57db2 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 628
diff changeset
2024 def Choice(name, db, *options):
4063
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
2025 """Quick helper to create a simple class with choices
625915ce35b8 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4048
diff changeset
2026 """
648
07422fe57db2 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 628
diff changeset
2027 cl = Class(db, name, name=String(), order=String())
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2028 for i in range(len(options)):
648
07422fe57db2 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 628
diff changeset
2029 cl.create(name=options[i], order=i)
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3802
diff changeset
2030 return Link(name)
25
4cf1daf2f2eb More Grande Splite
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2031
2909
2fc6c508b537 Database instances must have method close()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
2032 # vim: set filetype=python sts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/