Mercurial > p > roundup > code

annotate roundup/backends/back_anydbm.py @ 5444:167f0d25ea8e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Python 3 preparation: convert dbm keys back from bytes to strings. The Python 3 dbm module uses bytes for keys and values, converting any passed in strings to bytes and always returning bytes when keys are listed or values extracted. Bytes for values is fine with Roundup (which uses the marshal module to produce the values stored, which produces bytes anyway in Python 3), but bytes for keys need converting back to strings when keys are enumerated.
author Joseph Myers <jsm@polyomino.org.uk>
date Wed, 25 Jul 2018 12:31:32 +0000
parents 23b8e6067f7c
children 630a00b20394
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
1 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
4 # under the same terms as Python, so long as this copyright message and
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
5 # disclaimer are retained in their original form.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
6 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
11 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
17 #
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
18 """This module defines a backend that saves the hyperdatabase in a
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
19 database chosen by anydbm. It is guaranteed to always be available in python
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
20 versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
21 serious bugs, and is not available)
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
22 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
23 __docformat__ = 'restructuredtext'
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
25 import os, marshal, re, weakref, string, copy, time, shutil, logging
1809
bd127cafe3a8 Simplify backend importing, by moving the imports into the backend modules.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1800
diff changeset
26
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
27 from roundup.anypy.dbm_ import anydbm, whichdb
5444
167f0d25ea8e Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5395
diff changeset
28 from roundup.anypy.strings import b2s
2908
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
29
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
30 from roundup import hyperdb, date, password, roundupdb, security, support
2908
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
31 from roundup.backends import locking
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
32 from roundup.i18n import _
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
33
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
34 from roundup.backends.blobfiles import FileStorage
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
35 from roundup.backends.sessions_dbm import Sessions, OneTimeKeys
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
36
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
37 from roundup.backends.indexer_common import get_indexer
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
39 def db_exists(config):
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
40 # check for the user db
2637
11811b313459 The demo.py script works again using the new configuration system.
Richard Jones <richard@users.sourceforge.net>
parents: 2633
diff changeset
41 for db in 'nodes.user nodes.user.db'.split():
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
42 if os.path.exists(os.path.join(config.DATABASE, db)):
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
43 return 1
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
44 return 0
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
45
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
46 def db_nuke(config):
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
47 shutil.rmtree(config.DATABASE)
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
48
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
49 class Binary:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
50
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
51 def __init__(self, x, y):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
52 self.x = x
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
53 self.y = y
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
54
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
55 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
56 self.x.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
57 self.y.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
58
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
59 class Unary:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
60
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
61 def __init__(self, x):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
62 self.x = x
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
63
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
64 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
65 return atom(self)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
66
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
67 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
68 self.x.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
69
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
70 class Equals(Unary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
71
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
72 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
73 return self.x in v
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
74
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
75 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
76 visitor(self)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
77
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
78 class Not(Unary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
79
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
80 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
81 return not self.x.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
82
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
83 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
84 return "NOT(%s)" % self.x.generate(atom)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
85
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
86 class Or(Binary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
87
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
88 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
89 return self.x.evaluate(v) or self.y.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
90
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
91 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
92 return "(%s)OR(%s)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
93 self.x.generate(atom),
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
94 self.y.generate(atom))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
95
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
96 class And(Binary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
97
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
98 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
99 return self.x.evaluate(v) and self.y.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
100
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
101 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
102 return "(%s)AND(%s)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
103 self.x.generate(atom),
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
104 self.y.generate(atom))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
105
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
106 def compile_expression(opcodes):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
107
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
108 stack = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
109 push, pop = stack.append, stack.pop
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
110 for opcode in opcodes:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
111 if opcode == -2: push(Not(pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
112 elif opcode == -3: push(And(pop(), pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
113 elif opcode == -4: push(Or(pop(), pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
114 else: push(Equals(opcode))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
115
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
116 return pop()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
117
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
118 class Expression:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
119
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
120 def __init__(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
121 try:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
122 opcodes = [int(x) for x in v]
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
123 if min(opcodes) >= -1: raise ValueError()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
124
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
125 compiled = compile_expression(opcodes)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
126 self.evaluate = lambda x: compiled.evaluate([int(y) for y in x])
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
127 except:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
128 self.evaluate = lambda x: bool(set(x) & set(v))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
129
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 # Now the database
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
132 #
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
133 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
134 """A database for storing records containing flexible data types.
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
135
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
136 Transaction stuff TODO:
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
137
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
138 - check the timestamp of the class file and nuke the cache if it's
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
139 modified. Do some sort of conflict checking on the dirty stuff.
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
140 - perhaps detect write collisions (related to above)?
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
141
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
142 attributes:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
143 dbtype:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
144 holds the value for the type of db. It is used by indexer to
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
145 identify the database type so it can import the correct indexer
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
146 module when using native text search mode.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
147 """
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
148
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
149 dbtype = "anydbm"
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
150
5100
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
151 # used by migrate roundup_admin command. Is a no-op for anydbm.
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
152 # but needed to stop traceback in admin.
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
153 db_version_updated = False
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
154
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
155 def __init__(self, config, journaltag=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
156 """Open a hyperdatabase given a specifier to some storage.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
158 The 'storagelocator' is obtained from config.DATABASE.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 The meaning of 'storagelocator' depends on the particular
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 implementation of the hyperdatabase. It could be a file name,
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
161 a directory path, a socket descriptor for a connection to a
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
162 database over the network, etc.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
163
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164 The 'journaltag' is a token that will be attached to the journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 entries for any edits done on the database. If 'journaltag' is
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
166 None, the database is opened in read-only mode: the Class.create(),
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
167 Class.set(), Class.retire(), and Class.restore() methods are
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
168 disabled.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
169 """
3897
e2c2d91932ad respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3872
diff changeset
170 FileStorage.__init__(self, config.UMASK)
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
171 self.config, self.journaltag = config, journaltag
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
172 self.dir = config.DATABASE
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 self.classes = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
174 self.cache = {} # cache of nodes loaded or created
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
175 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
176 'filtering': 0}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
177 self.dirtynodes = {} # keep track of the dirty nodes by class
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
178 self.newnodes = {} # keep track of the new nodes by class
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
179 self.destroyednodes = {}# keep track of the destroyed nodes by class
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
180 self.transactions = []
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
181 self.indexer = get_indexer(config, self)
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
182 self.security = security.Security(self)
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
183 os.umask(config.UMASK)
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
184
5041
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
185 # make sure the database directory exists
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
186 if not os.path.isdir(self.config.DATABASE):
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
187 os.makedirs(self.config.DATABASE)
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
188
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
189 # lock it
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
190 lockfilenm = os.path.join(self.dir, 'lock')
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
191 self.lockfile = locking.acquire_lock(lockfilenm)
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
192 self.lockfile.write(str(os.getpid()))
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
193 self.lockfile.flush()
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
194
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
195 self.Session = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
196 self.Otk = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
197
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
198 def post_init(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
199 """Called once the schema initialisation has finished.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
200 """
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
201 # reindex the db if necessary
826
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
202 if self.indexer.should_reindex():
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
203 self.reindex()
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
204
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
205 def refresh_database(self):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
206 """Rebuild the database
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
207 """
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
208 self.reindex()
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
209
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
210 def getSessionManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
211 if not self.Session:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
212 self.Session = Sessions(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
213 return self.Session
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
214
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
215 def getOTKManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
216 if not self.Otk:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
217 self.Otk = OneTimeKeys(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
218 return self.Otk
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
219
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
220 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
221 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
222 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
223 else:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
224 classes = self.classes.values()
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
225 for klass in classes:
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
226 if show_progress:
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
227 for nodeid in support.Progress('Reindex %s'%klass.classname,
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
228 klass.list()):
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
229 klass.index(nodeid)
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
230 else:
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
231 for nodeid in klass.list():
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
232 klass.index(nodeid)
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
233 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
234
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
235 def __repr__(self):
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
236 return '<back_anydbm instance at %x>'%id(self)
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
237
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 # Classes
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241 def __getattr__(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
242 """A convenient way of calling self.getclass(classname)."""
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
243 if classname in self.classes:
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
244 return self.classes[classname]
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
245 raise AttributeError(classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
247 def addclass(self, cl):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
248 cn = cl.classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
249 if cn in self.classes:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
250 raise ValueError(cn)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251 self.classes[cn] = cl
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
252
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
253 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
254 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
255 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
256 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
257 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
258 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
259 description="User is allowed to access "+cn)
4518
e572c5e28d0b Fixed typo in permission "Reture" -> "Retire".
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4517
diff changeset
260 self.security.addPermission(name="Retire", klass=cn,
4517
f8e85cf5f0fe how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents: 4483
diff changeset
261 description="User is allowed to retire "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
262
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
263 def getclasses(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
264 """Return a list of the names of all existing classes."""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
265 return sorted(self.classes.keys())
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
266
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
267 def getclass(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
268 """Get the Class object representing a particular class.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270 If 'classname' is not a valid class name, a KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
271 """
1145
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
272 try:
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
273 return self.classes[classname]
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
274 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
275 raise KeyError('There is no class called "%s"'%classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
276
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
277 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
278 # Class DBs
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
279 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
280 def clear(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
281 """Delete all database contents
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
282 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
283 logging.getLogger('roundup.hyperdb.backend').info('clear')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
284 for cn in self.classes:
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
285 for dummy in 'nodes', 'journals':
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
286 path = os.path.join(self.dir, 'journals.%s'%cn)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
287 if os.path.exists(path):
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
288 os.remove(path)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
289 elif os.path.exists(path+'.db'): # dbm appends .db
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
290 os.remove(path+'.db')
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
291 # reset id sequences
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
292 path = os.path.join(os.getcwd(), self.dir, '_ids')
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
293 if os.path.exists(path):
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
294 os.remove(path)
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
295 elif os.path.exists(path+'.db'): # dbm appends .db
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
296 os.remove(path+'.db')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
297
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
298 def getclassdb(self, classname, mode='r'):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
299 """ grab a connection to the class db that will be used for
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
300 multiple actions
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
301 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
302 return self.opendb('nodes.%s'%classname, mode)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
303
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
304 def determine_db_type(self, path):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
305 """ determine which DB wrote the class file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
306 """
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
307 db_type = ''
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
308 if os.path.exists(path):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
309 db_type = whichdb(path)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
310 if not db_type:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
311 raise hyperdb.DatabaseError(_("Couldn't identify database type"))
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
312 elif os.path.exists(path+'.db'):
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
313 # if the path ends in '.db', it's a dbm database, whether
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
314 # anydbm says it's dbhash or not!
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
315 db_type = 'dbm'
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
316 return db_type
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
317
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
318 def opendb(self, name, mode):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
319 """Low-level database opener that gets around anydbm/dbm
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
320 eccentricities.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
321 """
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
322 # figure the class db type
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
323 path = os.path.join(os.getcwd(), self.dir, name)
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
324 db_type = self.determine_db_type(path)
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
325
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
326 # new database? let anydbm pick the best dbm
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
327 # in Python 3+ the "dbm" ("anydbm" to us) module already uses the
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
328 # whichdb() function to do this
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
329 if not db_type or hasattr(anydbm, 'whichdb'):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
330 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
331 logging.getLogger('roundup.hyperdb.backend').debug(
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
332 "opendb anydbm.open(%r, 'c')"%path)
1028
16498e77e3ff allow overiding of the index args roundup/cgi/templating.py
Richard Jones <richard@users.sourceforge.net>
parents: 1022
diff changeset
333 return anydbm.open(path, 'c')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
334
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
335 # in Python <3 it anydbm was a little dumb so manually open the
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
336 # database with the correct module
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
337 try:
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
338 dbm = __import__(db_type)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
339 except ImportError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
340 raise hyperdb.DatabaseError(_("Couldn't open database - the "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
341 "required module '%s' is not available")%db_type)
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
342 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
343 logging.getLogger('roundup.hyperdb.backend').debug(
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
344 "opendb %r.open(%r, %r)"%(db_type, path, mode))
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
345 return dbm.open(path, mode)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
346
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
347 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
348 # Node IDs
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
349 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
350 def newid(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
351 """ Generate a new id for the given class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
352 """
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
353 # open the ids DB - create if if doesn't exist
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
354 db = self.opendb('_ids', 'c')
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
355 if classname in db:
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
356 newid = db[classname] = str(int(db[classname]) + 1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
357 else:
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
358 # the count() bit is transitional - older dbs won't start at 1
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
359 newid = str(self.getclass(classname).count()+1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
360 db[classname] = newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
361 db.close()
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
362 return newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
363
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
364 def setid(self, classname, setid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
365 """ Set the id counter: used during import of database
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
366 """
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
367 # open the ids DB - create if if doesn't exist
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
368 db = self.opendb('_ids', 'c')
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
369 db[classname] = str(setid)
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
370 db.close()
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
371
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
372 #
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
373 # Nodes
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
374 #
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
375 def addnode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
376 """ add the specified node to its class's db
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
377 """
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
378 # we'll be supplied these props if we're doing an import
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
379 if 'creator' not in node:
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
380 # add in the "calculated" properties (dupe so we don't affect
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
381 # calling code's node assumptions)
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
382 node = node.copy()
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
383 node['creator'] = self.getuid()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
384 node['actor'] = self.getuid()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
385 node['creation'] = node['activity'] = date.Date()
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
386
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
387 self.newnodes.setdefault(classname, {})[nodeid] = 1
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
388 self.cache.setdefault(classname, {})[nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
389 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
390
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
391 def setnode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
392 """ change the specified node
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
393 """
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
394 self.dirtynodes.setdefault(classname, {})[nodeid] = 1
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
395
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
396 # can't set without having already loaded the node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
397 self.cache[classname][nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
398 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
399
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
400 def savenode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
401 """ perform the saving of data specified by the set/addnode
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
402 """
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
403 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
404 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
405 'save %s%s %r'%(classname, nodeid, node))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
406 self.transactions.append((self.doSaveNode, (classname, nodeid, node)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
407
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
408 def getnode(self, classname, nodeid, db=None, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
409 """ get a node from the database
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
410
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
411 Note the "cache" parameter is not used, and exists purely for
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
412 backward compatibility!
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
413 """
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
414 # try the cache
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
415 cache_dict = self.cache.setdefault(classname, {})
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
416 if nodeid in cache_dict:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
417 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
418 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
419 'get %s%s cached'%(classname, nodeid))
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
420 self.stats['cache_hits'] += 1
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
421 return cache_dict[nodeid]
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
422
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
423 if __debug__:
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
424 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
425 start_t = time.time()
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
426 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
427 'get %s%s'%(classname, nodeid))
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
428
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
429 # get from the database and save in the cache
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
430 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
431 db = self.getclassdb(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
432 if nodeid not in db:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
433 raise IndexError("no such %s %s"%(classname, nodeid))
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
434
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
435 # check the uncommitted, destroyed nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
436 if (classname in self.destroyednodes and
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
437 nodeid in self.destroyednodes[classname]):
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
438 raise IndexError("no such %s %s"%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
439
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
440 # decode
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
441 res = marshal.loads(db[nodeid])
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
442
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
443 # reverse the serialisation
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
444 res = self.unserialise(classname, res)
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
445
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
446 # store off in the cache dict
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
447 if cache:
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
448 cache_dict[nodeid] = res
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
449
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
450 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
451 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
452
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
453 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
454
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
455 def destroynode(self, classname, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
456 """Remove a node from the database. Called exclusively by the
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
457 destroy() method on Class.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
458 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
459 logging.getLogger('roundup.hyperdb.backend').info(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
460 'destroy %s%s'%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
461
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
462 # remove from cache and newnodes if it's there
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
463 if (classname in self.cache and nodeid in self.cache[classname]):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
464 del self.cache[classname][nodeid]
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
465 if (classname in self.newnodes and nodeid in self.newnodes[classname]):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
466 del self.newnodes[classname][nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
467
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
468 # see if there's any obvious commit actions that we should get rid of
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
469 for entry in self.transactions[:]:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
470 if entry[1][:2] == (classname, nodeid):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
471 self.transactions.remove(entry)
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
472
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
473 # add to the destroyednodes map
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
474 self.destroyednodes.setdefault(classname, {})[nodeid] = 1
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
475
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
476 # add the destroy commit action
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
477 self.transactions.append((self.doDestroyNode, (classname, nodeid)))
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
478 self.transactions.append((FileStorage.destroy, (self, classname, nodeid)))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
479
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
480 def serialise(self, classname, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
481 """Copy the node contents, converting non-marshallable data into
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
482 marshallable data.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
483 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
484 properties = self.getclass(classname).getprops()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
485 d = {}
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
486 for k, v in node.items():
2964
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
487 if k == self.RETIRED_FLAG:
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
488 d[k] = v
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
489 continue
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
490
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
491 # if the property doesn't exist then we really don't care
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
492 if k not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
493 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
494
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
495 # get the property spec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
496 prop = properties[k]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
497
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
498 if isinstance(prop, hyperdb.Password) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
499 d[k] = str(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
500 elif isinstance(prop, hyperdb.Date) and v is not None:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
501 d[k] = v.serialise()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
502 elif isinstance(prop, hyperdb.Interval) and v is not None:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
503 d[k] = v.serialise()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
504 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
505 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
506 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
507
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
508 def unserialise(self, classname, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
509 """Decode the marshalled node data
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
510 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
511 properties = self.getclass(classname).getprops()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
512 d = {}
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
513 for k, v in node.items():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
514 # if the property doesn't exist, or is the "retired" flag then
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
515 # it won't be in the properties dict
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
516 if k not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
517 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
518 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
519
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
520 # get the property spec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
521 prop = properties[k]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
522
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
523 if isinstance(prop, hyperdb.Date) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
524 d[k] = date.Date(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
525 elif isinstance(prop, hyperdb.Interval) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
526 d[k] = date.Interval(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
527 elif isinstance(prop, hyperdb.Password) and v is not None:
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
528 d[k] = password.Password(encrypted=v)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
529 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
530 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
531 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
532
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
533 def hasnode(self, classname, nodeid, db=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
534 """ determine if the database has a given node
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
535 """
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
536 # try the cache
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
537 cache = self.cache.setdefault(classname, {})
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
538 if nodeid in cache:
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
539 return 1
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
540
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
541 # not in the cache - check the database
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
542 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
543 db = self.getclassdb(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
544 return nodeid in db
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
545
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
546 def countnodes(self, classname, db=None):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
547 count = 0
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
548
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
549 # include the uncommitted nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
550 if classname in self.newnodes:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
551 count += len(self.newnodes[classname])
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
552 if classname in self.destroyednodes:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
553 count -= len(self.destroyednodes[classname])
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
554
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
555 # and count those in the DB
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
556 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
557 db = self.getclassdb(classname)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
558 return count + len(db)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
559
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
560
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
561 #
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
562 # Files - special node properties
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
563 # inherited from FileStorage
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
564
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
565 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
566 # Journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
567 #
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
568 def addjournal(self, classname, nodeid, action, params, creator=None,
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
569 creation=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
570 """ Journal the Action
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
571 'action' may be:
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
572
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
573 'set' -- 'params' is a dictionary of property values
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
574 'create' -- 'params' is an empty dictionary as of
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
575 Wed Nov 06 11:38:43 2002 +0000
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
576 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
577 'retired' or 'restored' -- 'params' is None
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
578
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
579 'creator' -- the user performing the action, which defaults to
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
580 the current user.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
581 """
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
582 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
583 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
584 'addjournal %s%s %s %r %s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
585 nodeid, action, params, creator, creation))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
586 if creator is None:
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
587 creator = self.getuid()
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
588 self.transactions.append((self.doSaveJournal, (classname, nodeid,
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
589 action, params, creator, creation)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
590
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
591 def setjournal(self, classname, nodeid, journal):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
592 """Set the journal to the "journal" list."""
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
593 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
594 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
595 'setjournal %s%s %r'%(classname, nodeid, journal))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
596 self.transactions.append((self.doSetJournal, (classname, nodeid,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
597 journal)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
598
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
599 def fix_journal(self, classname, journal):
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
600 """ fix password entries to correct type """
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
601 pwprops = {}
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
602 for pn, prop in self.getclass(classname).properties.items():
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
603 if isinstance(prop, hyperdb.Password):
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
604 pwprops [pn] = 1
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
605 if not pwprops:
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
606 return journal
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
607 for j in journal:
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
608 if j[3] == 'set':
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
609 for k, v in j[4].items():
4538
fd972e18b21a - fix import/export regression test for anydbm for latest journal fix
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4534
diff changeset
610 if k in pwprops and j[4][k]:
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
611 j[4][k] = password.JournalPassword(j[4][k])
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
612 return journal
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
613
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
614 def getjournal(self, classname, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
615 """ get the journal for id
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
616
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
617 Raise IndexError if the node doesn't exist (as per history()'s
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
618 API)
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
619 """
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
620 # our journal result
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
621 res = []
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
622
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
623 # add any journal entries for transactions not committed to the
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
624 # database
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
625 for method, args in self.transactions:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
626 if method != self.doSaveJournal:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
627 continue
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
628 (cache_classname, cache_nodeid, cache_action, cache_params,
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
629 cache_creator, cache_creation) = args
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
630 if cache_classname == classname and cache_nodeid == nodeid:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
631 if not cache_creator:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
632 cache_creator = self.getuid()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
633 if not cache_creation:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
634 cache_creation = date.Date()
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
635 res.append((cache_nodeid, cache_creation, cache_creator,
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
636 cache_action, cache_params))
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
637
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
638 # attempt to open the journal - in some rare cases, the journal may
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
639 # not exist
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
640 try:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
641 db = self.opendb('journals.%s'%classname, 'r')
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
642 except anydbm.error as error:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
643 if str(error) == "need 'c' or 'n' flag to open new db":
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
644 raise IndexError('no such %s %s'%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
645 elif error.args[0] != 2:
1920
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
646 # this isn't a "not found" error, be alarmed!
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
647 raise
1920
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
648 if res:
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
649 # we have unsaved journal entries, return them
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
650 return self.fix_journal (classname, res)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
651 raise IndexError('no such %s %s'%(classname, nodeid))
787
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
652 try:
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
653 journal = marshal.loads(db[nodeid])
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
654 except KeyError:
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
655 db.close()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
656 if res:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
657 # we have some unsaved journal entries, be happy!
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
658 return self.fix_journal (classname, res)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
659 raise IndexError('no such %s %s'%(classname, nodeid))
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
660 db.close()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
661
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
662 # add all the saved journal entries for this node
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
663 for nodeid, date_stamp, user, action, params in journal:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
664 res.append((nodeid, date.Date(date_stamp), user, action, params))
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
665 return self.fix_journal (classname, res)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
666
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
667 def pack(self, pack_before):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
668 """ Delete all journal entries except "create" before 'pack_before'.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
669 """
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
670 pack_before = pack_before.serialise()
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
671 for classname in self.getclasses():
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
672 packed = 0
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
673 # get the journal db
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
674 db_name = 'journals.%s'%classname
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
675 path = os.path.join(os.getcwd(), self.dir, classname)
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
676 db_type = self.determine_db_type(path)
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
677 db = self.opendb(db_name, 'w')
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
678
5444
167f0d25ea8e Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5395
diff changeset
679 for key in map(b2s, db.keys()):
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
680 # get the journal for this db entry
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
681 journal = marshal.loads(db[key])
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
682 l = []
567
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
683 last_set_entry = None
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
684 for entry in journal:
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
685 # unpack the entry
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
686 (nodeid, date_stamp, self.journaltag, action,
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
687 params) = entry
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
688 # if the entry is after the pack date, _or_ the initial
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
689 # create entry, then it stays
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
690 if date_stamp > pack_before or action == 'create':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
691 l.append(entry)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
692 else:
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
693 packed += 1
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
694 db[key] = marshal.dumps(l)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
695
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
696 logging.getLogger('roundup.hyperdb.backend').info(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
697 'packed %d %s items'%(packed, classname))
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
698
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
699 if db_type == 'gdbm':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
700 db.reorganize()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
701 db.close()
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
702
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
703
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
704 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
705 # Basic transaction support
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
706 #
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
707 def commit(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
708 """ Commit the current transactions.
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
709
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
710 Save all data changed since the database was opened or since the
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
711 last commit() or rollback().
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
712 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
713 logging.getLogger('roundup.hyperdb.backend').info('commit %s transactions'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
714 len(self.transactions)))
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
715
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
716 # keep a handle to all the database files opened
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
717 self.databases = {}
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
718
1904
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
719 try:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
720 # now, do all the transactions
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
721 reindex = {}
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
722 for method, args in self.transactions:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
723 reindex[method(*args)] = 1
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
724 finally:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
725 # make sure we close all the database files
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
726 for db in self.databases.values():
1904
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
727 db.close()
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
728 del self.databases
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
729
3960
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
730 # clear the transactions list now so the blobfile implementation
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
731 # doesn't think there's still pending file commits when it tries
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
732 # to access the file data
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
733 self.transactions = []
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
734
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
735 # reindex the nodes that request it
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
736 for classname, nodeid in [k for k in reindex if k]:
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
737 self.getclass(classname).index(nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
738
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
739 # save the indexer state
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
740 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
741
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
742 self.clearCache()
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
743
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
744 def clearCache(self):
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
745 # all transactions committed, back to normal
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
746 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
747 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
748 self.newnodes = {}
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
749 self.destroyednodes = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
750 self.transactions = []
4652
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4538
diff changeset
751 # upcall is necessary!
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4538
diff changeset
752 roundupdb.Database.clearCache(self)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
753
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
754 def getCachedClassDB(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
755 """ get the class db, looking in our cache of databases for commit
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
756 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
757 # get the database handle
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
758 db_name = 'nodes.%s'%classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
759 if db_name not in self.databases:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
760 self.databases[db_name] = self.getclassdb(classname, 'c')
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
761 return self.databases[db_name]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
762
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
763 def doSaveNode(self, classname, nodeid, node):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
764 db = self.getCachedClassDB(classname)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
765
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
766 # now save the marshalled data
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
767 db[nodeid] = marshal.dumps(self.serialise(classname, node))
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
768
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
769 # return the classname, nodeid so we reindex this content
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
770 return (classname, nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
771
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
772 def getCachedJournalDB(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
773 """ get the journal db, looking in our cache of databases for commit
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
774 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
775 # get the database handle
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
776 db_name = 'journals.%s'%classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
777 if db_name not in self.databases:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
778 self.databases[db_name] = self.opendb(db_name, 'c')
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
779 return self.databases[db_name]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
780
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
781 def doSaveJournal(self, classname, nodeid, action, params, creator,
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
782 creation):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
783 # serialise the parameters now if necessary
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
784 if isinstance(params, type({})):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
785 if action in ('set', 'create'):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
786 params = self.serialise(classname, params)
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
787
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
788 # handle supply of the special journalling parameters (usually
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
789 # supplied on importing an existing database)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
790 journaltag = creator
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
791 if creation:
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
792 journaldate = creation.serialise()
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
793 else:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
794 journaldate = date.Date().serialise()
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
795
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
796 # create the journal entry
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
797 entry = (nodeid, journaldate, journaltag, action, params)
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
798
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
799 db = self.getCachedJournalDB(classname)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
800
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
801 # now insert the journal entry
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
802 if nodeid in db:
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
803 # append to existing
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
804 s = db[nodeid]
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
805 l = marshal.loads(s)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
806 l.append(entry)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
807 else:
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
808 l = [entry]
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
809
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
810 db[nodeid] = marshal.dumps(l)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
811
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
812 def doSetJournal(self, classname, nodeid, journal):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
813 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
814 for nodeid, journaldate, journaltag, action, params in journal:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
815 # serialise the parameters now if necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
816 if isinstance(params, type({})):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
817 if action in ('set', 'create'):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
818 params = self.serialise(classname, params)
2512
f5542d92307a fix anydbm journal import [SF#983166]
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
819 journaldate = journaldate.serialise()
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
820 l.append((nodeid, journaldate, journaltag, action, params))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
821 db = self.getCachedJournalDB(classname)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
822 db[nodeid] = marshal.dumps(l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
823
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
824 def doDestroyNode(self, classname, nodeid):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
825 # delete from the class database
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
826 db = self.getCachedClassDB(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
827 if nodeid in db:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
828 del db[nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
829
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
830 # delete from the database
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
831 db = self.getCachedJournalDB(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
832 if nodeid in db:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
833 del db[nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
834
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
835 def rollback(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
836 """ Reverse all actions from the current transaction.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
837 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
838 logging.getLogger('roundup.hyperdb.backend').info('rollback %s transactions'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
839 len(self.transactions)))
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
840
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
841 for method, args in self.transactions:
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
842 # delete temporary files
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
843 if method == self.doStoreFile:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
844 self.rollbackStoreFile(*args)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
845 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
846 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
847 self.newnodes = {}
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
848 self.destroyednodes = {}
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
849 self.transactions = []
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
850
1131
92e92ae58494 add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents: 1127
diff changeset
851 def close(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
852 """ Nothing to do
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
853 """
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
854 if self.lockfile is not None:
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
855 locking.release_lock(self.lockfile)
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
856 self.lockfile.close()
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
857 self.lockfile = None
1131
92e92ae58494 add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents: 1127
diff changeset
858
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
859 _marker = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
860 class Class(hyperdb.Class):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
861 """The handle to a particular class of nodes in a hyperdatabase."""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
862
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
863 def enableJournalling(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
864 """Turn journalling on for this class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
865 """
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
866 self.do_journal = 1
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
867
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
868 def disableJournalling(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
869 """Turn journalling off for this class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
870 """
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
871 self.do_journal = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
872
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
873 # Editing nodes:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
874
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
875 def create(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
876 """Create a new node of this class and return its id.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
877
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
878 The keyword arguments in 'propvalues' map property names to values.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
879
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
880 The values of arguments must be acceptable for the types of their
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
881 corresponding properties or a TypeError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
882
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
883 If this class has a key property, it must be present and its value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
884 must not collide with other key strings or a ValueError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
885
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
886 Any other properties on this class that are missing from the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
887 'propvalues' dictionary are set to None.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
888
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
889 If an id in a link or multilink property does not refer to a valid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
890 node, an IndexError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
891
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
892 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
893 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
894 """
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
895 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
896 raise hyperdb.DatabaseError(_('Database open read-only'))
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
897 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
898 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
899 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
900 return newid
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
901
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
902 def create_inner(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
903 """ Called by create, in-between the audit and react calls.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
904 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
905 if 'id' in propvalues:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
906 raise KeyError('"id" is reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
907
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
908 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
909 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
910
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
911 if 'creation' in propvalues or 'activity' in propvalues:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
912 raise KeyError('"creation" and "activity" are reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
913 # new node's id
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
914 newid = self.db.newid(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
915
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
916 # validate propvalues
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
917 num_re = re.compile('^\d+$')
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
918 for key, value in propvalues.items():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
919 if key == self.key:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
920 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
921 self.lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
922 except KeyError:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
923 pass
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
924 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
925 raise ValueError('node with key "%s" exists'%value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
926
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
927 # try to handle this property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
928 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
929 prop = self.properties[key]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
930 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
931 raise KeyError('"%s" has no property "%s"'%(self.classname,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
932 key))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
933
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
934 if value is not None and isinstance(prop, hyperdb.Link):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
935 if type(value) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
936 raise ValueError('link value must be String')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
937 link_class = self.properties[key].classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
938 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
939 if not num_re.match(value):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
940 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
941 value = self.db.classes[link_class].lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
942 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
943 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
944 key, value, link_class))
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
945 elif not self.db.getclass(link_class).hasnode(value):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
946 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
947 value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
948
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
949 # save off the value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
950 propvalues[key] = value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
951
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
952 # register the link with the newly linked node
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
953 if self.do_journal and self.properties[key].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
954 self.db.addjournal(link_class, value, 'link',
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
955 (self.classname, newid, key))
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
956
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
957 elif isinstance(prop, hyperdb.Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
958 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
959 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3826
diff changeset
960 if not hasattr(value, '__iter__'):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
961 raise TypeError('new property "%s" not an iterable of ids'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
962
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
963 # clean up and validate the list of links
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
964 link_class = self.properties[key].classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
965 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
966 for entry in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
967 if type(entry) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
968 raise ValueError('"%s" multilink value (%r) '\
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
969 'must contain Strings'%(key, value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
970 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
971 if not num_re.match(entry):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
972 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
973 entry = self.db.classes[link_class].lookup(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
974 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
975 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
976 key, entry, self.properties[key].classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
977 l.append(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
978 value = l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
979 propvalues[key] = value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
980
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
981 # handle additions
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
982 for nodeid in value:
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
983 if not self.db.getclass(link_class).hasnode(nodeid):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
984 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
985 nodeid))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
986 # register the link with the newly linked node
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
987 if self.do_journal and self.properties[key].do_journal:
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
988 self.db.addjournal(link_class, nodeid, 'link',
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
989 (self.classname, newid, key))
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
990
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
991 elif isinstance(prop, hyperdb.String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1351
diff changeset
992 if type(value) != type('') and type(value) != type(u''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
993 raise TypeError('new property "%s" not a string'%key)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
994 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
995 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
996 value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
997
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
998 elif isinstance(prop, hyperdb.Password):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
999 if not isinstance(value, password.Password):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1000 raise TypeError('new property "%s" not a Password'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1001
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1002 elif isinstance(prop, hyperdb.Date):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1003 if value is not None and not isinstance(value, date.Date):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1004 raise TypeError('new property "%s" not a Date'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1005
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1006 elif isinstance(prop, hyperdb.Interval):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1007 if value is not None and not isinstance(value, date.Interval):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1008 raise TypeError('new property "%s" not an Interval'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1009
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1010 elif value is not None and isinstance(prop, hyperdb.Number):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1011 try:
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1012 float(value)
887
e7169d6e6e45 added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents: 886
diff changeset
1013 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1014 raise TypeError('new property "%s" not numeric'%key)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1015
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1016 elif value is not None and isinstance(prop, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1017 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1018 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1019 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1020 raise TypeError('new property "%s" not an integer'%key)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1021
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1022 elif value is not None and isinstance(prop, hyperdb.Boolean):
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1023 try:
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1024 int(value)
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1025 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1026 raise TypeError('new property "%s" not boolean'%key)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1027
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1028 # make sure there's data where there needs to be
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1029 for key, prop in self.properties.items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1030 if key in propvalues:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1031 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1032 if key == self.key:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1033 raise ValueError('key property "%s" is required'%key)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1034 if isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1035 propvalues[key] = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1036
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1037 # done
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1038 self.db.addnode(self.classname, newid, propvalues)
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1039 if self.do_journal:
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1040 self.db.addjournal(self.classname, newid, 'create', {})
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1041
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1042 return newid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1043
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1044 def get(self, nodeid, propname, default=_marker, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1045 """Get the value of a property on an existing node of this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1046
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1047 'nodeid' must be the id of an existing node of this class or an
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1048 IndexError is raised. 'propname' must be the name of a property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1049 of this class or a KeyError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1050
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1051 'cache' exists for backward compatibility, and is not used.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1052
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1053 Attempts to get the "creation" or "activity" properties should
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1054 do the right thing.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1055 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1056 if propname == 'id':
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1057 return nodeid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1058
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1059 # get the node's dict
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1060 d = self.db.getnode(self.classname, nodeid)
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1061
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1062 # check for one of the special props
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1063 if propname == 'creation':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1064 if 'creation' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1065 return d['creation']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1066 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1067 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1068 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1069 if journal:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents: 4075
diff changeset
1070 return journal[0][1]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1071 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1072 # on the strange chance that there's no journal
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1073 return date.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1074 if propname == 'activity':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1075 if 'activity' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1076 return d['activity']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1077 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1078 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1079 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1080 if journal:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1081 return self.db.getjournal(self.classname, nodeid)[-1][1]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1082 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1083 # on the strange chance that there's no journal
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1084 return date.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1085 if propname == 'creator':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1086 if 'creator' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1087 return d['creator']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1088 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1089 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1090 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1091 if journal:
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1092 num_re = re.compile('^\d+$')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1093 value = journal[0][2]
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1094 if num_re.match(value):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1095 return value
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1096 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1097 # old-style "username" journal tag
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1098 try:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1099 return self.db.user.lookup(value)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1100 except KeyError:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1101 # user's been retired, return admin
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1102 return '1'
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1103 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1104 return self.db.getuid()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1105 if propname == 'actor':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1106 if 'actor' in d:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1107 return d['actor']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1108 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1109 raise ValueError('Journalling is disabled for this class')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1110 journal = self.db.getjournal(self.classname, nodeid)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1111 if journal:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1112 num_re = re.compile('^\d+$')
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1113 value = journal[-1][2]
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1114 if num_re.match(value):
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1115 return value
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1116 else:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1117 # old-style "username" journal tag
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1118 try:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1119 return self.db.user.lookup(value)
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1120 except KeyError:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1121 # user's been retired, return admin
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1122 return '1'
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1123 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1124 return self.db.getuid()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1125
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1126 # get the property (raises KeyErorr if invalid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1127 prop = self.properties[propname]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1128
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1129 if propname not in d:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1130 if default is _marker:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1131 if isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1132 return []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1133 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1134 return None
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1135 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1136 return default
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1137
1014
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1138 # return a dupe of the list so code doesn't get confused
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1139 if isinstance(prop, hyperdb.Multilink):
1014
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1140 return d[propname][:]
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1141
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1142 return d[propname]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1143
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1144 def set(self, nodeid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1145 """Modify a property on an existing node of this class.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1146
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1147 'nodeid' must be the id of an existing node of this class or an
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1148 IndexError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1149
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1150 Each key in 'propvalues' must be the name of a property of this
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1151 class or a KeyError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1152
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1153 All values in 'propvalues' must be acceptable types for their
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1154 corresponding properties or a TypeError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1155
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1156 If the value of the key property is set, it must not collide with
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1157 other key strings or a ValueError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1158
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1159 If the value of a Link or Multilink property contains an invalid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1160 node id, a ValueError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1161
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1162 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1163 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1164 """
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
1165 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1166 raise hyperdb.DatabaseError(_('Database open read-only'))
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
1167
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1168 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1169 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1170 for name, prop in self.getprops(protected=0).items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1171 if name in oldvalues:
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1172 continue
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1173 if isinstance(prop, hyperdb.Multilink):
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1174 oldvalues[name] = []
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1175 else:
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1176 oldvalues[name] = None
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1177 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1178 self.fireReactors('set', nodeid, oldvalues)
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1179 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1180
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1181 def set_inner(self, nodeid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1182 """ Called by set, in-between the audit and react calls.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1183 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1184 if not propvalues:
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1185 return propvalues
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1186
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1187 if 'creation' in propvalues or 'activity' in propvalues:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
1188 raise KeyError('"creation" and "activity" are reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1189
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1190 if 'id' in propvalues:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
1191 raise KeyError('"id" is reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1192
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1193 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1194 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1195
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1196 node = self.db.getnode(self.classname, nodeid)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1197 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1198 raise IndexError
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1199 num_re = re.compile('^\d+$')
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1200
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1201 # if the journal value is to be different, store it in here
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1202 journalvalues = {}
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1203
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1204 # omit quiet properties from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1205 quiet_props = []
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1206
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1207 # list() propvalues 'cos it might be modified by the loop
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1208 for propname, value in list(propvalues.items()):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1209 # check to make sure we're not duplicating an existing key
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1210 if propname == self.key and node[propname] != value:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1211 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1212 self.lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1213 except KeyError:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1214 pass
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1215 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1216 raise ValueError('node with key "%s" exists'%value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1217
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1218 # this will raise the KeyError if the property isn't valid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1219 # ... we don't use getprops() here because we only care about
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1220 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1221 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1222 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1223 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1224 raise KeyError('"%s" has no property named "%s"'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1225 self.classname, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1226
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1227 # if the value's the same as the existing value, no sense in
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1228 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1229 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1230 if value == current:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1231 del propvalues[propname]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1232 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1233 journalvalues[propname] = current
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1234
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1235 # do stuff based on the prop type
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1236 if isinstance(prop, hyperdb.Link):
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1237 link_class = prop.classname
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1238 # if it isn't a number, it's a key
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1239 if value is not None and not isinstance(value, type('')):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1240 raise ValueError('property "%s" link value be a string'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1241 propname))
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1242 if isinstance(value, type('')) and not num_re.match(value):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1243 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1244 value = self.db.classes[link_class].lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1245 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1246 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1247 propname, value, prop.classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1248
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1249 if (value is not None and
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1250 not self.db.getclass(link_class).hasnode(value)):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1251 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1252 value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1253
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1254 if self.do_journal and prop.do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1255 # register the unlink with the old linked node
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1256 if propname in node and node[propname] is not None:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1257 self.db.addjournal(link_class, node[propname], 'unlink',
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1258 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1259
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1260 # register the link with the newly linked node
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1261 if value is not None:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1262 self.db.addjournal(link_class, value, 'link',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1263 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1264
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1265 elif isinstance(prop, hyperdb.Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1266 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1267 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3826
diff changeset
1268 if not hasattr(value, '__iter__'):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1269 raise TypeError('new property "%s" not an iterable of'
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1270 ' ids'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1271 link_class = self.properties[propname].classname
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1272 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1273 for entry in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1274 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1275 if type(entry) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1276 raise ValueError('new property "%s" link value '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1277 'must be a string'%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1278 if not num_re.match(entry):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1279 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1280 entry = self.db.classes[link_class].lookup(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1281 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1282 raise IndexError('new property "%s": %s not a %s'%(
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1283 propname, entry,
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1284 self.properties[propname].classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1285 l.append(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1286 value = l
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1287 propvalues[propname] = value
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1288
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1289 # figure the journal entry for this property
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1290 add = []
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1291 remove = []
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1292
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1293 # handle removals
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1294 if propname in node:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1295 l = node[propname]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1296 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1297 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1298 for id in l[:]:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1299 if id in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1300 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1301 # register the unlink with the old linked node
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1302 if self.do_journal and self.properties[propname].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1303 self.db.addjournal(link_class, id, 'unlink',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1304 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1305 l.remove(id)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1306 remove.append(id)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1307
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1308 # handle additions
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1309 for id in value:
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
1310 if not self.db.getclass(link_class).hasnode(id):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1311 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1312 id))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1313 if id in l:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1314 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1315 # register the link with the newly linked node
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1316 if self.do_journal and self.properties[propname].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1317 self.db.addjournal(link_class, id, 'link',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1318 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1319 l.append(id)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1320 add.append(id)
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1321
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1322 # figure the journal entry
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1323 l = []
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1324 if add:
967
dd35bab19dd9 use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents: 964
diff changeset
1325 l.append(('+', add))
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1326 if remove:
967
dd35bab19dd9 use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents: 964
diff changeset
1327 l.append(('-', remove))
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1328 if l:
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1329 journalvalues[propname] = tuple(l)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1330
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1331 elif isinstance(prop, hyperdb.String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1351
diff changeset
1332 if value is not None and type(value) != type('') and type(value) != type(u''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1333 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1334 'string'%propname)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1335 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1336 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1337 value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1338
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1339 elif isinstance(prop, hyperdb.Password):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1340 if not isinstance(value, password.Password):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1341 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1342 'Password'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1343 propvalues[propname] = value
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1344 journalvalues[propname] = \
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1345 current and password.JournalPassword(current)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1346
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1347 elif value is not None and isinstance(prop, hyperdb.Date):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1348 if not isinstance(value, date.Date):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1349 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1350 'Date'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1351 propvalues[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1352
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1353 elif value is not None and isinstance(prop, hyperdb.Interval):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1354 if not isinstance(value, date.Interval):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1355 raise TypeError('new property "%s" not an '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1356 'Interval'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1357 propvalues[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1358
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1359 elif value is not None and isinstance(prop, hyperdb.Number):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1360 try:
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1361 float(value)
887
e7169d6e6e45 added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents: 886
diff changeset
1362 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1363 raise TypeError('new property "%s" not '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1364 'numeric'%propname)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1365
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1366 elif value is not None and isinstance(prop, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1367 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1368 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1369 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1370 raise TypeError('new property "%s" not '
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1371 'numeric'%propname)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1372
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1373 elif value is not None and isinstance(prop, hyperdb.Boolean):
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1374 try:
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1375 int(value)
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1376 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1377 raise TypeError('new property "%s" not '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1378 'boolean'%propname)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1379
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1380 node[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1381
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1382 # record quiet properties to omit from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1383 if prop.quiet:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1384 quiet_props.append(propname)
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1385
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1386 # nothing to do?
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1387 if not propvalues:
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1388 return propvalues
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1389
3083
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1390 # update the activity time
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1391 node['activity'] = date.Date()
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1392 node['actor'] = self.db.getuid()
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1393
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1394 # do the set, and journal it
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1395 self.db.setnode(self.classname, nodeid, node)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1396
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1397 if self.do_journal:
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1398 self.db.addjournal(self.classname, nodeid, 'set', journalvalues)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1399
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1400 # remove quiet properties from output
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1401 for propname in quiet_props:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1402 if propname in propvalues:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1403 del propvalues[propname]
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1404
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1405 return propvalues
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1406
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1407 def retire(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1408 """Retire a node.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1409
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1410 The properties on the node remain available from the get() method,
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1411 and the node's id is never reused.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1412
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1413 Retired nodes are not returned by the find(), list(), or lookup()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1414 methods, and other nodes may reuse the values of their key properties.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1415
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1416 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1417 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1418 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1419 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1420 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1421
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1422 self.fireAuditors('retire', nodeid, None)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1423
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1424 node = self.db.getnode(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1425 node[self.db.RETIRED_FLAG] = 1
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1426 self.db.setnode(self.classname, nodeid, node)
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1427 if self.do_journal:
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1428 self.db.addjournal(self.classname, nodeid, 'retired', None)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1429
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1430 self.fireReactors('retire', nodeid, None)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1431
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1432 def restore(self, nodeid):
5107
33ac07a7ef96 spelling error fixed in comment
John Rouillard <rouilj@ieee.org>
parents: 5100
diff changeset
1433 """Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1434
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1435 Make node available for all operations like it was before retirement.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1436 """
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1437 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1438 raise hyperdb.DatabaseError(_('Database open read-only'))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1439
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1440 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1441 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1442 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1443 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1444 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1445 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1446 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1447 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1448 raise KeyError("Key property (%s) of retired node clashes "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1449 "with existing one (%s)" % (key, node[key]))
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1450 # Now we can safely restore node
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1451 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1452 del node[self.db.RETIRED_FLAG]
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1453 self.db.setnode(self.classname, nodeid, node)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1454 if self.do_journal:
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1455 self.db.addjournal(self.classname, nodeid, 'restored', None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1456
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1457 self.fireReactors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1458
1476
5a01e90b7dc9 fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents: 1467
diff changeset
1459 def is_retired(self, nodeid, cldb=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1460 """Return true if the node is retired.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1461 """
1476
5a01e90b7dc9 fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents: 1467
diff changeset
1462 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1463 if self.db.RETIRED_FLAG in node:
940
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1464 return 1
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1465 return 0
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1466
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1467 def destroy(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1468 """Destroy a node.
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
1469
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1470 WARNING: this method should never be used except in extremely rare
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1471 situations where there could never be links to the node being
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1472 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1473
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1474 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1475
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1476 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1477
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1478 WARNING: really, use retire() instead
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1479
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1480 Well, I think that's enough warnings. This method exists mostly to
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1481 support the session storage of the cgi interface.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1482 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1483 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1484 raise hyperdb.DatabaseError(_('Database open read-only'))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1485 self.db.destroynode(self.classname, nodeid)
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1486
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1487 # Locating nodes:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1488 def hasnode(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1489 """Determine if the given nodeid actually exists
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1490 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1491 return self.db.hasnode(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1492
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1493 def setkey(self, propname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1494 """Select a String property of this class to be the key property.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1495
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1496 'propname' must be the name of a String property of this class or
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1497 None, or a TypeError is raised. The values of the key property on
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1498 all existing nodes must be unique or a ValueError is raised. If the
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1499 property doesn't exist, KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1500 """
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1501 prop = self.getprops()[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1502 if not isinstance(prop, hyperdb.String):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1503 raise TypeError('key properties must be String')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1504 self.key = propname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1505
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1506 def getkey(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1507 """Return the name of the key property for this class or None."""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1508 return self.key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1509
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1510 # TODO: set up a separate index db file for this? profile?
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1511 def lookup(self, keyvalue):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1512 """Locate a particular node by its key property and return its id.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1513
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1514 If this class has no key property, a TypeError is raised. If the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1515 'keyvalue' matches one of the values for the key property among
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1516 the nodes in this class, the matching node's id is returned;
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1517 otherwise a KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1518 """
969
ddcb527491ba Consistent quoting
Richard Jones <richard@users.sourceforge.net>
parents: 967
diff changeset
1519 if not self.key:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1520 raise TypeError('No key property set for '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1521 'class %s'%self.classname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1522 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1523 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1524 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1525 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1526 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1527 continue
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1528 if self.key not in node:
2606
17eb5aeada7f fix bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2601
diff changeset
1529 continue
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1530 if node[self.key] == keyvalue:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1531 return nodeid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1532 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1533 cldb.close()
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1534 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1535 keyvalue, self.classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1536
1103
db787cef1385 handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents: 1099
diff changeset
1537 # change from spec - allows multiple props to match
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1538 def find(self, **propspec):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1539 """Get the ids of nodes in this class which link to the given nodes.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1540
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1541 'propspec' consists of keyword args propname=nodeid or
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1542 propname={nodeid:1, }
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1543 'propname' must be the name of a property in this class, or a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1544 KeyError is raised. That property must be a Link or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1545 Multilink property, or a TypeError is raised.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1546
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1547 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1548 the nodeids will be returned. Examples::
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1549
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1550 db.issue.find(messages='1')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1551 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1552 """
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1553 for propname, itemids in propspec.items():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1554 # check the prop is OK
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1555 prop = self.properties[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1556 if not isinstance(prop, hyperdb.Link) and not isinstance(prop, hyperdb.Multilink):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1557 raise TypeError("'%s' not a Link/Multilink "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1558 "property"%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1559
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1560 # ok, now do the find
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1561 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1562 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1563 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1564 for id in self.getnodeids(db=cldb):
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1565 item = self.db.getnode(self.classname, id, db=cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1566 if self.db.RETIRED_FLAG in item:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1567 continue
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1568 for propname, itemids in propspec.items():
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1569 if type(itemids) is not type({}):
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1570 itemids = {itemids:1}
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1571
2450
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1572 # special case if the item doesn't have this property
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1573 if propname not in item:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1574 if None in itemids:
2450
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1575 l.append(id)
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1576 break
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1577 continue
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1578
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1579 # grab the property definition and its value on this item
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1580 prop = self.properties[propname]
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1581 value = item[propname]
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1582 if isinstance(prop, hyperdb.Link) and value in itemids:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1583 l.append(id)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1584 break
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1585 elif isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1586 hit = 0
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1587 for v in value:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1588 if v in itemids:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1589 l.append(id)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1590 hit = 1
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1591 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1592 if hit:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1593 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1594 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1595 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1596 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1597
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1598 def stringFind(self, **requirements):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1599 """Locate a particular node by matching a set of its String
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1600 properties in a caseless search.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1601
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1602 If the property is not a String property, a TypeError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1603
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1604 The return is a list of the id of all nodes that match.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1605 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1606 for propname in requirements:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1607 prop = self.properties[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1608 if not isinstance(prop, hyperdb.String):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1609 raise TypeError("'%s' not a String property"%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1610 requirements[propname] = requirements[propname].lower()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1611 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1612 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1613 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1614 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1615 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1616 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1617 continue
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1618 for key, value in requirements.items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1619 if key not in node:
1296
34b1e6490e65 hardening for stringFind with missing props
Richard Jones <richard@users.sourceforge.net>
parents: 1252
diff changeset
1620 break
905
502a5ae11cc5 Very close now. The cgi and mailgw now use the new security API.
Richard Jones <richard@users.sourceforge.net>
parents: 902
diff changeset
1621 if node[key] is None or node[key].lower() != value:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1622 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1623 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1624 l.append(nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1625 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1626 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1627 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1628
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1629 def list(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1630 """ Return a list of the ids of the active nodes in this class.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1631 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1632 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1633 cn = self.classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1634 cldb = self.db.getclassdb(cn)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1635 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1636 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1637 node = self.db.getnode(cn, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1638 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1639 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1640 l.append(nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1641 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1642 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1643 l.sort()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1644 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1645
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1646 def getnodeids(self, db=None, retired=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1647 """ Return a list of ALL nodeids
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1648
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1649 Set retired=None to get all nodes. Otherwise it'll get all the
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1650 retired or non-retired nodes, depending on the flag.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1651 """
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1652 res = []
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1653
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1654 # start off with the new nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1655 if self.classname in self.db.newnodes:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1656 res.extend(self.db.newnodes[self.classname])
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1657
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1658 must_close = False
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1659 if db is None:
1486
f5f60c75a458 added test for error in sqlite backend, and fixed *dbm backend error
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1660 db = self.db.getclassdb(self.classname)
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1661 must_close = True
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1662 try:
5444
167f0d25ea8e Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5395
diff changeset
1663 res.extend(map(b2s, db.keys()))
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1664
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1665 # remove the uncommitted, destroyed nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1666 if self.classname in self.db.destroyednodes:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1667 for nodeid in self.db.destroyednodes[self.classname]:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
1668 if nodeid in db:
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1669 res.remove(nodeid)
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1670
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1671 # check retired flag
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1672 if retired is False or retired is True:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1673 l = []
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1674 for nodeid in res:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1675 node = self.db.getnode(self.classname, nodeid, db)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1676 is_ret = self.db.RETIRED_FLAG in node
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1677 if retired == is_ret:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1678 l.append(nodeid)
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1679 res = l
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1680 finally:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1681 if must_close:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1682 db.close()
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1683 return res
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1684
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1685 def _filter(self, search_matches, filterspec, proptree,
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1686 num_re = re.compile('^\d+$'), retired=False):
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1687 """Return a list of the ids of the nodes in this class that
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1688 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1689 sort spec.
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1690
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1691 "filterspec" is {propname: value(s)}
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1692
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1693 "sort" and "group" are (dir, prop) where dir is '+', '-' or None
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1694 and prop is a prop name or None
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1695
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 3979
diff changeset
1696 "search_matches" is a sequence type or None
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1159
diff changeset
1697
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1698 "retired" specifies if we should find only non-retired nodes
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1699 (default) or only retired node (value True) or all nodes.
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1700
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1701 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1702 value to match is a list:
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1703
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1704 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1705 2. Other properties must match any of the elements in the list.
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1706 """
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1707 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1708 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1709
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1710 cn = self.classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1711
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1712 # optimise filterspec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1713 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1714 props = self.getprops()
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1715 LINK = 'spec:link'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1716 MULTILINK = 'spec:multilink'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1717 STRING = 'spec:string'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1718 DATE = 'spec:date'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1719 INTERVAL = 'spec:interval'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1720 OTHER = 'spec:other'
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1721
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
1722 for k, v in filterspec.items():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1723 propclass = props[k]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1724 if isinstance(propclass, hyperdb.Link):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1725 if type(v) is not type([]):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1726 v = [v]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1727 u = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1728 for entry in v:
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1729 # the value -1 is a special "not set" sentinel
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1730 if entry == '-1':
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1731 entry = None
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1732 u.append(entry)
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1733 l.append((LINK, k, u))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1734 elif isinstance(propclass, hyperdb.Multilink):
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1735 # the value -1 is a special "not set" sentinel
1558
Richard Jones <richard@users.sourceforge.net>
parents: 1556
diff changeset
1736 if v in ('-1', ['-1']):
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1737 v = []
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1738 elif type(v) is not type([]):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1739 v = [v]
1955
b00ad075bb2f more unit tests, fixes and cleanups
Richard Jones <richard@users.sourceforge.net>
parents: 1951
diff changeset
1740 l.append((MULTILINK, k, v))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1741 elif isinstance(propclass, hyperdb.String) and k != 'id':
1631
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1742 if type(v) is not type([]):
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1743 v = [v]
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1744 for v in v:
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1745 # simple glob searching
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1746 v = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', v)
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1747 v = v.replace('?', '.')
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1748 v = v.replace('*', '.*?')
3147
dbf80d7db63f consistent text searching behaviour (AND everywhere) [SF#1101036]
Richard Jones <richard@users.sourceforge.net>
parents: 3083
diff changeset
1749 l.append((STRING, k, re.compile(v, re.I)))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1750 elif isinstance(propclass, hyperdb.Date):
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1751 try:
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
1752 date_rng = propclass.range_from_raw(v, self.db)
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1753 l.append((DATE, k, date_rng))
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1754 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1755 # If range creation fails - ignore that search parameter
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1756 pass
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1757 elif isinstance(propclass, hyperdb.Interval):
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1758 try:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1759 intv_rng = date.Range(v, date.Interval)
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1760 l.append((INTERVAL, k, intv_rng))
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1761 except ValueError:
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1762 # If range creation fails - ignore that search parameter
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1763 pass
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1764
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1765 elif isinstance(propclass, hyperdb.Boolean):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1766 if type(v) == type(""):
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1767 v = v.split(',')
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1768 if type(v) != type([]):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1769 v = [v]
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1770 bv = []
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1771 for val in v:
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1772 if type(val) is type(''):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1773 bv.append(propclass.from_raw (val))
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1774 else:
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1775 bv.append(val)
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1776 l.append((OTHER, k, bv))
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1777
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1778 elif k == 'id':
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1779 if type(v) != type([]):
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1780 v = v.split(',')
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1781 l.append((OTHER, k, [str(int(val)) for val in v]))
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1782
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1783 elif isinstance(propclass, hyperdb.Number):
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1784 if type(v) != type([]):
4364
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1785 try :
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1786 v = v.split(',')
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1787 except AttributeError :
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1788 v = [v]
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1789 l.append((OTHER, k, [float(val) for val in v]))
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1790
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1791 elif isinstance(propclass, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1792 if type(v) != type([]):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1793 try :
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1794 v = v.split(',')
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1795 except AttributeError :
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1796 v = [v]
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1797 l.append((OTHER, k, [int(val) for val in v]))
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1798
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1799 filterspec = l
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1800
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1801 # now, find all the nodes that pass filtering
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1802 matches = []
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1803 cldb = self.db.getclassdb(cn)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1804 t = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1805 try:
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1806 # TODO: only full-scan once (use items())
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1807 for nodeid in self.getnodeids(cldb, retired=retired):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1808 node = self.db.getnode(cn, nodeid, cldb)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1809 # apply filter
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1810 for t, k, v in filterspec:
1303
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1811 # handle the id prop
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1812 if k == 'id':
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1813 if nodeid not in v:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1814 break
1303
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1815 continue
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1816
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1817 # get the node value
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1818 nv = node.get(k, None)
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1819
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1820 match = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1821
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1822 # now apply the property filter
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1823 if t == LINK:
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1824 # link - if this node's property doesn't appear in the
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1825 # filterspec's nodeid list, skip it
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1826 match = nv in v
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1827 elif t == MULTILINK:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1828 # multilink - if any of the nodeids required by the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1829 # filterspec aren't in this node's property, then skip
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1830 # it
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1831 nv = node.get(k, [])
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1832
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1833 # check for matching the absence of multilink values
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1834 if not v:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1835 match = not nv
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1836 else:
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1837 # otherwise, make sure this node has each of the
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1838 # required values
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1839 expr = Expression(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1840 if expr.evaluate(nv): match = 1
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1841 elif t == STRING:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1842 if nv is None:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1843 nv = ''
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1844 # RE search
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1845 match = v.search(nv)
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1846 elif t == DATE or t == INTERVAL:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1847 if nv is None:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1848 match = v is None
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1849 else:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1850 if v.to_value:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1851 if v.from_value <= nv and v.to_value >= nv:
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1852 match = 1
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1853 else:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1854 if v.from_value <= nv:
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1855 match = 1
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1856 elif t == OTHER:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1857 # straight value comparison for the other types
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1858 match = nv in v
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1859 if not match:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1860 break
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1861 else:
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1862 matches.append([nodeid, node])
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1863
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1864 # filter based on full text search
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1865 if search_matches is not None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1866 k = []
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1867 for v in matches:
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 3979
diff changeset
1868 if v[0] in search_matches:
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1869 k.append(v)
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1870 matches = k
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1871
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1872 # add sorting information to the proptree
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1873 JPROPS = {'actor':1, 'activity':1, 'creator':1, 'creation':1}
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1874 children = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1875 if proptree:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1876 children = proptree.sortable_children()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1877 for pt in children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1878 dir = pt.sort_direction
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1879 prop = pt.name
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1880 assert (dir and prop)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1881 propclass = props[prop]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1882 pt.sort_ids = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1883 is_pointer = isinstance(propclass,(hyperdb.Link,
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1884 hyperdb.Multilink))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1885 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1886 pt.sort_result = []
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1887 try:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1888 # cache the opened link class db, if needed.
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1889 lcldb = None
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1890 # cache the linked class items too
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1891 lcache = {}
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1892
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1893 for entry in matches:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1894 itemid = entry[-2]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1895 item = entry[-1]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1896 # handle the properties that might be "faked"
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1897 # also, handle possible missing properties
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1898 try:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1899 v = item[prop]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1900 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1901 if prop in JPROPS:
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1902 # force lookup of the special journal prop
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1903 v = self.get(itemid, prop)
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1904 else:
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1905 # the node doesn't have a value for this
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1906 # property
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1907 v = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1908 if isinstance(propclass, hyperdb.Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1909 v = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1910 if prop == 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1911 v = int (itemid)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1912 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1913 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1914 pt.sort_result.append(v)
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1915 continue
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1916
2285
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1917 # missing (None) values are always sorted first
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1918 if v is None:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1919 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1920 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1921 pt.sort_result.append(v)
2285
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1922 continue
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1923
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1924 if isinstance(propclass, hyperdb.Link):
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1925 lcn = propclass.classname
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1926 link = self.db.classes[lcn]
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1927 key = link.orderprop()
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1928 child = pt.propdict[key]
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1929 if key!='id':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1930 if v not in lcache:
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1931 # open the link class db if it's not already
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1932 if lcldb is None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1933 lcldb = self.db.getclassdb(lcn)
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1934 lcache[v] = self.db.getnode(lcn, v, lcldb)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1935 r = lcache[v][key]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1936 child.propdict[key].sort_ids.append(r)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1937 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1938 child.propdict[key].sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1939 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1940 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1941 r = propclass.sort_repr(pt.parent.cls, v, pt.name)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1942 pt.sort_result.append(r)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1943 finally:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1944 # if we opened the link class db, close it now
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1945 if lcldb is not None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1946 lcldb.close()
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1947 del lcache
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1948 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1949 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1950
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1951 # pull the id out of the individual entries
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1952 matches = [entry[-2] for entry in matches]
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1953 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1954 self.db.stats['filtering'] += (time.time() - start_t)
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1955 return matches
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1956
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1957 def count(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1958 """Get the number of nodes in this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1959
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1960 If the returned integer is 'numnodes', the ids of all the nodes
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1961 in this class run from 1 to numnodes, and numnodes+1 will be the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1962 id of the next node to be created in this class.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1963 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1964 return self.db.countnodes(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1965
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1966 # Manipulating properties:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1967
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1968 def getprops(self, protected=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1969 """Return a dictionary mapping property names to property objects.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1970 If the "protected" flag is true, we include protected properties -
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1971 those which may not be modified.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1972
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1973 In addition to the actual properties on the node, these
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1974 methods provide the "creation" and "activity" properties. If the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1975 "protected" flag is true, we include protected properties - those
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1976 which may not be modified.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1977 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1978 d = self.properties.copy()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1979 if protected:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1980 d['id'] = hyperdb.String()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1981 d['creation'] = hyperdb.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1982 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1983 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1984 d['actor'] = hyperdb.Link('user')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1985 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1986
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1987 def addprop(self, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1988 """Add properties to this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1989
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1990 The keyword arguments in 'properties' must map names to property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1991 objects, or a TypeError is raised. None of the keys in 'properties'
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1992 may collide with the names of existing properties, or a ValueError
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1993 is raised before any properties have been added.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1994 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1995 for key in properties:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1996 if key in self.properties:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1997 raise ValueError(key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1998 self.properties.update(properties)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1999
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2000 def index(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2001 """ Add (or refresh) the node to search indexes """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2002 # find all the String properties that have indexme
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
2003 for prop, propclass in self.getprops().items():
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2004 if isinstance(propclass, hyperdb.String) and propclass.indexme:
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2005 # index them under (classname, nodeid, property)
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2006 try:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2007 value = str(self.get(nodeid, prop))
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2008 except IndexError:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2009 # node has been destroyed
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2010 continue
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2011 self.db.indexer.add_text((self.classname, nodeid, prop), value)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2012
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2013 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2014 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2015 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2016 def export_list(self, propnames, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2017 """ Export a node - generate a list of CSV-able data in the order
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2018 specified by propnames for the given node.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2019 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2020 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2021 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2022 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2023 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2024 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2025 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2026 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2027 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2028 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2029 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2030 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2031 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2032 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2033 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2034 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2035
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2036 # append retired flag
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2037 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2038
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2039 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2040
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2041 def import_list(self, propnames, proplist):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2042 """ Import a node - all information including "id" is present and
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2043 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2044 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2045 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2046
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2047 Return the nodeid of the node imported.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2048 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2049 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2050 raise hyperdb.DatabaseError(_('Database open read-only'))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2051 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2052
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2053 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2054 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2055 newid = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2056 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2057 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2058 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2059
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2060 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2061 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2062
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2063 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2064 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2065 newid = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2066 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2067 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2068 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2069 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2070 d[self.db.RETIRED_FLAG] = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2071 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2072 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2073 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2074 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2075
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2076 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2077 if isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2078 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2079 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2080 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2081 elif isinstance(prop, hyperdb.Password):
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2082 value = password.Password(encrypted=value)
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2083 d[propname] = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2084
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2085 # get a new id if necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2086 if newid is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2087 newid = self.db.newid(self.classname)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2088
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2089 # add the node and journal
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2090 self.db.addnode(self.classname, newid, d)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2091 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2092
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2093 def export_journals(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2094 """Export a class's journal - generate a list of lists of
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2095 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2096
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2097 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2098
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2099 No heading here - the columns are fixed.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2100 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2101 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2102 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2103 for nodeid in self.getnodeids():
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
2104 for nodeid, date, user, action, params in self.history(nodeid,
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
2105 enforceperm=False, skipquiet=False):
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2106 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2107 if action == 'set':
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2108 export_data = {}
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
2109 for propname, value in params.items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2110 if propname not in properties:
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2111 # property no longer in the schema
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2112 continue
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2113
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2114 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2115 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2116 if value is None:
3826
bf2e9535da00 Journal and database testing.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3687
diff changeset
2117 pass
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2118 elif isinstance(prop, hyperdb.Date):
3166
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2119 # this is a hack - some dates are stored as strings
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2120 if not isinstance(value, type('')):
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2121 value = value.get_tuple()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2122 elif isinstance(prop, hyperdb.Interval):
3390
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2123 # hack too - some intervals are stored as strings
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2124 if not isinstance(value, type('')):
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2125 value = value.get_tuple()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2126 elif isinstance(prop, hyperdb.Password):
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2127 value = str(value)
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2128 export_data[propname] = value
3052
230f7394cce9 fix to fix
Richard Jones <richard@users.sourceforge.net>
parents: 3003
diff changeset
2129 params = export_data
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2130 r.append([repr(nodeid), repr(date), repr(user),
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2131 repr(action), repr(params)])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2132 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2133
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
2134 class FileClass(hyperdb.FileClass, Class):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2135 """This class defines a large chunk of data. To support this, it has a
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2136 mandatory String property "content" which is typically saved off
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2137 externally to the hyperdb.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2138
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2139 The default MIME type of this data is defined by the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2140 "default_mime_type" class attribute, which may be overridden by each
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2141 node if the class defines a "type" String property.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2142 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2143 def __init__(self, db, classname, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2144 """The newly-created class automatically includes the "content"
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2145 and "type" properties.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2146 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2147 if 'content' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2148 properties['content'] = hyperdb.String(indexme='yes')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2149 if 'type' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2150 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2151 Class.__init__(self, db, classname, **properties)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2152
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2153 def create(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2154 """ Snarf the "content" propvalue and store in a file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2155 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2156 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2157 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2158 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2159
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2160 # now remove the content property so it's not stored in the db
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2161 content = propvalues['content']
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2162 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2163
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2164 # make sure we have a MIME type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2165 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2166
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2167 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2168 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2169
3979
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2170 # store off the content as a file
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2171 self.db.storefile(self.classname, newid, None, content)
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2172
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2173 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2174 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2175
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2176 return newid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2177
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2178 def get(self, nodeid, propname, default=_marker, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2179 """ Trap the content propname and get it from the file
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2180
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2181 'cache' exists for backwards compatibility, and is not used.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2182 """
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
2183 poss_msg = 'Possibly an access right configuration problem.'
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2184 if propname == 'content':
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2185 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2186 return self.db.getfile(self.classname, nodeid, None)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
2187 except IOError as strerror:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2188 # XXX by catching this we don't see an error in the log.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2189 return 'ERROR reading file: %s%s\n%s\n%s'%(
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2190 self.classname, nodeid, poss_msg, strerror)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2191 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2192 return Class.get(self, nodeid, propname, default)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2193 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2194 return Class.get(self, nodeid, propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2195
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2196 def set(self, itemid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2197 """ Snarf the "content" propvalue and update it in a file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2198 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2199 self.fireAuditors('set', itemid, propvalues)
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2200
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2201 # create the oldvalues dict - fill in any missing values
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2202 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
2203 for name, prop in self.getprops(protected=0).items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2204 if name in oldvalues:
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2205 continue
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2206 if isinstance(prop, hyperdb.Multilink):
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2207 oldvalues[name] = []
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2208 else:
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2209 oldvalues[name] = None
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2210
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2211 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2212 content = None
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2213 if 'content' in propvalues:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2214 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2215 del propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2216
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2217 # do the database update
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2218 propvalues = self.set_inner(itemid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2219
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2220 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2221 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2222 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2223 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2224 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2225 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2226 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2227 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2228 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2229
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2230 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2231 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2232 return propvalues
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2233
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2234 def index(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2235 """ Add (or refresh) the node to search indexes.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2236
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2237 Use the content-type property for the content property.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2238 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2239 # find all the String properties that have indexme
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
2240 for prop, propclass in self.getprops().items():
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2241 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2242 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2243 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2244 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2245 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2246 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2247 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2248 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2249 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2250 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2251 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2252 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2253
1103
db787cef1385 handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents: 1099
diff changeset
2254 # deviation from spec - was called ItemClass
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2255 class IssueClass(Class, roundupdb.IssueClass):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2256 # Overridden methods:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2257 def __init__(self, db, classname, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2258 """The newly-created class automatically includes the "messages",
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2259 "files", "nosy", and "superseder" properties. If the 'properties'
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2260 dictionary attempts to specify any of these properties or a
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2261 "creation" or "activity" property, a ValueError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2262 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2263 if 'title' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2264 properties['title'] = hyperdb.String(indexme='yes')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2265 if 'messages' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2266 properties['messages'] = hyperdb.Multilink("msg")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2267 if 'files' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2268 properties['files'] = hyperdb.Multilink("file")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2269 if 'nosy' not in properties:
1002
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2270 # note: journalling is turned off as it really just wastes
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2271 # space. this behaviour may be overridden in an instance
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2272 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2273 if 'superseder' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2274 properties['superseder'] = hyperdb.Multilink(classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2275 Class.__init__(self, db, classname, **properties)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2276
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
2277 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/