Mercurial > p > roundup > code

annotate roundup/backends/back_anydbm.py @ 5380:64c4e43fbb84

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Python 3 preparation: numeric literal syntax. Fixes octal constants to use leading 0o, and removes 'L' suffixes. Tool-assisted patch.
author Joseph Myers <jsm@polyomino.org.uk>
date Tue, 24 Jul 2018 21:43:32 +0000
parents 35ea9b1efc14
children 23b8e6067f7c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
1 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
4 # under the same terms as Python, so long as this copyright message and
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
5 # disclaimer are retained in their original form.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
6 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
11 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
17 #
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
18 """This module defines a backend that saves the hyperdatabase in a
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
19 database chosen by anydbm. It is guaranteed to always be available in python
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
20 versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
21 serious bugs, and is not available)
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
22 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
23 __docformat__ = 'restructuredtext'
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
25 import os, marshal, re, weakref, string, copy, time, shutil, logging
1809
bd127cafe3a8 Simplify backend importing, by moving the imports into the backend modules.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1800
diff changeset
26
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
27 from roundup.anypy.dbm_ import anydbm, whichdb
2908
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
28
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
29 from roundup import hyperdb, date, password, roundupdb, security, support
2908
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
30 from roundup.backends import locking
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
31 from roundup.i18n import _
95813789cf70 translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2906
diff changeset
32
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
33 from roundup.backends.blobfiles import FileStorage
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
34 from roundup.backends.sessions_dbm import Sessions, OneTimeKeys
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3239
diff changeset
35
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
36 from roundup.backends.indexer_common import get_indexer
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
37
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
38 def db_exists(config):
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
39 # check for the user db
2637
11811b313459 The demo.py script works again using the new configuration system.
Richard Jones <richard@users.sourceforge.net>
parents: 2633
diff changeset
40 for db in 'nodes.user nodes.user.db'.split():
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
41 if os.path.exists(os.path.join(config.DATABASE, db)):
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
42 return 1
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
43 return 0
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
44
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
45 def db_nuke(config):
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
46 shutil.rmtree(config.DATABASE)
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2617
diff changeset
47
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
48 class Binary:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
49
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
50 def __init__(self, x, y):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
51 self.x = x
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
52 self.y = y
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
53
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
54 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
55 self.x.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
56 self.y.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
57
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
58 class Unary:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
59
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
60 def __init__(self, x):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
61 self.x = x
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
62
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
63 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
64 return atom(self)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
65
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
66 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
67 self.x.visit(visitor)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
68
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
69 class Equals(Unary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
70
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
71 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
72 return self.x in v
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
73
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
74 def visit(self, visitor):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
75 visitor(self)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
76
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
77 class Not(Unary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
78
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
79 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
80 return not self.x.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
81
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
82 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
83 return "NOT(%s)" % self.x.generate(atom)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
84
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
85 class Or(Binary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
86
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
87 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
88 return self.x.evaluate(v) or self.y.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
89
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
90 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
91 return "(%s)OR(%s)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
92 self.x.generate(atom),
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
93 self.y.generate(atom))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
94
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
95 class And(Binary):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
96
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
97 def evaluate(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
98 return self.x.evaluate(v) and self.y.evaluate(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
99
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
100 def generate(self, atom):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
101 return "(%s)AND(%s)" % (
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
102 self.x.generate(atom),
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
103 self.y.generate(atom))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
104
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
105 def compile_expression(opcodes):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
106
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
107 stack = []
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
108 push, pop = stack.append, stack.pop
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
109 for opcode in opcodes:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
110 if opcode == -2: push(Not(pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
111 elif opcode == -3: push(And(pop(), pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
112 elif opcode == -4: push(Or(pop(), pop()))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
113 else: push(Equals(opcode))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
114
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
115 return pop()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
116
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
117 class Expression:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
118
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
119 def __init__(self, v):
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
120 try:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
121 opcodes = [int(x) for x in v]
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
122 if min(opcodes) >= -1: raise ValueError()
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
123
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
124 compiled = compile_expression(opcodes)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
125 self.evaluate = lambda x: compiled.evaluate([int(y) for y in x])
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
126 except:
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
127 self.evaluate = lambda x: bool(set(x) & set(v))
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
128
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 # Now the database
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 #
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
132 class Database(FileStorage, hyperdb.Database, roundupdb.Database):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
133 """A database for storing records containing flexible data types.
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
134
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
135 Transaction stuff TODO:
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
136
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
137 - check the timestamp of the class file and nuke the cache if it's
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
138 modified. Do some sort of conflict checking on the dirty stuff.
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
139 - perhaps detect write collisions (related to above)?
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
140
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
141 attributes:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
142 dbtype:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
143 holds the value for the type of db. It is used by indexer to
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
144 identify the database type so it can import the correct indexer
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
145 module when using native text search mode.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
146 """
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
147
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
148 dbtype = "anydbm"
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
149
5100
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
150 # used by migrate roundup_admin command. Is a no-op for anydbm.
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
151 # but needed to stop traceback in admin.
5775959d81a3 issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
152 db_version_updated = False
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
153
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
154 def __init__(self, config, journaltag=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
155 """Open a hyperdatabase given a specifier to some storage.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
157 The 'storagelocator' is obtained from config.DATABASE.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 The meaning of 'storagelocator' depends on the particular
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 implementation of the hyperdatabase. It could be a file name,
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 a directory path, a socket descriptor for a connection to a
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
161 database over the network, etc.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
162
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
163 The 'journaltag' is a token that will be attached to the journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164 entries for any edits done on the database. If 'journaltag' is
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 None, the database is opened in read-only mode: the Class.create(),
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
166 Class.set(), Class.retire(), and Class.restore() methods are
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
167 disabled.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
168 """
3897
e2c2d91932ad respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3872
diff changeset
169 FileStorage.__init__(self, config.UMASK)
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
170 self.config, self.journaltag = config, journaltag
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
171 self.dir = config.DATABASE
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
172 self.classes = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
173 self.cache = {} # cache of nodes loaded or created
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
174 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
175 'filtering': 0}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
176 self.dirtynodes = {} # keep track of the dirty nodes by class
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
177 self.newnodes = {} # keep track of the new nodes by class
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
178 self.destroyednodes = {}# keep track of the destroyed nodes by class
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
179 self.transactions = []
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
180 self.indexer = get_indexer(config, self)
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
181 self.security = security.Security(self)
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3601
diff changeset
182 os.umask(config.UMASK)
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
183
5041
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
184 # make sure the database directory exists
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
185 if not os.path.isdir(self.config.DATABASE):
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
186 os.makedirs(self.config.DATABASE)
5251e97b1de0 Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents: 5011
diff changeset
187
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
188 # lock it
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
189 lockfilenm = os.path.join(self.dir, 'lock')
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
190 self.lockfile = locking.acquire_lock(lockfilenm)
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
191 self.lockfile.write(str(os.getpid()))
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
192 self.lockfile.flush()
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
193
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
194 self.Session = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
195 self.Otk = None
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
196
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
197 def post_init(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
198 """Called once the schema initialisation has finished.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
199 """
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
200 # reindex the db if necessary
826
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
201 if self.indexer.should_reindex():
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
202 self.reindex()
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
203
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
204 def refresh_database(self):
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
205 """Rebuild the database
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
206 """
1840
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
207 self.reindex()
91a4619b1a14 hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 1809
diff changeset
208
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
209 def getSessionManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
210 if not self.Session:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
211 self.Session = Sessions(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
212 return self.Session
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
213
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
214 def getOTKManager(self):
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
215 if not self.Otk:
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
216 self.Otk = OneTimeKeys(self)
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
217 return self.Otk
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
218
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
219 def reindex(self, classname=None, show_progress=False):
2650
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
220 if classname:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
221 classes = [self.getclass(classname)]
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
222 else:
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
223 classes = self.classes.values()
d68a444fcce3 roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents: 2649
diff changeset
224 for klass in classes:
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
225 if show_progress:
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
226 for nodeid in support.Progress('Reindex %s'%klass.classname,
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
227 klass.list()):
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
228 klass.index(nodeid)
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
229 else:
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
230 for nodeid in klass.list():
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3586
diff changeset
231 klass.index(nodeid)
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
232 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
233
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
234 def __repr__(self):
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
235 return '<back_anydbm instance at %x>'%id(self)
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
236
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 # Classes
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 def __getattr__(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
241 """A convenient way of calling self.getclass(classname)."""
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
242 if classname in self.classes:
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
243 return self.classes[classname]
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
244 raise AttributeError(classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
245
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246 def addclass(self, cl):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
247 cn = cl.classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
248 if cn in self.classes:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
249 raise ValueError(cn)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
250 self.classes[cn] = cl
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
252 # add default Edit and View permissions
2649
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
253 self.security.addPermission(name="Create", klass=cn,
1df7d4a41da4 Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents: 2640
diff changeset
254 description="User is allowed to create "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
255 self.security.addPermission(name="Edit", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
256 description="User is allowed to edit "+cn)
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
257 self.security.addPermission(name="View", klass=cn,
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
258 description="User is allowed to access "+cn)
4518
e572c5e28d0b Fixed typo in permission "Reture" -> "Retire".
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4517
diff changeset
259 self.security.addPermission(name="Retire", klass=cn,
4517
f8e85cf5f0fe how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents: 4483
diff changeset
260 description="User is allowed to retire "+cn)
2076
2a4309450202 security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
261
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
262 def getclasses(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
263 """Return a list of the names of all existing classes."""
4447
9d37875416c3 python2.3 compatibility fixes
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
264 l = self.classes.keys()
9d37875416c3 python2.3 compatibility fixes
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
265 l.sort()
9d37875416c3 python2.3 compatibility fixes
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4430
diff changeset
266 return l
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
267
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
268 def getclass(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
269 """Get the Class object representing a particular class.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 If 'classname' is not a valid class name, a KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
272 """
1145
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
273 try:
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
274 return self.classes[classname]
81941abedb0a nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents: 1143
diff changeset
275 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
276 raise KeyError('There is no class called "%s"'%classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
277
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
278 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
279 # Class DBs
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
280 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
281 def clear(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
282 """Delete all database contents
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
283 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
284 logging.getLogger('roundup.hyperdb.backend').info('clear')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
285 for cn in self.classes:
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
286 for dummy in 'nodes', 'journals':
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
287 path = os.path.join(self.dir, 'journals.%s'%cn)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
288 if os.path.exists(path):
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
289 os.remove(path)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
290 elif os.path.exists(path+'.db'): # dbm appends .db
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
291 os.remove(path+'.db')
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
292 # reset id sequences
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
293 path = os.path.join(os.getcwd(), self.dir, '_ids')
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
294 if os.path.exists(path):
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
295 os.remove(path)
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
296 elif os.path.exists(path+'.db'): # dbm appends .db
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2699
diff changeset
297 os.remove(path+'.db')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
298
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
299 def getclassdb(self, classname, mode='r'):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
300 """ grab a connection to the class db that will be used for
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
301 multiple actions
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
302 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
303 return self.opendb('nodes.%s'%classname, mode)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
304
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
305 def determine_db_type(self, path):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
306 """ determine which DB wrote the class file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
307 """
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
308 db_type = ''
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
309 if os.path.exists(path):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
310 db_type = whichdb(path)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
311 if not db_type:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
312 raise hyperdb.DatabaseError(_("Couldn't identify database type"))
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
313 elif os.path.exists(path+'.db'):
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
314 # if the path ends in '.db', it's a dbm database, whether
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
315 # anydbm says it's dbhash or not!
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
316 db_type = 'dbm'
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
317 return db_type
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
318
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
319 def opendb(self, name, mode):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
320 """Low-level database opener that gets around anydbm/dbm
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
321 eccentricities.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
322 """
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
323 # figure the class db type
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
324 path = os.path.join(os.getcwd(), self.dir, name)
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
325 db_type = self.determine_db_type(path)
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
326
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
327 # new database? let anydbm pick the best dbm
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
328 # in Python 3+ the "dbm" ("anydbm" to us) module already uses the
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
329 # whichdb() function to do this
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
330 if not db_type or hasattr(anydbm, 'whichdb'):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
331 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
332 logging.getLogger('roundup.hyperdb.backend').debug(
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
333 "opendb anydbm.open(%r, 'c')"%path)
1028
16498e77e3ff allow overiding of the index args roundup/cgi/templating.py
Richard Jones <richard@users.sourceforge.net>
parents: 1022
diff changeset
334 return anydbm.open(path, 'c')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
335
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
336 # in Python <3 it anydbm was a little dumb so manually open the
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
337 # database with the correct module
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
338 try:
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
339 dbm = __import__(db_type)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
340 except ImportError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
341 raise hyperdb.DatabaseError(_("Couldn't open database - the "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
342 "required module '%s' is not available")%db_type)
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
343 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
344 logging.getLogger('roundup.hyperdb.backend').debug(
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
345 "opendb %r.open(%r, %r)"%(db_type, path, mode))
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
346 return dbm.open(path, mode)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
347
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
348 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
349 # Node IDs
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
350 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
351 def newid(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
352 """ Generate a new id for the given class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
353 """
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
354 # open the ids DB - create if if doesn't exist
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
355 db = self.opendb('_ids', 'c')
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
356 if classname in db:
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
357 newid = db[classname] = str(int(db[classname]) + 1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
358 else:
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
359 # the count() bit is transitional - older dbs won't start at 1
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
360 newid = str(self.getclass(classname).count()+1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
361 db[classname] = newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
362 db.close()
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
363 return newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
364
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
365 def setid(self, classname, setid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
366 """ Set the id counter: used during import of database
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
367 """
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
368 # open the ids DB - create if if doesn't exist
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
369 db = self.opendb('_ids', 'c')
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
370 db[classname] = str(setid)
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
371 db.close()
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
372
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
373 #
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
374 # Nodes
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
375 #
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
376 def addnode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
377 """ add the specified node to its class's db
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
378 """
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
379 # we'll be supplied these props if we're doing an import
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
380 if 'creator' not in node:
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
381 # add in the "calculated" properties (dupe so we don't affect
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
382 # calling code's node assumptions)
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
383 node = node.copy()
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
384 node['creator'] = self.getuid()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
385 node['actor'] = self.getuid()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
386 node['creation'] = node['activity'] = date.Date()
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
387
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
388 self.newnodes.setdefault(classname, {})[nodeid] = 1
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
389 self.cache.setdefault(classname, {})[nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
390 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
391
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
392 def setnode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
393 """ change the specified node
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
394 """
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
395 self.dirtynodes.setdefault(classname, {})[nodeid] = 1
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
396
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
397 # can't set without having already loaded the node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
398 self.cache[classname][nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
399 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
400
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
401 def savenode(self, classname, nodeid, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
402 """ perform the saving of data specified by the set/addnode
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
403 """
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
404 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
405 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
406 'save %s%s %r'%(classname, nodeid, node))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
407 self.transactions.append((self.doSaveNode, (classname, nodeid, node)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
408
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
409 def getnode(self, classname, nodeid, db=None, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
410 """ get a node from the database
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
411
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
412 Note the "cache" parameter is not used, and exists purely for
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
413 backward compatibility!
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
414 """
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
415 # try the cache
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
416 cache_dict = self.cache.setdefault(classname, {})
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
417 if nodeid in cache_dict:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
418 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
419 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
420 'get %s%s cached'%(classname, nodeid))
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
421 self.stats['cache_hits'] += 1
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
422 return cache_dict[nodeid]
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
423
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
424 if __debug__:
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
425 self.stats['cache_misses'] += 1
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
426 start_t = time.time()
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
427 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
428 'get %s%s'%(classname, nodeid))
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
429
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
430 # get from the database and save in the cache
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
431 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
432 db = self.getclassdb(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
433 if nodeid not in db:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
434 raise IndexError("no such %s %s"%(classname, nodeid))
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
435
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
436 # check the uncommitted, destroyed nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
437 if (classname in self.destroyednodes and
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
438 nodeid in self.destroyednodes[classname]):
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
439 raise IndexError("no such %s %s"%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
440
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
441 # decode
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
442 res = marshal.loads(db[nodeid])
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
443
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
444 # reverse the serialisation
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
445 res = self.unserialise(classname, res)
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
446
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
447 # store off in the cache dict
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
448 if cache:
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
449 cache_dict[nodeid] = res
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
450
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
451 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
452 self.stats['get_items'] += (time.time() - start_t)
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
453
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
454 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
455
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
456 def destroynode(self, classname, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
457 """Remove a node from the database. Called exclusively by the
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
458 destroy() method on Class.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
459 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
460 logging.getLogger('roundup.hyperdb.backend').info(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
461 'destroy %s%s'%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
462
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
463 # remove from cache and newnodes if it's there
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
464 if (classname in self.cache and nodeid in self.cache[classname]):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
465 del self.cache[classname][nodeid]
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
466 if (classname in self.newnodes and nodeid in self.newnodes[classname]):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
467 del self.newnodes[classname][nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
468
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
469 # see if there's any obvious commit actions that we should get rid of
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
470 for entry in self.transactions[:]:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
471 if entry[1][:2] == (classname, nodeid):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
472 self.transactions.remove(entry)
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
473
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
474 # add to the destroyednodes map
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
475 self.destroyednodes.setdefault(classname, {})[nodeid] = 1
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
476
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
477 # add the destroy commit action
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
478 self.transactions.append((self.doDestroyNode, (classname, nodeid)))
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
479 self.transactions.append((FileStorage.destroy, (self, classname, nodeid)))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
480
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
481 def serialise(self, classname, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
482 """Copy the node contents, converting non-marshallable data into
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
483 marshallable data.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
484 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
485 properties = self.getclass(classname).getprops()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
486 d = {}
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
487 for k, v in node.iteritems():
2964
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
488 if k == self.RETIRED_FLAG:
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
489 d[k] = v
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
490 continue
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
491
b71f6d0a463d merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2908
diff changeset
492 # if the property doesn't exist then we really don't care
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
493 if k not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
494 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
495
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
496 # get the property spec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
497 prop = properties[k]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
498
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
499 if isinstance(prop, hyperdb.Password) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
500 d[k] = str(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
501 elif isinstance(prop, hyperdb.Date) and v is not None:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
502 d[k] = v.serialise()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
503 elif isinstance(prop, hyperdb.Interval) and v is not None:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
504 d[k] = v.serialise()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
505 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
506 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
507 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
508
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
509 def unserialise(self, classname, node):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
510 """Decode the marshalled node data
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
511 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
512 properties = self.getclass(classname).getprops()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
513 d = {}
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
514 for k, v in node.iteritems():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
515 # if the property doesn't exist, or is the "retired" flag then
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
516 # it won't be in the properties dict
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
517 if k not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
518 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
519 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
520
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
521 # get the property spec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
522 prop = properties[k]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
523
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
524 if isinstance(prop, hyperdb.Date) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
525 d[k] = date.Date(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
526 elif isinstance(prop, hyperdb.Interval) and v is not None:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
527 d[k] = date.Interval(v)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
528 elif isinstance(prop, hyperdb.Password) and v is not None:
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
529 d[k] = password.Password(encrypted=v)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
530 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
531 d[k] = v
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
532 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
533
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
534 def hasnode(self, classname, nodeid, db=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
535 """ determine if the database has a given node
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
536 """
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
537 # try the cache
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
538 cache = self.cache.setdefault(classname, {})
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
539 if nodeid in cache:
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
540 return 1
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
541
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
542 # not in the cache - check the database
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
543 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
544 db = self.getclassdb(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
545 return nodeid in db
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
546
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
547 def countnodes(self, classname, db=None):
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
548 count = 0
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
549
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
550 # include the uncommitted nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
551 if classname in self.newnodes:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
552 count += len(self.newnodes[classname])
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
553 if classname in self.destroyednodes:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
554 count -= len(self.destroyednodes[classname])
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
555
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
556 # and count those in the DB
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
557 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
558 db = self.getclassdb(classname)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
559 return count + len(db)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
560
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
561
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
562 #
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
563 # Files - special node properties
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
564 # inherited from FileStorage
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
565
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
566 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
567 # Journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
568 #
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
569 def addjournal(self, classname, nodeid, action, params, creator=None,
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
570 creation=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
571 """ Journal the Action
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
572 'action' may be:
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
573
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
574 'set' -- 'params' is a dictionary of property values
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
575 'create' -- 'params' is an empty dictionary as of
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
576 Wed Nov 06 11:38:43 2002 +0000
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
577 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
578 'retired' or 'restored' -- 'params' is None
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
579
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
580 'creator' -- the user performing the action, which defaults to
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
581 the current user.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
582 """
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
583 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
584 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
585 'addjournal %s%s %s %r %s %r'%(classname,
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
586 nodeid, action, params, creator, creation))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
587 if creator is None:
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
588 creator = self.getuid()
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
589 self.transactions.append((self.doSaveJournal, (classname, nodeid,
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
590 action, params, creator, creation)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
591
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
592 def setjournal(self, classname, nodeid, journal):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
593 """Set the journal to the "journal" list."""
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
594 if __debug__:
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
595 logging.getLogger('roundup.hyperdb.backend').debug(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
596 'setjournal %s%s %r'%(classname, nodeid, journal))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
597 self.transactions.append((self.doSetJournal, (classname, nodeid,
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
598 journal)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
599
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
600 def fix_journal(self, classname, journal):
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
601 """ fix password entries to correct type """
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
602 pwprops = {}
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
603 for pn, prop in self.getclass(classname).properties.iteritems():
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
604 if isinstance(prop, hyperdb.Password):
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
605 pwprops [pn] = 1
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
606 if not pwprops:
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
607 return journal
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
608 for j in journal:
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
609 if j[3] == 'set':
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
610 for k, v in j[4].items():
4538
fd972e18b21a - fix import/export regression test for anydbm for latest journal fix
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4534
diff changeset
611 if k in pwprops and j[4][k]:
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
612 j[4][k] = password.JournalPassword(j[4][k])
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
613 return journal
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
614
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
615 def getjournal(self, classname, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
616 """ get the journal for id
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
617
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
618 Raise IndexError if the node doesn't exist (as per history()'s
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
619 API)
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
620 """
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
621 # our journal result
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
622 res = []
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
623
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
624 # add any journal entries for transactions not committed to the
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
625 # database
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
626 for method, args in self.transactions:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
627 if method != self.doSaveJournal:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
628 continue
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
629 (cache_classname, cache_nodeid, cache_action, cache_params,
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
630 cache_creator, cache_creation) = args
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
631 if cache_classname == classname and cache_nodeid == nodeid:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
632 if not cache_creator:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
633 cache_creator = self.getuid()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
634 if not cache_creation:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
635 cache_creation = date.Date()
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
636 res.append((cache_nodeid, cache_creation, cache_creator,
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
637 cache_action, cache_params))
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
638
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
639 # attempt to open the journal - in some rare cases, the journal may
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
640 # not exist
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
641 try:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
642 db = self.opendb('journals.%s'%classname, 'r')
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
643 except anydbm.error as error:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
644 if str(error) == "need 'c' or 'n' flag to open new db":
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
645 raise IndexError('no such %s %s'%(classname, nodeid))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
646 elif error.args[0] != 2:
1920
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
647 # this isn't a "not found" error, be alarmed!
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
648 raise
1920
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
649 if res:
f9316d2cd5ba Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents: 1904
diff changeset
650 # we have unsaved journal entries, return them
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
651 return self.fix_journal (classname, res)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
652 raise IndexError('no such %s %s'%(classname, nodeid))
787
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
653 try:
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
654 journal = marshal.loads(db[nodeid])
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
655 except KeyError:
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
656 db.close()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
657 if res:
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
658 # we have some unsaved journal entries, be happy!
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
659 return self.fix_journal (classname, res)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
660 raise IndexError('no such %s %s'%(classname, nodeid))
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
661 db.close()
1567
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
662
fc8998ce6274 fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents: 1563
diff changeset
663 # add all the saved journal entries for this node
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
664 for nodeid, date_stamp, user, action, params in journal:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
665 res.append((nodeid, date.Date(date_stamp), user, action, params))
4534
0dd6bdeb2eb5 issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4518
diff changeset
666 return self.fix_journal (classname, res)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
667
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
668 def pack(self, pack_before):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
669 """ Delete all journal entries except "create" before 'pack_before'.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
670 """
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
671 pack_before = pack_before.serialise()
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
672 for classname in self.getclasses():
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
673 packed = 0
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
674 # get the journal db
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
675 db_name = 'journals.%s'%classname
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
676 path = os.path.join(os.getcwd(), self.dir, classname)
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
677 db_type = self.determine_db_type(path)
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
678 db = self.opendb(db_name, 'w')
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
679
4383
0d9369d35483 fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4365
diff changeset
680 for key in db.keys():
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
681 # get the journal for this db entry
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
682 journal = marshal.loads(db[key])
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
683 l = []
567
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
684 last_set_entry = None
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
685 for entry in journal:
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
686 # unpack the entry
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
687 (nodeid, date_stamp, self.journaltag, action,
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
688 params) = entry
990
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
689 # if the entry is after the pack date, _or_ the initial
d374545c8eb0 minor edits
Richard Jones <richard@users.sourceforge.net>
parents: 969
diff changeset
690 # create entry, then it stays
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
691 if date_stamp > pack_before or action == 'create':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
692 l.append(entry)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
693 else:
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
694 packed += 1
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
695 db[key] = marshal.dumps(l)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
696
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
697 logging.getLogger('roundup.hyperdb.backend').info(
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
698 'packed %d %s items'%(packed, classname))
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
699
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
700 if db_type == 'gdbm':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
701 db.reorganize()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
702 db.close()
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
703
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
704
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
705 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
706 # Basic transaction support
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
707 #
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
708 def commit(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
709 """ Commit the current transactions.
3687
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
710
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
711 Save all data changed since the database was opened or since the
ff9f4ca42454 Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents: 3682
diff changeset
712 last commit() or rollback().
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
713 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
714 logging.getLogger('roundup.hyperdb.backend').info('commit %s transactions'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
715 len(self.transactions)))
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
716
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
717 # keep a handle to all the database files opened
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
718 self.databases = {}
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
719
1904
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
720 try:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
721 # now, do all the transactions
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
722 reindex = {}
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
723 for method, args in self.transactions:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
724 reindex[method(*args)] = 1
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
725 finally:
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
726 # make sure we close all the database files
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
727 for db in self.databases.itervalues():
1904
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
728 db.close()
9445caec3ff5 more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents: 1840
diff changeset
729 del self.databases
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
730
3960
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
731 # clear the transactions list now so the blobfile implementation
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
732 # doesn't think there's still pending file commits when it tries
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
733 # to access the file data
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
734 self.transactions = []
bc412bb2ccd3 Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents: 3955
diff changeset
735
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
736 # reindex the nodes that request it
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
737 for classname, nodeid in [k for k in reindex if k]:
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
738 self.getclass(classname).index(nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
739
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
740 # save the indexer state
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
741 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
742
1181
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
743 self.clearCache()
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
744
49aebf5a8691 some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents: 1176
diff changeset
745 def clearCache(self):
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
746 # all transactions committed, back to normal
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
747 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
748 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
749 self.newnodes = {}
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
750 self.destroyednodes = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
751 self.transactions = []
4652
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4538
diff changeset
752 # upcall is necessary!
dfbc0cfa9811 Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents: 4538
diff changeset
753 roundupdb.Database.clearCache(self)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
754
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
755 def getCachedClassDB(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
756 """ get the class db, looking in our cache of databases for commit
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
757 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
758 # get the database handle
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
759 db_name = 'nodes.%s'%classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
760 if db_name not in self.databases:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
761 self.databases[db_name] = self.getclassdb(classname, 'c')
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
762 return self.databases[db_name]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
763
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
764 def doSaveNode(self, classname, nodeid, node):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
765 db = self.getCachedClassDB(classname)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
766
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
767 # now save the marshalled data
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
768 db[nodeid] = marshal.dumps(self.serialise(classname, node))
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
769
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
770 # return the classname, nodeid so we reindex this content
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
771 return (classname, nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
772
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
773 def getCachedJournalDB(self, classname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
774 """ get the journal db, looking in our cache of databases for commit
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
775 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
776 # get the database handle
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
777 db_name = 'journals.%s'%classname
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
778 if db_name not in self.databases:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
779 self.databases[db_name] = self.opendb(db_name, 'c')
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
780 return self.databases[db_name]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
781
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
782 def doSaveJournal(self, classname, nodeid, action, params, creator,
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
783 creation):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
784 # serialise the parameters now if necessary
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
785 if isinstance(params, type({})):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
786 if action in ('set', 'create'):
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
787 params = self.serialise(classname, params)
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
788
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
789 # handle supply of the special journalling parameters (usually
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
790 # supplied on importing an existing database)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
791 journaltag = creator
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
792 if creation:
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
793 journaldate = creation.serialise()
Richard Jones <richard@users.sourceforge.net>
parents: 1131
diff changeset
794 else:
964
832d1209aaa2 Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents: 952
diff changeset
795 journaldate = date.Date().serialise()
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
796
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
797 # create the journal entry
952
f615fbd02c18 full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents: 950
diff changeset
798 entry = (nodeid, journaldate, journaltag, action, params)
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
799
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
800 db = self.getCachedJournalDB(classname)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
801
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
802 # now insert the journal entry
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
803 if nodeid in db:
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
804 # append to existing
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
805 s = db[nodeid]
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
806 l = marshal.loads(s)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
807 l.append(entry)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
808 else:
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
809 l = [entry]
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
810
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
811 db[nodeid] = marshal.dumps(l)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
812
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
813 def doSetJournal(self, classname, nodeid, journal):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
814 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
815 for nodeid, journaldate, journaltag, action, params in journal:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
816 # serialise the parameters now if necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
817 if isinstance(params, type({})):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
818 if action in ('set', 'create'):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
819 params = self.serialise(classname, params)
2512
f5542d92307a fix anydbm journal import [SF#983166]
Richard Jones <richard@users.sourceforge.net>
parents: 2505
diff changeset
820 journaldate = journaldate.serialise()
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
821 l.append((nodeid, journaldate, journaltag, action, params))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
822 db = self.getCachedJournalDB(classname)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
823 db[nodeid] = marshal.dumps(l)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
824
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
825 def doDestroyNode(self, classname, nodeid):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
826 # delete from the class database
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
827 db = self.getCachedClassDB(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
828 if nodeid in db:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
829 del db[nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
830
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
831 # delete from the database
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
832 db = self.getCachedJournalDB(classname)
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
833 if nodeid in db:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
834 del db[nodeid]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
835
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
836 def rollback(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
837 """ Reverse all actions from the current transaction.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
838 """
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
839 logging.getLogger('roundup.hyperdb.backend').info('rollback %s transactions'%(
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
840 len(self.transactions)))
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2512
diff changeset
841
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
842 for method, args in self.transactions:
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
843 # delete temporary files
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
844 if method == self.doStoreFile:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
845 self.rollbackStoreFile(*args)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
846 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
847 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
848 self.newnodes = {}
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
849 self.destroyednodes = {}
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
850 self.transactions = []
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
851
1131
92e92ae58494 add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents: 1127
diff changeset
852 def close(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
853 """ Nothing to do
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
854 """
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
855 if self.lockfile is not None:
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
856 locking.release_lock(self.lockfile)
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
857 self.lockfile.close()
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
858 self.lockfile = None
1131
92e92ae58494 add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents: 1127
diff changeset
859
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
860 _marker = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
861 class Class(hyperdb.Class):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
862 """The handle to a particular class of nodes in a hyperdatabase."""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
863
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
864 def enableJournalling(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
865 """Turn journalling on for this class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
866 """
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
867 self.do_journal = 1
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
868
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
869 def disableJournalling(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
870 """Turn journalling off for this class
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
871 """
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
872 self.do_journal = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
873
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
874 # Editing nodes:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
875
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
876 def create(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
877 """Create a new node of this class and return its id.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
878
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
879 The keyword arguments in 'propvalues' map property names to values.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
880
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
881 The values of arguments must be acceptable for the types of their
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
882 corresponding properties or a TypeError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
883
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
884 If this class has a key property, it must be present and its value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
885 must not collide with other key strings or a ValueError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
886
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
887 Any other properties on this class that are missing from the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
888 'propvalues' dictionary are set to None.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
889
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
890 If an id in a link or multilink property does not refer to a valid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
891 node, an IndexError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
892
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
893 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
894 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
895 """
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
896 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
897 raise hyperdb.DatabaseError(_('Database open read-only'))
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
898 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
899 newid = self.create_inner(**propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
900 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
901 return newid
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
902
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
903 def create_inner(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
904 """ Called by create, in-between the audit and react calls.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
905 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
906 if 'id' in propvalues:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
907 raise KeyError('"id" is reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
908
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
909 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
910 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
911
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
912 if 'creation' in propvalues or 'activity' in propvalues:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
913 raise KeyError('"creation" and "activity" are reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
914 # new node's id
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
915 newid = self.db.newid(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
916
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
917 # validate propvalues
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
918 num_re = re.compile('^\d+$')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
919 for key, value in propvalues.iteritems():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
920 if key == self.key:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
921 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
922 self.lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
923 except KeyError:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
924 pass
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
925 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
926 raise ValueError('node with key "%s" exists'%value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
927
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
928 # try to handle this property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
929 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
930 prop = self.properties[key]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
931 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
932 raise KeyError('"%s" has no property "%s"'%(self.classname,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
933 key))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
934
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
935 if value is not None and isinstance(prop, hyperdb.Link):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
936 if type(value) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
937 raise ValueError('link value must be String')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
938 link_class = self.properties[key].classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
939 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
940 if not num_re.match(value):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
941 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
942 value = self.db.classes[link_class].lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
943 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
944 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
945 key, value, link_class))
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
946 elif not self.db.getclass(link_class).hasnode(value):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
947 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
948 value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
949
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
950 # save off the value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
951 propvalues[key] = value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
952
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
953 # register the link with the newly linked node
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
954 if self.do_journal and self.properties[key].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
955 self.db.addjournal(link_class, value, 'link',
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
956 (self.classname, newid, key))
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
957
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
958 elif isinstance(prop, hyperdb.Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
959 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
960 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3826
diff changeset
961 if not hasattr(value, '__iter__'):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
962 raise TypeError('new property "%s" not an iterable of ids'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
963
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
964 # clean up and validate the list of links
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
965 link_class = self.properties[key].classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
966 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
967 for entry in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
968 if type(entry) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
969 raise ValueError('"%s" multilink value (%r) '\
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
970 'must contain Strings'%(key, value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
971 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
972 if not num_re.match(entry):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
973 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
974 entry = self.db.classes[link_class].lookup(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
975 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
976 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
977 key, entry, self.properties[key].classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
978 l.append(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
979 value = l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
980 propvalues[key] = value
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
981
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
982 # handle additions
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
983 for nodeid in value:
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
984 if not self.db.getclass(link_class).hasnode(nodeid):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
985 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
986 nodeid))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
987 # register the link with the newly linked node
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
988 if self.do_journal and self.properties[key].do_journal:
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
989 self.db.addjournal(link_class, nodeid, 'link',
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
990 (self.classname, newid, key))
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
991
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
992 elif isinstance(prop, hyperdb.String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1351
diff changeset
993 if type(value) != type('') and type(value) != type(u''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
994 raise TypeError('new property "%s" not a string'%key)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
995 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
996 self.db.indexer.add_text((self.classname, newid, key),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
997 value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
998
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
999 elif isinstance(prop, hyperdb.Password):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1000 if not isinstance(value, password.Password):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1001 raise TypeError('new property "%s" not a Password'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1002
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1003 elif isinstance(prop, hyperdb.Date):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1004 if value is not None and not isinstance(value, date.Date):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1005 raise TypeError('new property "%s" not a Date'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1006
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1007 elif isinstance(prop, hyperdb.Interval):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1008 if value is not None and not isinstance(value, date.Interval):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1009 raise TypeError('new property "%s" not an Interval'%key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1010
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1011 elif value is not None and isinstance(prop, hyperdb.Number):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1012 try:
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1013 float(value)
887
e7169d6e6e45 added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents: 886
diff changeset
1014 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1015 raise TypeError('new property "%s" not numeric'%key)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1016
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1017 elif value is not None and isinstance(prop, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1018 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1019 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1020 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1021 raise TypeError('new property "%s" not an integer'%key)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1022
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1023 elif value is not None and isinstance(prop, hyperdb.Boolean):
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1024 try:
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1025 int(value)
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1026 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1027 raise TypeError('new property "%s" not boolean'%key)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1028
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1029 # make sure there's data where there needs to be
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1030 for key, prop in self.properties.iteritems():
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1031 if key in propvalues:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1032 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1033 if key == self.key:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1034 raise ValueError('key property "%s" is required'%key)
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1035 if isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1036 propvalues[key] = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1037
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1038 # done
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1039 self.db.addnode(self.classname, newid, propvalues)
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1040 if self.do_journal:
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1041 self.db.addjournal(self.classname, newid, 'create', {})
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1042
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1043 return newid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1044
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1045 def get(self, nodeid, propname, default=_marker, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1046 """Get the value of a property on an existing node of this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1047
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1048 'nodeid' must be the id of an existing node of this class or an
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1049 IndexError is raised. 'propname' must be the name of a property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1050 of this class or a KeyError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1051
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1052 'cache' exists for backward compatibility, and is not used.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1053
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1054 Attempts to get the "creation" or "activity" properties should
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1055 do the right thing.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1056 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1057 if propname == 'id':
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1058 return nodeid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1059
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1060 # get the node's dict
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
1061 d = self.db.getnode(self.classname, nodeid)
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1062
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1063 # check for one of the special props
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1064 if propname == 'creation':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1065 if 'creation' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1066 return d['creation']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1067 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1068 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1069 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1070 if journal:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents: 4075
diff changeset
1071 return journal[0][1]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1072 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1073 # on the strange chance that there's no journal
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1074 return date.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1075 if propname == 'activity':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1076 if 'activity' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1077 return d['activity']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1078 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1079 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1080 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1081 if journal:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1082 return self.db.getjournal(self.classname, nodeid)[-1][1]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1083 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1084 # on the strange chance that there's no journal
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1085 return date.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1086 if propname == 'creator':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1087 if 'creator' in d:
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1088 return d['creator']
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1089 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1090 raise ValueError('Journalling is disabled for this class')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1091 journal = self.db.getjournal(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1092 if journal:
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1093 num_re = re.compile('^\d+$')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1094 value = journal[0][2]
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1095 if num_re.match(value):
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1096 return value
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1097 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1098 # old-style "username" journal tag
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1099 try:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1100 return self.db.user.lookup(value)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1101 except KeyError:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1102 # user's been retired, return admin
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1103 return '1'
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1104 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1105 return self.db.getuid()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1106 if propname == 'actor':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1107 if 'actor' in d:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1108 return d['actor']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1109 if not self.do_journal:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1110 raise ValueError('Journalling is disabled for this class')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1111 journal = self.db.getjournal(self.classname, nodeid)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1112 if journal:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1113 num_re = re.compile('^\d+$')
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1114 value = journal[-1][2]
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1115 if num_re.match(value):
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1116 return value
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1117 else:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1118 # old-style "username" journal tag
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1119 try:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1120 return self.db.user.lookup(value)
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1121 except KeyError:
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1122 # user's been retired, return admin
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1123 return '1'
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1124 else:
1800
a3b1b1dcf639 Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 1794
diff changeset
1125 return self.db.getuid()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1126
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1127 # get the property (raises KeyErorr if invalid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1128 prop = self.properties[propname]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1129
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1130 if propname not in d:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1131 if default is _marker:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1132 if isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1133 return []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1134 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1135 return None
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1136 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1137 return default
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1138
1014
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1139 # return a dupe of the list so code doesn't get confused
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1140 if isinstance(prop, hyperdb.Multilink):
1014
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1141 return d[propname][:]
8816534e6a1a Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents: 1002
diff changeset
1142
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1143 return d[propname]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1144
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1145 def set(self, nodeid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1146 """Modify a property on an existing node of this class.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1147
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1148 'nodeid' must be the id of an existing node of this class or an
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1149 IndexError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1150
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1151 Each key in 'propvalues' must be the name of a property of this
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1152 class or a KeyError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1153
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1154 All values in 'propvalues' must be acceptable types for their
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1155 corresponding properties or a TypeError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1156
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1157 If the value of the key property is set, it must not collide with
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1158 other key strings or a ValueError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1159
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1160 If the value of a Link or Multilink property contains an invalid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1161 node id, a ValueError is raised.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1162
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1163 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1164 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1165 """
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
1166 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1167 raise hyperdb.DatabaseError(_('Database open read-only'))
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4342
diff changeset
1168
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1169 self.fireAuditors('set', nodeid, propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1170 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid))
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1171 for name, prop in self.getprops(protected=0).iteritems():
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1172 if name in oldvalues:
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1173 continue
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1174 if isinstance(prop, hyperdb.Multilink):
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1175 oldvalues[name] = []
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1176 else:
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
1177 oldvalues[name] = None
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1178 propvalues = self.set_inner(nodeid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1179 self.fireReactors('set', nodeid, oldvalues)
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1180 return propvalues
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1181
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1182 def set_inner(self, nodeid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1183 """ Called by set, in-between the audit and react calls.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1184 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1185 if not propvalues:
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1186 return propvalues
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1187
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1188 if 'creation' in propvalues or 'activity' in propvalues:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
1189 raise KeyError('"creation" and "activity" are reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1190
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1191 if 'id' in propvalues:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
1192 raise KeyError('"id" is reserved')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1193
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1194 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1195 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1196
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1197 node = self.db.getnode(self.classname, nodeid)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1198 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1199 raise IndexError
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1200 num_re = re.compile('^\d+$')
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1201
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1202 # if the journal value is to be different, store it in here
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1203 journalvalues = {}
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1204
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1205 # omit quiet properties from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1206 quiet_props = []
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1207
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1208 # list() propvalues 'cos it might be modified by the loop
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1209 for propname, value in list(propvalues.items()):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1210 # check to make sure we're not duplicating an existing key
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1211 if propname == self.key and node[propname] != value:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1212 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1213 self.lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1214 except KeyError:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1215 pass
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1216 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1217 raise ValueError('node with key "%s" exists'%value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1218
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1219 # this will raise the KeyError if the property isn't valid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1220 # ... we don't use getprops() here because we only care about
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1221 # the writeable properties.
1174
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1222 try:
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1223 prop = self.properties[propname]
8e318dfaf479 Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents: 1170
diff changeset
1224 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1225 raise KeyError('"%s" has no property named "%s"'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1226 self.classname, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1227
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1228 # if the value's the same as the existing value, no sense in
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1229 # doing anything
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1230 current = node.get(propname, None)
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1231 if value == current:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1232 del propvalues[propname]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1233 continue
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1234 journalvalues[propname] = current
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1235
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1236 # do stuff based on the prop type
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1237 if isinstance(prop, hyperdb.Link):
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1238 link_class = prop.classname
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1239 # if it isn't a number, it's a key
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1240 if value is not None and not isinstance(value, type('')):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1241 raise ValueError('property "%s" link value be a string'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1242 propname))
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1243 if isinstance(value, type('')) and not num_re.match(value):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1244 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1245 value = self.db.classes[link_class].lookup(value)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1246 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1247 raise IndexError('new property "%s": %s not a %s'%(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1248 propname, value, prop.classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1249
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1250 if (value is not None and
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1251 not self.db.getclass(link_class).hasnode(value)):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1252 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1253 value))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1254
927
51519406b73e web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents: 924
diff changeset
1255 if self.do_journal and prop.do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1256 # register the unlink with the old linked node
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1257 if propname in node and node[propname] is not None:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1258 self.db.addjournal(link_class, node[propname], 'unlink',
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1259 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1260
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1261 # register the link with the newly linked node
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1262 if value is not None:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1263 self.db.addjournal(link_class, value, 'link',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1264 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1265
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1266 elif isinstance(prop, hyperdb.Multilink):
3872
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1267 if value is None:
34128a809e22 Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3869
diff changeset
1268 value = []
3869
16faac822fe5 Allow Multilinks to take any iterable
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3826
diff changeset
1269 if not hasattr(value, '__iter__'):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1270 raise TypeError('new property "%s" not an iterable of'
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1271 ' ids'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1272 link_class = self.properties[propname].classname
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1273 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1274 for entry in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1275 # if it isn't a number, it's a key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1276 if type(entry) != type(''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1277 raise ValueError('new property "%s" link value '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1278 'must be a string'%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1279 if not num_re.match(entry):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1280 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1281 entry = self.db.classes[link_class].lookup(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1282 except (TypeError, KeyError):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1283 raise IndexError('new property "%s": %s not a %s'%(
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1284 propname, entry,
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1285 self.properties[propname].classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1286 l.append(entry)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1287 value = l
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1288 propvalues[propname] = value
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1289
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1290 # figure the journal entry for this property
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1291 add = []
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1292 remove = []
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1293
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1294 # handle removals
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1295 if propname in node:
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1296 l = node[propname]
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1297 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1298 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1299 for id in l[:]:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1300 if id in value:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1301 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1302 # register the unlink with the old linked node
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1303 if self.do_journal and self.properties[propname].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1304 self.db.addjournal(link_class, id, 'unlink',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1305 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1306 l.remove(id)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1307 remove.append(id)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1308
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1309 # handle additions
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1310 for id in value:
902
b0d3d3535998 Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents: 891
diff changeset
1311 if not self.db.getclass(link_class).hasnode(id):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1312 raise IndexError('%s has no node %s'%(link_class,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1313 id))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1314 if id in l:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1315 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1316 # register the link with the newly linked node
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1317 if self.do_journal and self.properties[propname].do_journal:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1318 self.db.addjournal(link_class, id, 'link',
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1319 (self.classname, nodeid, propname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1320 l.append(id)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1321 add.append(id)
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1322
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1323 # figure the journal entry
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1324 l = []
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1325 if add:
967
dd35bab19dd9 use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents: 964
diff changeset
1326 l.append(('+', add))
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1327 if remove:
967
dd35bab19dd9 use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents: 964
diff changeset
1328 l.append(('-', remove))
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1329 if l:
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1330 journalvalues[propname] = tuple(l)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1331
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1332 elif isinstance(prop, hyperdb.String):
1383
f19dde90e473 applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1351
diff changeset
1333 if value is not None and type(value) != type('') and type(value) != type(u''):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1334 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1335 'string'%propname)
2892
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1336 if prop.indexme:
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1337 self.db.indexer.add_text((self.classname, nodeid, propname),
2eae5848912d always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents: 2736
diff changeset
1338 value)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1339
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1340 elif isinstance(prop, hyperdb.Password):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1341 if not isinstance(value, password.Password):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1342 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1343 'Password'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1344 propvalues[propname] = value
4483
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1345 journalvalues[propname] = \
22bc0426e348 Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4480
diff changeset
1346 current and password.JournalPassword(current)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1347
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1348 elif value is not None and isinstance(prop, hyperdb.Date):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1349 if not isinstance(value, date.Date):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1350 raise TypeError('new property "%s" not a '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1351 'Date'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1352 propvalues[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1353
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1354 elif value is not None and isinstance(prop, hyperdb.Interval):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1355 if not isinstance(value, date.Interval):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1356 raise TypeError('new property "%s" not an '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1357 'Interval'%propname)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1358 propvalues[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1359
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1360 elif value is not None and isinstance(prop, hyperdb.Number):
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1361 try:
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1362 float(value)
887
e7169d6e6e45 added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents: 886
diff changeset
1363 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1364 raise TypeError('new property "%s" not '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1365 'numeric'%propname)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1366
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1367 elif value is not None and isinstance(prop, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1368 try:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1369 int(value)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1370 except ValueError:
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1371 raise TypeError('new property "%s" not '
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1372 'numeric'%propname)
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1373
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1374 elif value is not None and isinstance(prop, hyperdb.Boolean):
890
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1375 try:
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1376 int(value)
a568596dbea7 Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 887
diff changeset
1377 except ValueError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1378 raise TypeError('new property "%s" not '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1379 'boolean'%propname)
880
de3da99a7c02 Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 869
diff changeset
1380
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1381 node[propname] = value
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1382
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1383 # record quiet properties to omit from history/changelog
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1384 if prop.quiet:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1385 quiet_props.append(propname)
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1386
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1387 # nothing to do?
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1388 if not propvalues:
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1389 return propvalues
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1390
3083
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1391 # update the activity time
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1392 node['activity'] = date.Date()
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1393 node['actor'] = self.db.getuid()
b9a819ef7554 actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3052
diff changeset
1394
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1395 # do the set, and journal it
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1396 self.db.setnode(self.classname, nodeid, node)
869
6d98bec4e52e fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents: 862
diff changeset
1397
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1398 if self.do_journal:
1304
61ad556cfc8d working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents: 1303
diff changeset
1399 self.db.addjournal(self.classname, nodeid, 'set', journalvalues)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1400
5112
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1401 # remove quiet properties from output
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1402 for propname in quiet_props:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1403 if propname in propvalues:
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1404 del propvalues[propname]
8901cc4ef0e0 - issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents: 5107
diff changeset
1405
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
1406 return propvalues
930
3c344e942055 Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents: 927
diff changeset
1407
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1408 def retire(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1409 """Retire a node.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1410
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1411 The properties on the node remain available from the get() method,
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1412 and the node's id is never reused.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1413
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1414 Retired nodes are not returned by the find(), list(), or lookup()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1415 methods, and other nodes may reuse the values of their key properties.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1416
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1417 These operations trigger detectors and can be vetoed. Attempts
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1418 to modify the "creation" or "activity" properties cause a KeyError.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1419 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1420 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1421 raise hyperdb.DatabaseError(_('Database open read-only'))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1422
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1423 self.fireAuditors('retire', nodeid, None)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1424
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1425 node = self.db.getnode(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1426 node[self.db.RETIRED_FLAG] = 1
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1427 self.db.setnode(self.classname, nodeid, node)
860
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1428 if self.do_journal:
2df32a73eb45 Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
1429 self.db.addjournal(self.classname, nodeid, 'retired', None)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1430
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1431 self.fireReactors('retire', nodeid, None)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1432
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1433 def restore(self, nodeid):
5107
33ac07a7ef96 spelling error fixed in comment
John Rouillard <rouilj@ieee.org>
parents: 5100
diff changeset
1434 """Restore a retired node.
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1435
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1436 Make node available for all operations like it was before retirement.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1437 """
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1438 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1439 raise hyperdb.DatabaseError(_('Database open read-only'))
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1440
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1441 node = self.db.getnode(self.classname, nodeid)
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1442 # check if key property was overrided
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1443 key = self.getkey()
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1444 try:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1445 id = self.lookup(node[key])
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1446 except KeyError:
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1447 pass
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1448 else:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1449 raise KeyError("Key property (%s) of retired node clashes "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1450 "with existing one (%s)" % (key, node[key]))
1523
63aa7be52d2c checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1519
diff changeset
1451 # Now we can safely restore node
1519
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1452 self.fireAuditors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1453 del node[self.db.RETIRED_FLAG]
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1454 self.db.setnode(self.classname, nodeid, node)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1455 if self.do_journal:
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1456 self.db.addjournal(self.classname, nodeid, 'restored', None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1457
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1458 self.fireReactors('restore', nodeid, None)
6fede2aa6a12 added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1505
diff changeset
1459
1476
5a01e90b7dc9 fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents: 1467
diff changeset
1460 def is_retired(self, nodeid, cldb=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1461 """Return true if the node is retired.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1462 """
1476
5a01e90b7dc9 fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents: 1467
diff changeset
1463 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1464 if self.db.RETIRED_FLAG in node:
940
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1465 return 1
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1466 return 0
301a02ea6020 added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents: 930
diff changeset
1467
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1468 def destroy(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1469 """Destroy a node.
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
1470
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1471 WARNING: this method should never be used except in extremely rare
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1472 situations where there could never be links to the node being
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1473 deleted
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1474
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1475 WARNING: use retire() instead
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1476
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1477 WARNING: the properties of this node will not be available ever again
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1478
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1479 WARNING: really, use retire() instead
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1480
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1481 Well, I think that's enough warnings. This method exists mostly to
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1482 support the session storage of the cgi interface.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1483 """
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1484 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1485 raise hyperdb.DatabaseError(_('Database open read-only'))
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1486 self.db.destroynode(self.classname, nodeid)
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
1487
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1488 # Locating nodes:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1489 def hasnode(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1490 """Determine if the given nodeid actually exists
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1491 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1492 return self.db.hasnode(self.classname, nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1493
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1494 def setkey(self, propname):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1495 """Select a String property of this class to be the key property.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1496
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1497 'propname' must be the name of a String property of this class or
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1498 None, or a TypeError is raised. The values of the key property on
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1499 all existing nodes must be unique or a ValueError is raised. If the
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1500 property doesn't exist, KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1501 """
862
37fb48c3a136 Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents: 860
diff changeset
1502 prop = self.getprops()[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1503 if not isinstance(prop, hyperdb.String):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1504 raise TypeError('key properties must be String')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1505 self.key = propname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1506
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1507 def getkey(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1508 """Return the name of the key property for this class or None."""
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1509 return self.key
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1510
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1511 # TODO: set up a separate index db file for this? profile?
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1512 def lookup(self, keyvalue):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1513 """Locate a particular node by its key property and return its id.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1514
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1515 If this class has no key property, a TypeError is raised. If the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1516 'keyvalue' matches one of the values for the key property among
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1517 the nodes in this class, the matching node's id is returned;
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1518 otherwise a KeyError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1519 """
969
ddcb527491ba Consistent quoting
Richard Jones <richard@users.sourceforge.net>
parents: 967
diff changeset
1520 if not self.key:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1521 raise TypeError('No key property set for '
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1522 'class %s'%self.classname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1523 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1524 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1525 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1526 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1527 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1528 continue
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1529 if self.key not in node:
2606
17eb5aeada7f fix bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2601
diff changeset
1530 continue
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1531 if node[self.key] == keyvalue:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1532 return nodeid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1533 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1534 cldb.close()
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1535 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key,
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1536 keyvalue, self.classname))
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1537
1103
db787cef1385 handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents: 1099
diff changeset
1538 # change from spec - allows multiple props to match
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1539 def find(self, **propspec):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1540 """Get the ids of nodes in this class which link to the given nodes.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1541
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1542 'propspec' consists of keyword args propname=nodeid or
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1543 propname={nodeid:1, }
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1544 'propname' must be the name of a property in this class, or a
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1545 KeyError is raised. That property must be a Link or
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1546 Multilink property, or a TypeError is raised.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1547
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1548 Any node in this class whose 'propname' property links to any of
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1549 the nodeids will be returned. Examples::
1222
bc3bc3248dd1 added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
1550
3239
440f0a6a2e3c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3166
diff changeset
1551 db.issue.find(messages='1')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1552 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1553 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1554 for propname, itemids in propspec.iteritems():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1555 # check the prop is OK
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1556 prop = self.properties[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1557 if not isinstance(prop, hyperdb.Link) and not isinstance(prop, hyperdb.Multilink):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1558 raise TypeError("'%s' not a Link/Multilink "
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1559 "property"%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1560
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1561 # ok, now do the find
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1562 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1563 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1564 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1565 for id in self.getnodeids(db=cldb):
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1566 item = self.db.getnode(self.classname, id, db=cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1567 if self.db.RETIRED_FLAG in item:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1568 continue
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1569 for propname, itemids in propspec.iteritems():
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1570 if type(itemids) is not type({}):
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1571 itemids = {itemids:1}
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1572
2450
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1573 # special case if the item doesn't have this property
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1574 if propname not in item:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1575 if None in itemids:
2450
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1576 l.append(id)
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1577 break
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1578 continue
c45ed2413044 fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2437
diff changeset
1579
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1580 # grab the property definition and its value on this item
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1581 prop = self.properties[propname]
1563
e2a8ce4d2317 Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents: 1558
diff changeset
1582 value = item[propname]
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1583 if isinstance(prop, hyperdb.Link) and value in itemids:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1584 l.append(id)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1585 break
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1586 elif isinstance(prop, hyperdb.Multilink):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1587 hit = 0
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1588 for v in value:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1589 if v in itemids:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1590 l.append(id)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1591 hit = 1
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1592 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1593 if hit:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1594 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1595 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1596 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1597 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1598
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1599 def stringFind(self, **requirements):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1600 """Locate a particular node by matching a set of its String
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1601 properties in a caseless search.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1602
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1603 If the property is not a String property, a TypeError is raised.
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1604
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1605 The return is a list of the id of all nodes that match.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1606 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1607 for propname in requirements:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1608 prop = self.properties[propname]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1609 if not isinstance(prop, hyperdb.String):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1610 raise TypeError("'%s' not a String property"%propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1611 requirements[propname] = requirements[propname].lower()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1612 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1613 cldb = self.db.getclassdb(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1614 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1615 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1616 node = self.db.getnode(self.classname, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1617 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1618 continue
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1619 for key, value in requirements.iteritems():
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1620 if key not in node:
1296
34b1e6490e65 hardening for stringFind with missing props
Richard Jones <richard@users.sourceforge.net>
parents: 1252
diff changeset
1621 break
905
502a5ae11cc5 Very close now. The cgi and mailgw now use the new security API.
Richard Jones <richard@users.sourceforge.net>
parents: 902
diff changeset
1622 if node[key] is None or node[key].lower() != value:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1623 break
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1624 else:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1625 l.append(nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1626 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1627 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1628 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1629
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1630 def list(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1631 """ Return a list of the ids of the active nodes in this class.
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1632 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1633 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1634 cn = self.classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1635 cldb = self.db.getclassdb(cn)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1636 try:
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1637 for nodeid in self.getnodeids(cldb):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1638 node = self.db.getnode(cn, nodeid, cldb)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1639 if self.db.RETIRED_FLAG in node:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1640 continue
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1641 l.append(nodeid)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1642 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1643 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1644 l.sort()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1645 return l
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1646
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1647 def getnodeids(self, db=None, retired=None):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1648 """ Return a list of ALL nodeids
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1649
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1650 Set retired=None to get all nodes. Otherwise it'll get all the
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1651 retired or non-retired nodes, depending on the flag.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1652 """
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1653 res = []
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1654
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1655 # start off with the new nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1656 if self.classname in self.db.newnodes:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1657 res.extend(self.db.newnodes[self.classname])
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1658
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1659 must_close = False
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1660 if db is None:
1486
f5f60c75a458 added test for error in sqlite backend, and fixed *dbm backend error
Richard Jones <richard@users.sourceforge.net>
parents: 1484
diff changeset
1661 db = self.db.getclassdb(self.classname)
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1662 must_close = True
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1663 try:
4383
0d9369d35483 fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4365
diff changeset
1664 res.extend(db.keys())
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1665
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1666 # remove the uncommitted, destroyed nodes
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1667 if self.classname in self.db.destroyednodes:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1668 for nodeid in self.db.destroyednodes[self.classname]:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4995
diff changeset
1669 if nodeid in db:
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1670 res.remove(nodeid)
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1671
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1672 # check retired flag
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1673 if retired is False or retired is True:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1674 l = []
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1675 for nodeid in res:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1676 node = self.db.getnode(self.classname, nodeid, db)
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1677 is_ret = self.db.RETIRED_FLAG in node
3486
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1678 if retired == is_ret:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1679 l.append(nodeid)
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1680 res = l
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1681 finally:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1682 if must_close:
34ada15b9936 all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents: 3476
diff changeset
1683 db.close()
1484
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1684 return res
b3f2484babce fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents: 1476
diff changeset
1685
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1686 def _filter(self, search_matches, filterspec, proptree,
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1687 num_re = re.compile('^\d+$'), retired=False):
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1688 """Return a list of the ids of the nodes in this class that
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1689 match the 'filter' spec, sorted by the group spec and then the
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1690 sort spec.
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1691
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1692 "filterspec" is {propname: value(s)}
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1693
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1694 "sort" and "group" are (dir, prop) where dir is '+', '-' or None
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1695 and prop is a prop name or None
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1696
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 3979
diff changeset
1697 "search_matches" is a sequence type or None
1170
af104fa52746 Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents: 1159
diff changeset
1698
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1699 "retired" specifies if we should find only non-retired nodes
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1700 (default) or only retired node (value True) or all nodes.
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1701
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1702 The filter must match all properties specificed. If the property
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1703 value to match is a list:
3906
666b70676ec6 destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3897
diff changeset
1704
3455
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1705 1. String properties must match all elements in the list, and
e01bc6d65fa9 fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents: 3419
diff changeset
1706 2. Other properties must match any of the elements in the list.
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1955
diff changeset
1707 """
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1708 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1709 start_t = time.time()
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1710
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1711 cn = self.classname
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1712
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1713 # optimise filterspec
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1714 l = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1715 props = self.getprops()
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1716 LINK = 'spec:link'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1717 MULTILINK = 'spec:multilink'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1718 STRING = 'spec:string'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1719 DATE = 'spec:date'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1720 INTERVAL = 'spec:interval'
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1721 OTHER = 'spec:other'
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1722
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1723 for k, v in filterspec.iteritems():
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1724 propclass = props[k]
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1725 if isinstance(propclass, hyperdb.Link):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1726 if type(v) is not type([]):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1727 v = [v]
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1728 u = []
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1729 for entry in v:
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1730 # the value -1 is a special "not set" sentinel
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1731 if entry == '-1':
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1732 entry = None
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1733 u.append(entry)
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1734 l.append((LINK, k, u))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1735 elif isinstance(propclass, hyperdb.Multilink):
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1736 # the value -1 is a special "not set" sentinel
1558
Richard Jones <richard@users.sourceforge.net>
parents: 1556
diff changeset
1737 if v in ('-1', ['-1']):
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1738 v = []
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1739 elif type(v) is not type([]):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1740 v = [v]
1955
b00ad075bb2f more unit tests, fixes and cleanups
Richard Jones <richard@users.sourceforge.net>
parents: 1951
diff changeset
1741 l.append((MULTILINK, k, v))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1742 elif isinstance(propclass, hyperdb.String) and k != 'id':
1631
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1743 if type(v) is not type([]):
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1744 v = [v]
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1745 for v in v:
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1746 # simple glob searching
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1747 v = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', v)
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1748 v = v.replace('?', '.')
8a908bbad1ef A couple of form value handling changes:
Richard Jones <richard@users.sourceforge.net>
parents: 1599
diff changeset
1749 v = v.replace('*', '.*?')
3147
dbf80d7db63f consistent text searching behaviour (AND everywhere) [SF#1101036]
Richard Jones <richard@users.sourceforge.net>
parents: 3083
diff changeset
1750 l.append((STRING, k, re.compile(v, re.I)))
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1751 elif isinstance(propclass, hyperdb.Date):
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1752 try:
3586
f47bddab5a49 date spec wasn't allowing week intervals
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
1753 date_rng = propclass.range_from_raw(v, self.db)
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1754 l.append((DATE, k, date_rng))
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1755 except ValueError:
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1756 # If range creation fails - ignore that search parameter
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1757 pass
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1758 elif isinstance(propclass, hyperdb.Interval):
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1759 try:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1760 intv_rng = date.Range(v, date.Interval)
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1761 l.append((INTERVAL, k, intv_rng))
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1762 except ValueError:
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1763 # If range creation fails - ignore that search parameter
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1764 pass
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
1765
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1766 elif isinstance(propclass, hyperdb.Boolean):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1767 if type(v) == type(""):
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1768 v = v.split(',')
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1769 if type(v) != type([]):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1770 v = [v]
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1771 bv = []
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1772 for val in v:
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1773 if type(val) is type(''):
4365
667c818f6a22 - unify bool searching (filter method) across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4364
diff changeset
1774 bv.append(propclass.from_raw (val))
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1775 else:
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1776 bv.append(val)
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1777 l.append((OTHER, k, bv))
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1778
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1779 elif k == 'id':
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1780 if type(v) != type([]):
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1781 v = v.split(',')
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1782 l.append((OTHER, k, [str(int(val)) for val in v]))
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1783
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1784 elif isinstance(propclass, hyperdb.Number):
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1785 if type(v) != type([]):
4364
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1786 try :
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1787 v = v.split(',')
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1788 except AttributeError :
0e81742d0e2f - unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4357
diff changeset
1789 v = [v]
2453
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1790 l.append((OTHER, k, [float(val) for val in v]))
0e2a0c2c8142 allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents: 2450
diff changeset
1791
5067
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1792 elif isinstance(propclass, hyperdb.Integer):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1793 if type(v) != type([]):
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1794 try :
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1795 v = v.split(',')
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1796 except AttributeError :
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1797 v = [v]
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1798 l.append((OTHER, k, [int(val) for val in v]))
e424987d294a Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents: 5041
diff changeset
1799
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1800 filterspec = l
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
1801
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1802 # now, find all the nodes that pass filtering
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1803 matches = []
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1804 cldb = self.db.getclassdb(cn)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1805 t = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1806 try:
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1807 # TODO: only full-scan once (use items())
5318
506c7ee9a385 Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5248
diff changeset
1808 for nodeid in self.getnodeids(cldb, retired=retired):
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1809 node = self.db.getnode(cn, nodeid, cldb)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1810 # apply filter
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1811 for t, k, v in filterspec:
1303
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1812 # handle the id prop
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1813 if k == 'id':
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1814 if nodeid not in v:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1815 break
1303
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1816 continue
71be6588904f fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents: 1296
diff changeset
1817
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1818 # get the node value
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1819 nv = node.get(k, None)
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1820
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1821 match = 0
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1822
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1823 # now apply the property filter
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1824 if t == LINK:
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1825 # link - if this node's property doesn't appear in the
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1826 # filterspec's nodeid list, skip it
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1827 match = nv in v
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1828 elif t == MULTILINK:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1829 # multilink - if any of the nodeids required by the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1830 # filterspec aren't in this node's property, then skip
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1831 # it
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1832 nv = node.get(k, [])
1555
948c7764d46c implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents: 1523
diff changeset
1833
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1834 # check for matching the absence of multilink values
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1835 if not v:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1836 match = not nv
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1837 else:
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1838 # otherwise, make sure this node has each of the
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1839 # required values
4466
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1840 expr = Expression(v)
f1fe6fd0aa61 Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4447
diff changeset
1841 if expr.evaluate(nv): match = 1
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1842 elif t == STRING:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1843 if nv is None:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1844 nv = ''
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1845 # RE search
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1846 match = v.search(nv)
1596
33a0d94c7658 searching on ranges of intervals is implemented
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1567
diff changeset
1847 elif t == DATE or t == INTERVAL:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1848 if nv is None:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1849 match = v is None
1499
8ee69708da0c added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents: 1496
diff changeset
1850 else:
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1851 if v.to_value:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1852 if v.from_value <= nv and v.to_value >= nv:
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1853 match = 1
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1854 else:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1855 if v.from_value <= nv:
2617
33fffbf7ae68 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2608
diff changeset
1856 match = 1
924
aa10112dd7d1 cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 916
diff changeset
1857 elif t == OTHER:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1858 # straight value comparison for the other types
2486
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1859 match = nv in v
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1860 if not match:
d7488e349a03 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 2453
diff changeset
1861 break
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1862 else:
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1863 matches.append([nodeid, node])
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1864
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1865 # filter based on full text search
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1866 if search_matches is not None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1867 k = []
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1868 for v in matches:
4044
27a9906cd8d1 Fix issue2550505.
Stefan Seefeld <stefan@seefeld.name>
parents: 3979
diff changeset
1869 if v[0] in search_matches:
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1870 k.append(v)
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1871 matches = k
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1872
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1873 # add sorting information to the proptree
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1874 JPROPS = {'actor':1, 'activity':1, 'creator':1, 'creation':1}
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1875 children = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1876 if proptree:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1877 children = proptree.sortable_children()
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1878 for pt in children:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1879 dir = pt.sort_direction
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1880 prop = pt.name
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1881 assert (dir and prop)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1882 propclass = props[prop]
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1883 pt.sort_ids = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1884 is_pointer = isinstance(propclass,(hyperdb.Link,
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1885 hyperdb.Multilink))
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1886 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1887 pt.sort_result = []
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1888 try:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1889 # cache the opened link class db, if needed.
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1890 lcldb = None
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1891 # cache the linked class items too
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1892 lcache = {}
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1893
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1894 for entry in matches:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1895 itemid = entry[-2]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1896 item = entry[-1]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1897 # handle the properties that might be "faked"
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1898 # also, handle possible missing properties
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1899 try:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1900 v = item[prop]
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1901 except KeyError:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1902 if prop in JPROPS:
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1903 # force lookup of the special journal prop
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1904 v = self.get(itemid, prop)
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1905 else:
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1906 # the node doesn't have a value for this
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1907 # property
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1908 v = None
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1909 if isinstance(propclass, hyperdb.Multilink):
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1910 v = []
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1911 if prop == 'id':
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1912 v = int (itemid)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1913 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1914 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1915 pt.sort_result.append(v)
2437
f37f3617b9e9 force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents: 2418
diff changeset
1916 continue
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1917
2285
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1918 # missing (None) values are always sorted first
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1919 if v is None:
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1920 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1921 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1922 pt.sort_result.append(v)
2285
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1923 continue
c7f780c24a87 fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents: 2252
diff changeset
1924
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1925 if isinstance(propclass, hyperdb.Link):
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1926 lcn = propclass.classname
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1927 link = self.db.classes[lcn]
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1928 key = link.orderprop()
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1929 child = pt.propdict[key]
3476
1142dafe0d7f added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents: 3455
diff changeset
1930 if key!='id':
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1931 if v not in lcache:
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1932 # open the link class db if it's not already
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1933 if lcldb is None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1934 lcldb = self.db.getclassdb(lcn)
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1935 lcache[v] = self.db.getnode(lcn, v, lcldb)
3682
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1936 r = lcache[v][key]
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1937 child.propdict[key].sort_ids.append(r)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1938 else:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1939 child.propdict[key].sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1940 pt.sort_ids.append(v)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1941 if not is_pointer:
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1942 r = propclass.sort_repr(pt.parent.cls, v, pt.name)
193f316dbbe9 More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
1943 pt.sort_result.append(r)
2252
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1944 finally:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1945 # if we opened the link class db, close it now
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1946 if lcldb is not None:
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1947 lcldb.close()
b63e455462ee some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents: 2249
diff changeset
1948 del lcache
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1949 finally:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1950 cldb.close()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1951
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1952 # pull the id out of the individual entries
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1953 matches = [entry[-2] for entry in matches]
2237
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1954 if __debug__:
f624fc20f8fe added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
1955 self.db.stats['filtering'] += (time.time() - start_t)
2239
c8a06e10e2c6 much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents: 2237
diff changeset
1956 return matches
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1957
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1958 def count(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1959 """Get the number of nodes in this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1960
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1961 If the returned integer is 'numnodes', the ids of all the nodes
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1962 in this class run from 1 to numnodes, and numnodes+1 will be the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1963 id of the next node to be created in this class.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1964 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1965 return self.db.countnodes(self.classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1966
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1967 # Manipulating properties:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1968
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1969 def getprops(self, protected=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1970 """Return a dictionary mapping property names to property objects.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1971 If the "protected" flag is true, we include protected properties -
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1972 those which may not be modified.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1973
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1974 In addition to the actual properties on the node, these
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1975 methods provide the "creation" and "activity" properties. If the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1976 "protected" flag is true, we include protected properties - those
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1977 which may not be modified.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1978 """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1979 d = self.properties.copy()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1980 if protected:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
1981 d['id'] = hyperdb.String()
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1982 d['creation'] = hyperdb.Date()
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1983 d['activity'] = hyperdb.Date()
1176
bd3b57859c37 On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents: 1174
diff changeset
1984 d['creator'] = hyperdb.Link('user')
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2076
diff changeset
1985 d['actor'] = hyperdb.Link('user')
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1986 return d
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1987
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1988 def addprop(self, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1989 """Add properties to this class.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1990
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1991 The keyword arguments in 'properties' must map names to property
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1992 objects, or a TypeError is raised. None of the keys in 'properties'
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1993 may collide with the names of existing properties, or a ValueError
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1994 is raised before any properties have been added.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
1995 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1996 for key in properties:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1997 if key in self.properties:
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
1998 raise ValueError(key)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
1999 self.properties.update(properties)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2000
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2001 def index(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2002 """ Add (or refresh) the node to search indexes """
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2003 # find all the String properties that have indexme
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2004 for prop, propclass in self.getprops().iteritems():
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2005 if isinstance(propclass, hyperdb.String) and propclass.indexme:
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2006 # index them under (classname, nodeid, property)
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2007 try:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2008 value = str(self.get(nodeid, prop))
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 890
diff changeset
2009 except IndexError:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2010 # node has been destroyed
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2011 continue
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2012 self.db.indexer.add_text((self.classname, nodeid, prop), value)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2013
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2014 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2015 # import / export support
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2016 #
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2017 def export_list(self, propnames, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2018 """ Export a node - generate a list of CSV-able data in the order
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2019 specified by propnames for the given node.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2020 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2021 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2022 l = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2023 for prop in propnames:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2024 proptype = properties[prop]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2025 value = self.get(nodeid, prop)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2026 # "marshal" data where needed
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2027 if value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2028 pass
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2029 elif isinstance(proptype, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2030 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2031 elif isinstance(proptype, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2032 value = value.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2033 elif isinstance(proptype, hyperdb.Password):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2034 value = str(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2035 l.append(repr(value))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2036
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2037 # append retired flag
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2038 l.append(repr(self.is_retired(nodeid)))
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2039
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2040 return l
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2041
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2042 def import_list(self, propnames, proplist):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2043 """ Import a node - all information including "id" is present and
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2044 should not be sanity checked. Triggers are not triggered. The
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2045 journal should be initialised using the "creator" and "created"
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2046 information.
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2047
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2048 Return the nodeid of the node imported.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2049 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2050 if self.db.journaltag is None:
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2051 raise hyperdb.DatabaseError(_('Database open read-only'))
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2052 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2053
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2054 # make the new node's property map
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2055 d = {}
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2056 newid = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2057 for i in range(len(propnames)):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2058 # Figure the property for this column
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2059 propname = propnames[i]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2060
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2061 # Use eval to reverse the repr() used to output the CSV
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2062 value = eval(proplist[i])
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2063
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2064 # "unmarshal" where necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2065 if propname == 'id':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2066 newid = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2067 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2068 elif propname == 'is retired':
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2069 # is the item retired?
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2070 if int(value):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2071 d[self.db.RETIRED_FLAG] = 1
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2072 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2073 elif value is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2074 d[propname] = None
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2075 continue
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2076
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2077 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2078 if isinstance(prop, hyperdb.Date):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2079 value = date.Date(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2080 elif isinstance(prop, hyperdb.Interval):
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2081 value = date.Interval(value)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2082 elif isinstance(prop, hyperdb.Password):
4480
1613754d2646 Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4466
diff changeset
2083 value = password.Password(encrypted=value)
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2084 d[propname] = value
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2085
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2086 # get a new id if necessary
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2087 if newid is None:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2088 newid = self.db.newid(self.classname)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2089
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2090 # add the node and journal
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2091 self.db.addnode(self.classname, newid, d)
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2092 return newid
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2093
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2094 def export_journals(self):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2095 """Export a class's journal - generate a list of lists of
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2096 CSV-able data:
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2097
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2098 nodeid, date, user, action, params
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2099
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2100 No heading here - the columns are fixed.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2101 """
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2102 properties = self.getprops()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2103 r = []
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2104 for nodeid in self.getnodeids():
5232
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
2105 for nodeid, date, user, action, params in self.history(nodeid,
462b0f76fce8 issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents: 5112
diff changeset
2106 enforceperm=False, skipquiet=False):
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2107 date = date.get_tuple()
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2108 if action == 'set':
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2109 export_data = {}
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2110 for propname, value in params.iteritems():
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2111 if propname not in properties:
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2112 # property no longer in the schema
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2113 continue
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2114
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2115 prop = properties[propname]
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2116 # make sure the params are eval()'able
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2117 if value is None:
3826
bf2e9535da00 Journal and database testing.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3687
diff changeset
2118 pass
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2119 elif isinstance(prop, hyperdb.Date):
3166
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2120 # this is a hack - some dates are stored as strings
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2121 if not isinstance(value, type('')):
3204488d20b5 hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents: 3155
diff changeset
2122 value = value.get_tuple()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2123 elif isinstance(prop, hyperdb.Interval):
3390
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2124 # hack too - some intervals are stored as strings
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2125 if not isinstance(value, type('')):
a2e4b8ab5b51 include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents: 3383
diff changeset
2126 value = value.get_tuple()
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2127 elif isinstance(prop, hyperdb.Password):
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2128 value = str(value)
3003
855e9f99c96c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 2966
diff changeset
2129 export_data[propname] = value
3052
230f7394cce9 fix to fix
Richard Jones <richard@users.sourceforge.net>
parents: 3003
diff changeset
2130 params = export_data
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2131 r.append([repr(nodeid), repr(date), repr(user),
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2132 repr(action), repr(params)])
2175
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2133 return r
723098a10677 Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
2134
2597
c86b2179085b fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
2135 class FileClass(hyperdb.FileClass, Class):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2136 """This class defines a large chunk of data. To support this, it has a
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2137 mandatory String property "content" which is typically saved off
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2138 externally to the hyperdb.
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2139
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2140 The default MIME type of this data is defined by the
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2141 "default_mime_type" class attribute, which may be overridden by each
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2142 node if the class defines a "type" String property.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2143 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2144 def __init__(self, db, classname, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2145 """The newly-created class automatically includes the "content"
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2146 and "type" properties.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2147 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2148 if 'content' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2149 properties['content'] = hyperdb.String(indexme='yes')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2150 if 'type' not in properties:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2151 properties['type'] = hyperdb.String()
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2152 Class.__init__(self, db, classname, **properties)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2153
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2154 def create(self, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2155 """ Snarf the "content" propvalue and store in a file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2156 """
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2157 # we need to fire the auditors now, or the content property won't
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2158 # be in propvalues for the auditors to play with
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2159 self.fireAuditors('create', None, propvalues)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2160
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2161 # now remove the content property so it's not stored in the db
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2162 content = propvalues['content']
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2163 del propvalues['content']
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2164
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2165 # make sure we have a MIME type
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2166 mime_type = propvalues.get('type', self.default_mime_type)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2167
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2168 # do the database create
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2169 newid = self.create_inner(**propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2170
3979
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2171 # store off the content as a file
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2172 self.db.storefile(self.classname, newid, None, content)
954971d612ee Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents: 3960
diff changeset
2173
1431
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2174 # fire reactors
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2175 self.fireReactors('create', newid, None)
c70068162e64 Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents: 1424
diff changeset
2176
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2177 return newid
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2178
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2179 def get(self, nodeid, propname, default=_marker, cache=1):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2180 """ Trap the content propname and get it from the file
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2181
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2182 'cache' exists for backwards compatibility, and is not used.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2183 """
1333
80d27b7d6db5 implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents: 1327
diff changeset
2184 poss_msg = 'Possibly an access right configuration problem.'
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2185 if propname == 'content':
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2186 try:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2187 return self.db.getfile(self.classname, nodeid, None)
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5232
diff changeset
2188 except IOError as strerror:
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2189 # XXX by catching this we don't see an error in the log.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2190 return 'ERROR reading file: %s%s\n%s\n%s'%(
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2191 self.classname, nodeid, poss_msg, strerror)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2192 if default is not _marker:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2193 return Class.get(self, nodeid, propname, default)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2194 else:
1780
d2801a2b0a77 Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents: 1751
diff changeset
2195 return Class.get(self, nodeid, propname)
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2196
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2197 def set(self, itemid, **propvalues):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2198 """ Snarf the "content" propvalue and update it in a file
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2199 """
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2200 self.fireAuditors('set', itemid, propvalues)
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2201
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2202 # create the oldvalues dict - fill in any missing values
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2203 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid))
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2204 for name, prop in self.getprops(protected=0).iteritems():
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2205 if name in oldvalues:
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2206 continue
2906
a8808157f892 fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents: 2899
diff changeset
2207 if isinstance(prop, hyperdb.Multilink):
2608
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2208 oldvalues[name] = []
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2209 else:
151ee7f0ca7d fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents: 2606
diff changeset
2210 oldvalues[name] = None
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2211
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2212 # now remove the content property so it's not stored in the db
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2213 content = None
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2214 if 'content' in propvalues:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2215 content = propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2216 del propvalues['content']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2217
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2218 # do the database update
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2219 propvalues = self.set_inner(itemid, **propvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2220
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2221 # do content?
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2222 if content:
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2223 # store and possibly index
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2224 self.db.storefile(self.classname, itemid, None, content)
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2225 if self.properties['content'].indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2226 mime_type = self.get(itemid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2227 self.db.indexer.add_text((self.classname, itemid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2228 content, mime_type)
2640
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2229 propvalues['content'] = content
b01eca163779 The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents: 2637
diff changeset
2230
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2231 # fire reactors
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2232 self.fireReactors('set', itemid, oldvalues)
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2233 return propvalues
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
2234
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2235 def index(self, nodeid):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2236 """ Add (or refresh) the node to search indexes.
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2237
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2238 Use the content-type property for the content property.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2239 """
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2240 # find all the String properties that have indexme
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2241 for prop, propclass in self.getprops().iteritems():
3601
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2242 if prop == 'content' and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2243 mime_type = self.get(nodeid, 'type', self.default_mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2244 self.db.indexer.add_text((self.classname, nodeid, 'content'),
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2245 str(self.get(nodeid, 'content')), mime_type)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2246 elif isinstance(propclass, hyperdb.String) and propclass.indexme:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2247 # index them under (classname, nodeid, property)
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2248 try:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2249 value = str(self.get(nodeid, prop))
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2250 except IndexError:
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2251 # node has been destroyed
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2252 continue
7b25567f0f54 indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents: 3589
diff changeset
2253 self.db.indexer.add_text((self.classname, nodeid, prop), value)
2505
bdd112cf61ba rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents: 2496
diff changeset
2254
1103
db787cef1385 handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents: 1099
diff changeset
2255 # deviation from spec - was called ItemClass
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2256 class IssueClass(Class, roundupdb.IssueClass):
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2257 # Overridden methods:
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2258 def __init__(self, db, classname, **properties):
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2259 """The newly-created class automatically includes the "messages",
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2260 "files", "nosy", and "superseder" properties. If the 'properties'
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2261 dictionary attempts to specify any of these properties or a
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2262 "creation" or "activity" property, a ValueError is raised.
4075
b87d022a2f40 Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 4044
diff changeset
2263 """
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2264 if 'title' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2265 properties['title'] = hyperdb.String(indexme='yes')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2266 if 'messages' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2267 properties['messages'] = hyperdb.Multilink("msg")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2268 if 'files' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2269 properties['files'] = hyperdb.Multilink("file")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2270 if 'nosy' not in properties:
1002
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2271 # note: journalling is turned off as it really just wastes
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2272 # space. this behaviour may be overridden in an instance
1798d2fa9fec Hack hack...
Richard Jones <richard@users.sourceforge.net>
parents: 990
diff changeset
2273 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4348
diff changeset
2274 if 'superseder' not in properties:
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2275 properties['superseder'] = hyperdb.Multilink(classname)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2276 Class.__init__(self, db, classname, **properties)
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 843
diff changeset
2277
2699
2f5bf63a4b2c trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2650
diff changeset
2278 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/