Mercurial > p > roundup > code

annotate roundup/backends/back_anydbm.py @ 843:01f9d02faea1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
...except of course it's nice to use valid Python syntax
author Richard Jones <richard@users.sourceforge.net>
date Wed, 10 Jul 2002 06:30:30 +0000
parents 4c39d8b48fbd
children 2dd862af72ee
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
1 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
4 # under the same terms as Python, so long as this copyright message and
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
5 # disclaimer are retained in their original form.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
6 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
11 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
17 #
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
18 #$Id: back_anydbm.py,v 1.43 2002-07-10 06:30:30 richard Exp $
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
19 '''
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
20 This module defines a backend that saves the hyperdatabase in a database
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
21 chosen by anydbm. It is guaranteed to always be available in python
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
22 versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
23 serious bugs, and is not available)
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
24 '''
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
26 import whichdb, anydbm, os, marshal
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
27 from roundup import hyperdb, date
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
28 from blobfiles import FileStorage
818
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
29 from roundup.indexer import Indexer
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
30 from locking import acquire_lock, release_lock
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 # Now the database
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34 #
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
35 class Database(FileStorage, hyperdb.Database):
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
36 """A database for storing records containing flexible data types.
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
37
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
38 Transaction stuff TODO:
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
39 . check the timestamp of the class file and nuke the cache if it's
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
40 modified. Do some sort of conflict checking on the dirty stuff.
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
41 . perhaps detect write collisions (related to above)?
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
42
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
43 """
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
44 def __init__(self, config, journaltag=None):
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45 """Open a hyperdatabase given a specifier to some storage.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
47 The 'storagelocator' is obtained from config.DATABASE.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 The meaning of 'storagelocator' depends on the particular
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 implementation of the hyperdatabase. It could be a file name,
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 a directory path, a socket descriptor for a connection to a
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 database over the network, etc.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 The 'journaltag' is a token that will be attached to the journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 entries for any edits done on the database. If 'journaltag' is
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 None, the database is opened in read-only mode: the Class.create(),
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 Class.set(), and Class.retire() methods are disabled.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 """
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
58 self.config, self.journaltag = config, journaltag
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
59 self.dir = config.DATABASE
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 self.classes = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
61 self.cache = {} # cache of nodes loaded or created
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
62 self.dirtynodes = {} # keep track of the dirty nodes by class
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
63 self.newnodes = {} # keep track of the new nodes by class
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
64 self.transactions = []
818
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
65 self.indexer = Indexer(self.dir)
697
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
66 # ensure files are group readable and writable
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
67 os.umask(0002)
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
68
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
69 def post_init(self):
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
70 """Called once the schema initialisation has finished."""
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
71 # reindex the db if necessary
826
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
72 if self.indexer.should_reindex():
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
73 self.reindex()
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
74
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
75 def reindex(self):
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
76 for klass in self.classes.values():
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
77 for nodeid in klass.list():
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
78 klass.index(nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
79 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
80
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
81 def __repr__(self):
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
82 return '<back_anydbm instance at %x>'%id(self)
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
83
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 # Classes
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 def __getattr__(self, classname):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 """A convenient way of calling self.getclass(classname)."""
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
89 if self.classes.has_key(classname):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
90 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
91 print >>hyperdb.DEBUG, '__getattr__', (self, classname)
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
92 return self.classes[classname]
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
93 raise AttributeError, classname
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 def addclass(self, cl):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
96 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
97 print >>hyperdb.DEBUG, 'addclass', (self, cl)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 cn = cl.classname
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 if self.classes.has_key(cn):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 raise ValueError, cn
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 self.classes[cn] = cl
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103 def getclasses(self):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
104 """Return a list of the names of all existing classes."""
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
105 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
106 print >>hyperdb.DEBUG, 'getclasses', (self,)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 l = self.classes.keys()
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 l.sort()
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 return l
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 def getclass(self, classname):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 """Get the Class object representing a particular class.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 If 'classname' is not a valid class name, a KeyError is raised.
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 """
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
116 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
117 print >>hyperdb.DEBUG, 'getclass', (self, classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 return self.classes[classname]
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 # Class DBs
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 def clear(self):
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
124 '''Delete all database contents
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
125 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
126 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
127 print >>hyperdb.DEBUG, 'clear', (self,)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 for cn in self.classes.keys():
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
129 for dummy in 'nodes', 'journals':
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
130 path = os.path.join(self.dir, 'journals.%s'%cn)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
131 if os.path.exists(path):
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
132 os.remove(path)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
133 elif os.path.exists(path+'.db'): # dbm appends .db
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
134 os.remove(path+'.db')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 def getclassdb(self, classname, mode='r'):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 ''' grab a connection to the class db that will be used for
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 multiple actions
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
140 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
141 print >>hyperdb.DEBUG, 'getclassdb', (self, classname, mode)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
142 return self._opendb('nodes.%s'%classname, mode)
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
143
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
144 def _opendb(self, name, mode):
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
145 '''Low-level database opener that gets around anydbm/dbm
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
146 eccentricities.
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
147 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
148 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
149 print >>hyperdb.DEBUG, '_opendb', (self, name, mode)
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
150
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
151 # determine which DB wrote the class file
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
152 db_type = ''
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
153 path = os.path.join(os.getcwd(), self.dir, name)
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
154 if os.path.exists(path):
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
155 db_type = whichdb.whichdb(path)
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
156 if not db_type:
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
157 raise hyperdb.DatabaseError, "Couldn't identify database type"
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
158 elif os.path.exists(path+'.db'):
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
159 # if the path ends in '.db', it's a dbm database, whether
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
160 # anydbm says it's dbhash or not!
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
161 db_type = 'dbm'
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
162
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
163 # new database? let anydbm pick the best dbm
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
164 if not db_type:
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
165 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
166 print >>hyperdb.DEBUG, "_opendb anydbm.open(%r, 'n')"%path
71
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
167 return anydbm.open(path, 'n')
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
168
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
169 # open the database with the correct module
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
170 try:
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
171 dbm = __import__(db_type)
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
172 except ImportError:
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
173 raise hyperdb.DatabaseError, \
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
174 "Couldn't open database - the required module '%s'"\
818
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
175 " is not available"%db_type
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
176 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
177 print >>hyperdb.DEBUG, "_opendb %r.open(%r, %r)"%(db_type, path,
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
178 mode)
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
179 return dbm.open(path, mode)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
180
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
181 def _lockdb(self, name):
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
182 ''' Lock a database file
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
183 '''
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
184 path = os.path.join(os.getcwd(), self.dir, '%s.lock'%name)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
185 return acquire_lock(path)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
186
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
187 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
188 # Node IDs
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
189 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
190 def newid(self, classname):
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
191 ''' Generate a new id for the given class
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
192 '''
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
193 # open the ids DB - create if if doesn't exist
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
194 lock = self._lockdb('_ids')
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
195 db = self._opendb('_ids', 'c')
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
196 if db.has_key(classname):
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
197 newid = db[classname] = str(int(db[classname]) + 1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
198 else:
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
199 # the count() bit is transitional - older dbs won't start at 1
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
200 newid = str(self.getclass(classname).count()+1)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
201 db[classname] = newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
202 db.close()
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
203 release_lock(lock)
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
204 return newid
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
205
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
206 #
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
207 # Nodes
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
208 #
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 def addnode(self, classname, nodeid, node):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
210 ''' add the specified node to its class's db
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
212 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
213 print >>hyperdb.DEBUG, 'addnode', (self, classname, nodeid, node)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
214 self.newnodes.setdefault(classname, {})[nodeid] = 1
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
215 self.cache.setdefault(classname, {})[nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
216 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
217
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
218 def setnode(self, classname, nodeid, node):
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
219 ''' change the specified node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
220 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
221 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
222 print >>hyperdb.DEBUG, 'setnode', (self, classname, nodeid, node)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
223 self.dirtynodes.setdefault(classname, {})[nodeid] = 1
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
224
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
225 # can't set without having already loaded the node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
226 self.cache[classname][nodeid] = node
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
227 self.savenode(classname, nodeid, node)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
228
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
229 def savenode(self, classname, nodeid, node):
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
230 ''' perform the saving of data specified by the set/addnode
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
231 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
232 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
233 print >>hyperdb.DEBUG, 'savenode', (self, classname, nodeid, node)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
234 self.transactions.append((self._doSaveNode, (classname, nodeid, node)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
236 def getnode(self, classname, nodeid, db=None, cache=1):
460
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
237 ''' get a node from the database
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
239 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
240 print >>hyperdb.DEBUG, 'getnode', (self, classname, nodeid, db)
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
241 if cache:
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
242 # try the cache
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
243 cache_dict = self.cache.setdefault(classname, {})
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
244 if cache_dict.has_key(nodeid):
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
245 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
246 print >>hyperdb.TRACE, 'get %s %s cached'%(classname,
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
247 nodeid)
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
248 return cache_dict[nodeid]
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
249
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
250 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
251 print >>hyperdb.TRACE, 'get %s %s'%(classname, nodeid)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
252
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
253 # get from the database and save in the cache
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
254 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
255 db = self.getclassdb(classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
256 if not db.has_key(nodeid):
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
257 raise IndexError, "no such %s %s"%(classname, nodeid)
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
258
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
259 # decode
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
260 res = marshal.loads(db[nodeid])
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
261
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
262 # reverse the serialisation
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
263 res = self.unserialise(classname, res)
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
264
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
265 # store off in the cache dict
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
266 if cache:
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
267 cache_dict[nodeid] = res
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
268
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
271 def hasnode(self, classname, nodeid, db=None):
460
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
272 ''' determine if the database has a given node
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
273 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
274 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
275 print >>hyperdb.DEBUG, 'hasnode', (self, classname, nodeid, db)
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
276
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
277 # try the cache
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
278 cache = self.cache.setdefault(classname, {})
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
279 if cache.has_key(nodeid):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
280 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
281 print >>hyperdb.TRACE, 'has %s %s cached'%(classname, nodeid)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
282 return 1
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
283 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
284 print >>hyperdb.TRACE, 'has %s %s'%(classname, nodeid)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
285
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
286 # not in the cache - check the database
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
287 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
288 db = self.getclassdb(classname)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
289 res = db.has_key(nodeid)
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
290 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
291
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
292 def countnodes(self, classname, db=None):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
293 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
294 print >>hyperdb.DEBUG, 'countnodes', (self, classname, db)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
295 # include the new nodes not saved to the DB yet
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
296 count = len(self.newnodes.get(classname, {}))
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
297
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
298 # and count those in the DB
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
299 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
300 db = self.getclassdb(classname)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
301 count = count + len(db.keys())
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
302 return count
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
303
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
304 def getnodeids(self, classname, db=None):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
305 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
306 print >>hyperdb.DEBUG, 'getnodeids', (self, classname, db)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
307 # start off with the new nodes
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
308 res = self.newnodes.get(classname, {}).keys()
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
309
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
310 if db is None:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
311 db = self.getclassdb(classname)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
312 res = res + db.keys()
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
313 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
315
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
316 #
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
317 # Files - special node properties
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
318 # inherited from FileStorage
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
319
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
320 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
321 # Journal
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
322 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
323 def addjournal(self, classname, nodeid, action, params):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
324 ''' Journal the Action
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325 'action' may be:
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
326
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
327 'create' or 'set' -- 'params' is a dictionary of property values
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
328 'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
329 'retire' -- 'params' is None
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
330 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
331 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
332 print >>hyperdb.DEBUG, 'addjournal', (self, classname, nodeid,
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
333 action, params)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
334 self.transactions.append((self._doSaveJournal, (classname, nodeid,
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
335 action, params)))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
336
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
337 def getjournal(self, classname, nodeid):
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
338 ''' get the journal for id
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
339 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
340 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
341 print >>hyperdb.DEBUG, 'getjournal', (self, classname, nodeid)
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
342 # attempt to open the journal - in some rare cases, the journal may
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
343 # not exist
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
344 try:
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
345 db = self._opendb('journals.%s'%classname, 'r')
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
346 except anydbm.error, error:
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
347 if str(error) == "need 'c' or 'n' flag to open new db": return []
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
348 elif error.args[0] != 2: raise
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
349 return []
787
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
350 try:
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
351 journal = marshal.loads(db[nodeid])
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
352 except KeyError:
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
353 db.close()
787
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
354 raise KeyError, 'no such %s %s'%(classname, nodeid)
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
355 db.close()
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
356 res = []
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
357 for entry in journal:
624
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
358 (nodeid, date_stamp, user, action, params) = entry
48
671203878652 Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents: 46
diff changeset
359 date_obj = date.Date(date_stamp)
624
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
360 res.append((nodeid, date_obj, user, action, params))
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 return res
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
363 def pack(self, pack_before):
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
364 ''' delete all journal entries before 'pack_before' '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
365 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
366 print >>hyperdb.DEBUG, 'packjournal', (self, pack_before)
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
367
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
368 pack_before = pack_before.get_tuple()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
369
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
370 classes = self.getclasses()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
371
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
372 # TODO: factor this out to method - we're already doing it in
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
373 # _opendb.
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
374 db_type = ''
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
375 path = os.path.join(os.getcwd(), self.dir, classes[0])
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
376 if os.path.exists(path):
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
377 db_type = whichdb.whichdb(path)
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
378 if not db_type:
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
379 raise hyperdb.DatabaseError, "Couldn't identify database type"
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
380 elif os.path.exists(path+'.db'):
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
381 db_type = 'dbm'
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
382
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
383 for classname in classes:
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
384 db_name = 'journals.%s'%classname
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
385 db = self._opendb(db_name, 'w')
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
386
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
387 for key in db.keys():
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
388 journal = marshal.loads(db[key])
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
389 l = []
567
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
390 last_set_entry = None
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
391 for entry in journal:
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
392 (nodeid, date_stamp, self.journaltag, action,
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
393 params) = entry
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
394 if date_stamp > pack_before or action == 'create':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
395 l.append(entry)
566
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
396 elif action == 'set':
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
397 # grab the last set entry to keep information on
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
398 # activity
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
399 last_set_entry = entry
567
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
400 if last_set_entry:
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
401 date_stamp = last_set_entry[1]
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
402 # if the last set entry was made after the pack date
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
403 # then it is already in the list
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
404 if date_stamp < pack_before:
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
405 l.append(last_set_entry)
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
406 db[key] = marshal.dumps(l)
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
407 if db_type == 'gdbm':
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
408 db.reorganize()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
409 db.close()
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
410
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
411
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
412 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
413 # Basic transaction support
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
414 #
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
415 def commit(self):
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
416 ''' Commit the current transactions.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
417 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
418 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
419 print >>hyperdb.DEBUG, 'commit', (self,)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
420 # TODO: lock the DB
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
421
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
422 # keep a handle to all the database files opened
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
423 self.databases = {}
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
424
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
425 # now, do all the transactions
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
426 reindex = {}
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
427 for method, args in self.transactions:
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
428 reindex[method(*args)] = 1
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
429
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
430 # now close all the database files
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
431 for db in self.databases.values():
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
432 db.close()
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
433 del self.databases
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
434 # TODO: unlock the DB
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
435
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
436 # reindex the nodes that request it
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
437 for classname, nodeid in filter(None, reindex.keys()):
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
438 print >>hyperdb.DEBUG, 'commit.reindex', (classname, nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
439 self.getclass(classname).index(nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
440
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
441 # save the indexer state
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
442 self.indexer.save_index()
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
443
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
444 # all transactions committed, back to normal
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
445 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
446 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
447 self.newnodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
448 self.transactions = []
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
449
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
450 def _doSaveNode(self, classname, nodeid, node):
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
451 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
452 print >>hyperdb.DEBUG, '_doSaveNode', (self, classname, nodeid,
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
453 node)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
454
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
455 # get the database handle
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
456 db_name = 'nodes.%s'%classname
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
457 if self.databases.has_key(db_name):
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
458 db = self.databases[db_name]
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
459 else:
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
460 db = self.databases[db_name] = self.getclassdb(classname, 'c')
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
461
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
462 # now save the marshalled data
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
463 db[nodeid] = marshal.dumps(self.serialise(classname, node))
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
464
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
465 # return the classname, nodeid so we reindex this content
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
466 return (classname, nodeid)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
467
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
468 def _doSaveJournal(self, classname, nodeid, action, params):
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
469 # serialise first
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
470 if action in ('set', 'create'):
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
471 params = self.serialise(classname, params)
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
472
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
473 # create the journal entry
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
474 entry = (nodeid, date.Date().get_tuple(), self.journaltag, action,
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
475 params)
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
476
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
477 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
478 print >>hyperdb.DEBUG, '_doSaveJournal', entry
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
479
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
480 # get the database handle
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
481 db_name = 'journals.%s'%classname
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
482 if self.databases.has_key(db_name):
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
483 db = self.databases[db_name]
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
484 else:
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
485 db = self.databases[db_name] = self._opendb(db_name, 'c')
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
486
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
487 # now insert the journal entry
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
488 if db.has_key(nodeid):
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
489 # append to existing
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
490 s = db[nodeid]
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
491 l = marshal.loads(s)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
492 l.append(entry)
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
493 else:
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
494 l = [entry]
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
495
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
496 db[nodeid] = marshal.dumps(l)
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
497
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
498 def rollback(self):
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
499 ''' Reverse all actions from the current transaction.
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
500 '''
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
501 if __debug__:
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
502 print >>hyperdb.DEBUG, 'rollback', (self, )
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
503 for method, args in self.transactions:
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
504 # delete temporary files
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
505 if method == self._doStoreFile:
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
506 self._rollbackStoreFile(*args)
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
507 self.cache = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
508 self.dirtynodes = {}
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
509 self.newnodes = {}
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
510 self.transactions = []
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
511
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
512 #
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
513 #$Log: not supported by cvs2svn $
843
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
514 #Revision 1.42 2002/07/10 06:21:38 richard
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
515 #Be extra safe
01f9d02faea1 ...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents: 842
diff changeset
516 #
842
4c39d8b48fbd Be extra safe
Richard Jones <richard@users.sourceforge.net>
parents: 838
diff changeset
517 #Revision 1.41 2002/07/10 00:21:45 richard
4c39d8b48fbd Be extra safe
Richard Jones <richard@users.sourceforge.net>
parents: 838
diff changeset
518 #explicit database closing
4c39d8b48fbd Be extra safe
Richard Jones <richard@users.sourceforge.net>
parents: 838
diff changeset
519 #
838
b31ac31ece69 explicit database closing
Richard Jones <richard@users.sourceforge.net>
parents: 826
diff changeset
520 #Revision 1.40 2002/07/09 04:19:09 richard
b31ac31ece69 explicit database closing
Richard Jones <richard@users.sourceforge.net>
parents: 826
diff changeset
521 #Added reindex command to roundup-admin.
b31ac31ece69 explicit database closing
Richard Jones <richard@users.sourceforge.net>
parents: 826
diff changeset
522 #Fixed reindex on first access.
b31ac31ece69 explicit database closing
Richard Jones <richard@users.sourceforge.net>
parents: 826
diff changeset
523 #Also fixed reindexing of entries that change.
b31ac31ece69 explicit database closing
Richard Jones <richard@users.sourceforge.net>
parents: 826
diff changeset
524 #
826
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
525 #Revision 1.39 2002/07/09 03:02:52 richard
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
526 #More indexer work:
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
527 #- all String properties may now be indexed too. Currently there's a bit of
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
528 # "issue" specific code in the actual searching which needs to be
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
529 # addressed. In a nutshell:
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
530 # + pass 'indexme="yes"' as a String() property initialisation arg, eg:
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
531 # file = FileClass(db, "file", name=String(), type=String(),
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
532 # comment=String(indexme="yes"))
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
533 # + the comment will then be indexed and be searchable, with the results
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
534 # related back to the issue that the file is linked to
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
535 #- as a result of this work, the FileClass has a default MIME type that may
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
536 # be overridden in a subclass, or by the use of a "type" property as is
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
537 # done in the default templates.
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
538 #- the regeneration of the indexes (if necessary) is done once the schema is
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
539 # set up in the dbinit.
6d7a45c8464a Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents: 825
diff changeset
540 #
825
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
541 #Revision 1.38 2002/07/08 06:58:15 richard
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
542 #cleaned up the indexer code:
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
543 # - it splits more words out (much simpler, faster splitter)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
544 # - removed code we'll never use (roundup.roundup_indexer has the full
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
545 # implementation, and replaces roundup.indexer)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
546 # - only index text/plain and rfc822/message (ideas for other text formats to
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
547 # index are welcome)
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
548 # - added simple unit test for indexer. Needs more tests for regression.
0779ea9f1f18 More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents: 818
diff changeset
549 #
818
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
550 #Revision 1.37 2002/06/20 23:52:35 richard
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
551 #More informative error message
254b8d112eec cleaned up the indexer code:
Richard Jones <richard@users.sourceforge.net>
parents: 787
diff changeset
552 #
787
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
553 #Revision 1.36 2002/06/19 03:07:19 richard
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
554 #Moved the file storage commit into blobfiles where it belongs.
b6b0a92e0738 More informative error message
Richard Jones <richard@users.sourceforge.net>
parents: 778
diff changeset
555 #
778
fb2ce2b8b2ef Moved the file storage commit into blobfiles where it belongs.
Richard Jones <richard@users.sourceforge.net>
parents: 749
diff changeset
556 #Revision 1.35 2002/05/25 07:16:24 rochecompaan
fb2ce2b8b2ef Moved the file storage commit into blobfiles where it belongs.
Richard Jones <richard@users.sourceforge.net>
parents: 749
diff changeset
557 #Merged search_indexing-branch with HEAD
fb2ce2b8b2ef Moved the file storage commit into blobfiles where it belongs.
Richard Jones <richard@users.sourceforge.net>
parents: 749
diff changeset
558 #
749
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
559 #Revision 1.34 2002/05/15 06:21:21 richard
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
560 # . node caching now works, and gives a small boost in performance
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
561 #
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
562 #As a part of this, I cleaned up the DEBUG output and implemented TRACE
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
563 #output (HYPERDBTRACE='file to trace to') with checkpoints at the start of
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
564 #CGI requests. Run roundup with python -O to skip all the DEBUG/TRACE stuff
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
565 #(using if __debug__ which is compiled out with -O)
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
566 #
719
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
567 #Revision 1.33 2002/04/24 10:38:26 rochecompaan
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
568 #All database files are now created group readable and writable.
fed4c363a7f3 node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents: 697
diff changeset
569 #
697
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
570 #Revision 1.32 2002/04/15 23:25:15 richard
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
571 #. node ids are now generated from a lockable store - no more race conditions
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
572 #
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
573 #We're using the portalocker code by Jonathan Feinberg that was contributed
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
574 #to the ASPN Python cookbook. This gives us locking across Unix and Windows.
210e1ae39ab1 All database files are now created group readable and writable.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 690
diff changeset
575 #
690
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
576 #Revision 1.31 2002/04/03 05:54:31 richard
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
577 #Fixed serialisation problem by moving the serialisation step out of the
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
578 #hyperdb.Class (get, set) into the hyperdb.Database.
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
579 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
580 #Also fixed htmltemplate after the showid changes I made yesterday.
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
581 #
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
582 #Unit tests for all of the above written.
509a101305da node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents: 676
diff changeset
583 #
749
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
584 #Revision 1.30.2.1 2002/04/03 11:55:57 rochecompaan
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
585 # . Added feature #526730 - search for messages capability
51c425129b35 Merged search_indexing-branch with HEAD
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 719
diff changeset
586 #
676
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
587 #Revision 1.30 2002/02/27 03:40:59 richard
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
588 #Ran it through pychecker, made fixes
bc46480e2a2b Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents: 650
diff changeset
589 #
650
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
590 #Revision 1.29 2002/02/25 14:34:31 grubert
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
591 # . use blobfiles in back_anydbm which is used in back_bsddb.
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
592 # change test_db as dirlist does not work for subdirectories.
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
593 # ATTENTION: blobfiles now creates subdirectories for files.
9b2575610953 Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents: 646
diff changeset
594 #
646
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
595 #Revision 1.28 2002/02/16 09:14:17 richard
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
596 # . #514854 ] History: "User" is always ticket creator
07abfe8f0c01 use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents: 624
diff changeset
597 #
624
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
598 #Revision 1.27 2002/01/22 07:21:13 richard
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
599 #. fixed back_bsddb so it passed the journal tests
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
600 #
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
601 #... it didn't seem happy using the back_anydbm _open method, which is odd.
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
602 #Yet another occurrance of whichdb not being able to recognise older bsddb
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
603 #databases. Yadda yadda. Made the HYPERDBDEBUG stuff more sane in the
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
604 #process.
352d67e22d6d [SF#514854] History: "User" is always ticket creator
Richard Jones <richard@users.sourceforge.net>
parents: 570
diff changeset
605 #
570
e346a9792335 fixed back_bsddb so it passed the journal tests
Richard Jones <richard@users.sourceforge.net>
parents: 567
diff changeset
606 #Revision 1.26 2002/01/22 05:18:38 rochecompaan
e346a9792335 fixed back_bsddb so it passed the journal tests
Richard Jones <richard@users.sourceforge.net>
parents: 567
diff changeset
607 #last_set_entry was referenced before assignment
e346a9792335 fixed back_bsddb so it passed the journal tests
Richard Jones <richard@users.sourceforge.net>
parents: 567
diff changeset
608 #
567
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
609 #Revision 1.25 2002/01/22 05:06:08 rochecompaan
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
610 #We need to keep the last 'set' entry in the journal to preserve
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
611 #information on 'activity' for nodes.
9ca63f7332a7 last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 566
diff changeset
612 #
566
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
613 #Revision 1.24 2002/01/21 16:33:20 rochecompaan
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
614 #You can now use the roundup-admin tool to pack the database
73af3e54c309 We need to keep the last 'set' entry in the journal...
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 562
diff changeset
615 #
562
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
616 #Revision 1.23 2002/01/18 04:32:04 richard
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
617 #Rollback was breaking because a message hadn't actually been written to the file. Needs
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
618 #more investigation.
62febbd7ffec You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 553
diff changeset
619 #
553
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
620 #Revision 1.22 2002/01/14 02:20:15 richard
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
621 # . changed all config accesses so they access either the instance or the
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
622 # config attriubute on the db. This means that all config is obtained from
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
623 # instance_config instead of the mish-mash of classes. This will make
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
624 # switching to a ConfigParser setup easier too, I hope.
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
625 #
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
626 #At a minimum, this makes migration a _little_ easier (a lot easier in the
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
627 #0.5.0 switch, I hope!)
10ee3427011f Rollback was breaking...
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
628 #
524
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
629 #Revision 1.21 2002/01/02 02:31:38 richard
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
630 #Sorry for the huge checkin message - I was only intending to implement #496356
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
631 #but I found a number of places where things had been broken by transactions:
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
632 # . modified ROUNDUPDBSENDMAILDEBUG to be SENDMAILDEBUG and hold a filename
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
633 # for _all_ roundup-generated smtp messages to be sent to.
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
634 # . the transaction cache had broken the roundupdb.Class set() reactors
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
635 # . newly-created author users in the mailgw weren't being committed to the db
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
636 #
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
637 #Stuff that made it into CHANGES.txt (ie. the stuff I was actually working
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
638 #on when I found that stuff :):
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
639 # . #496356 ] Use threading in messages
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
640 # . detectors were being registered multiple times
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
641 # . added tests for mailgw
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
642 # . much better attaching of erroneous messages in the mail gateway
dce4c75bef5a changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents: 475
diff changeset
643 #
475
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
644 #Revision 1.20 2001/12/18 15:30:34 rochecompaan
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
645 #Fixed bugs:
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
646 # . Fixed file creation and retrieval in same transaction in anydbm
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
647 # backend
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
648 # . Cgi interface now renders new issue after issue creation
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
649 # . Could not set issue status to resolved through cgi interface
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
650 # . Mail gateway was changing status back to 'chatting' if status was
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
651 # omitted as an argument
a1a44636bace Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents: 464
diff changeset
652 #
464
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
653 #Revision 1.19 2001/12/17 03:52:48 richard
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
654 #Implemented file store rollback. As a bonus, the hyperdb is now capable of
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
655 #storing more than one file per node - if a property name is supplied,
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
656 #the file is called designator.property.
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
657 #I decided not to migrate the existing files stored over to the new naming
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
658 #scheme - the FileClass just doesn't specify the property name.
29f5ac8a0d2b Fixed bugs:
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 461
diff changeset
659 #
461
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
660 #Revision 1.18 2001/12/16 10:53:38 richard
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
661 #take a copy of the node dict so that the subsequent set
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
662 #operation doesn't modify the oldvalues structure
b579418f7ed1 Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents: 460
diff changeset
663 #
460
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
664 #Revision 1.17 2001/12/14 23:42:57 richard
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
665 #yuck, a gdbm instance tests false :(
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
666 #I've left the debugging code in - it should be removed one day if we're ever
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
667 #_really_ anal about performace :)
9c895b44240a take a copy of the node dict...
Richard Jones <richard@users.sourceforge.net>
parents: 452
diff changeset
668 #
452
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
669 #Revision 1.16 2001/12/12 03:23:14 richard
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
670 #Cor blimey this anydbm/whichdb stuff is yecchy. Turns out that whichdb
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
671 #incorrectly identifies a dbm file as a dbhash file on my system. This has
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
672 #been submitted to the python bug tracker as issue #491888:
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
673 #https://sourceforge.net/tracker/index.php?func=detail&aid=491888&group_id=5470&atid=105470
7181efddce66 yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents: 444
diff changeset
674 #
444
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
675 #Revision 1.15 2001/12/12 02:30:51 richard
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
676 #I fixed the problems with people whose anydbm was using the dbm module at the
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
677 #backend. It turns out the dbm module modifies the file name to append ".db"
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
678 #and my check to determine if we're opening an existing or new db just
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
679 #tested os.path.exists() on the filename. Well, no longer! We now perform a
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
680 #much better check _and_ cope with the anydbm implementation module changing
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
681 #too!
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
682 #I also fixed the backends __init__ so only ImportError is squashed.
3fa2268041a8 Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents: 443
diff changeset
683 #
443
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
684 #Revision 1.14 2001/12/10 22:20:01 richard
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
685 #Enabled transaction support in the bsddb backend. It uses the anydbm code
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
686 #where possible, only replacing methods where the db is opened (it uses the
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
687 #btree opener specifically.)
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
688 #Also cleaned up some change note generation.
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
689 #Made the backends package work with pydoc too.
a0c598702f17 I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents: 440
diff changeset
690 #
440
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
691 #Revision 1.13 2001/12/02 05:06:16 richard
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
692 #. We now use weakrefs in the Classes to keep the database reference, so
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
693 # the close() method on the database is no longer needed.
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
694 # I bumped the minimum python requirement up to 2.1 accordingly.
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
695 #. #487480 ] roundup-server
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
696 #. #487476 ] INSTALL.txt
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
697 #
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
698 #I also cleaned up the change message / post-edit stuff in the cgi client.
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
699 #There's now a clearly marked "TODO: append the change note" where I believe
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
700 #the change note should be added there. The "changes" list will obviously
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
701 #have to be modified to be a dict of the changes, or somesuch.
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
702 #
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
703 #More testing needed.
de5bf4191f11 Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents: 431
diff changeset
704 #
431
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
705 #Revision 1.12 2001/12/01 07:17:50 richard
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
706 #. We now have basic transaction support! Information is only written to
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
707 # the database when the commit() method is called. Only the anydbm
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
708 # backend is modified in this way - neither of the bsddb backends have been.
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
709 # The mail, admin and cgi interfaces all use commit (except the admin tool
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
710 # doesn't have a commit command, so interactive users can't commit...)
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
711 #. Fixed login/registration forwarding the user to the right page (or not,
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
712 # on a failure)
a28a80b714f9 Eliminate database close method by using weakrefs.
Richard Jones <richard@users.sourceforge.net>
parents: 430
diff changeset
713 #
430
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
714 #Revision 1.11 2001/11/21 02:34:18 richard
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
715 #Added a target version field to the extended issue schema
350685601f37 Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents: 394
diff changeset
716 #
394
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
717 #Revision 1.10 2001/10/09 23:58:10 richard
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
718 #Moved the data stringification up into the hyperdb.Class class' get, set
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
719 #and create methods. This means that the data is also stringified for the
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
720 #journal call, and removes duplication of code from the backends. The
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
721 #backend code now only sees strings.
f43af1e97fdd Added a target version field to the extended issue schema
Richard Jones <richard@users.sourceforge.net>
parents: 275
diff changeset
722 #
275
1cc866cec608 Moved the data stringification up into the hyperdb.Class class's...
Richard Jones <richard@users.sourceforge.net>
parents: 270
diff changeset
723 #Revision 1.9 2001/10/09 07:25:59 richard
1cc866cec608 Moved the data stringification up into the hyperdb.Class class's...
Richard Jones <richard@users.sourceforge.net>
parents: 270
diff changeset
724 #Added the Password property type. See "pydoc roundup.password" for
1cc866cec608 Moved the data stringification up into the hyperdb.Class class's...
Richard Jones <richard@users.sourceforge.net>
parents: 270
diff changeset
725 #implementation details. Have updated some of the documentation too.
1cc866cec608 Moved the data stringification up into the hyperdb.Class class's...
Richard Jones <richard@users.sourceforge.net>
parents: 270
diff changeset
726 #
270
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 252
diff changeset
727 #Revision 1.8 2001/09/29 13:27:00 richard
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 252
diff changeset
728 #CGI interfaces now spit up a top-level index of all the instances they can
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 252
diff changeset
729 #serve.
a4241ddd22d7 Added the Password property type.
Richard Jones <richard@users.sourceforge.net>
parents: 252
diff changeset
730 #
252
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
731 #Revision 1.7 2001/08/12 06:32:36 richard
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
732 #using isinstance(blah, Foo) now instead of isFooType
76c6994aa4e8 CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents: 224
diff changeset
733 #
224
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
734 #Revision 1.6 2001/08/07 00:24:42 richard
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
735 #stupid typo
ad2c98faec97 using isinstance(blah, Foo) now instead of isFooType
Richard Jones <richard@users.sourceforge.net>
parents: 214
diff changeset
736 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
737 #Revision 1.5 2001/08/07 00:15:51 richard
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
738 #Added the copyright/license notice to (nearly) all files at request of
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
739 #Bizar Software.
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
740 #
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
741 #Revision 1.4 2001/07/30 01:41:36 richard
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
742 #Makes schema changes mucho easier.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
743 #
149
fbc77ecf133b Makes schema changes mucho easier.
Richard Jones <richard@users.sourceforge.net>
parents: 71
diff changeset
744 #Revision 1.3 2001/07/25 01:23:07 richard
fbc77ecf133b Makes schema changes mucho easier.
Richard Jones <richard@users.sourceforge.net>
parents: 71
diff changeset
745 #Added the Roundup spec to the new documentation directory.
fbc77ecf133b Makes schema changes mucho easier.
Richard Jones <richard@users.sourceforge.net>
parents: 71
diff changeset
746 #
71
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
747 #Revision 1.2 2001/07/23 08:20:44 richard
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
748 #Moved over to using marshal in the bsddb and anydbm backends.
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
749 #roundup-admin now has a "freshen" command that'll load/save all nodes (not
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
750 # retired - mod hyperdb.Class.list() so it lists retired nodes)
46
3c5920433866 *sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
751 #
71
5147b4c51fd5 Added the Roundup spec to the new documentation directory.
Richard Jones <richard@users.sourceforge.net>
parents: 48
diff changeset
752 #
Roundup Issue Tracker: http://roundup-tracker.org/