Mercurial > p > roundup > code

annotate roundup/backends/back_bsddb3.py @ 1143:c7119e74fcf8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fixed: . handling of None for Date/Interval/Password values in export/import . handling of journal values in export/import Also played with metakit backend some, fixing some of the unit tests breakages. Hrm.
author Richard Jones <richard@users.sourceforge.net>
date Fri, 13 Sep 2002 08:20:13 +0000
parents 32e41ddf2edb
children 8dd4f736370b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
1 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
4 # under the same terms as Python, so long as this copyright message and
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
5 # disclaimer are retained in their original form.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
6 #
214
18134bffab37 stupid typo
Richard Jones <richard@users.sourceforge.net>
parents: 213
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
213
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
11 #
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
d45384bc6420 Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents: 196
diff changeset
17 #
1143
Richard Jones <richard@users.sourceforge.net>
parents: 1088
diff changeset
18 #$Id: back_bsddb3.py,v 1.17 2002-09-13 08:20:12 richard Exp $
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
19
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
20 import bsddb3, os, marshal
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
21 from roundup import hyperdb, date
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
22
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
23 # these classes are so similar, we just use the anydbm methods
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 815
diff changeset
24 from back_anydbm import Database, Class, FileClass, IssueClass
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
25
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
26 #
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
27 # Now the database
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
28 #
858
2dd862af72ee all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents: 815
diff changeset
29 class Database(Database):
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
30 """A database for storing records containing flexible data types."""
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
31 #
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
32 # Class DBs
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
33 #
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
34 def clear(self):
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
35 for cn in self.classes.keys():
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
36 db = os.path.join(self.dir, 'nodes.%s'%cn)
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
37 bsddb3.btopen(db, 'n')
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
38 db = os.path.join(self.dir, 'journals.%s'%cn)
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
39 bsddb3.btopen(db, 'n')
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
40
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
41 def getclassdb(self, classname, mode='r'):
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
42 ''' grab a connection to the class db that will be used for
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
43 multiple actions
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
44 '''
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
45 path = os.path.join(os.getcwd(), self.dir, 'nodes.%s'%classname)
152
80c062af1b7d Handle non-existence of db files in the other backends (code from anydbm).
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
46 if os.path.exists(path):
80c062af1b7d Handle non-existence of db files in the other backends (code from anydbm).
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
47 return bsddb3.btopen(path, mode)
80c062af1b7d Handle non-existence of db files in the other backends (code from anydbm).
Richard Jones <richard@users.sourceforge.net>
parents: 149
diff changeset
48 else:
815
f4ead46205ac Was reopening the database with 'n'.
Richard Jones <richard@users.sourceforge.net>
parents: 726
diff changeset
49 return bsddb3.btopen(path, 'c')
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
50
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
51 def opendb(self, name, mode):
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
52 '''Low-level database opener that gets around anydbm/dbm
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
53 eccentricities.
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
54 '''
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
55 if __debug__:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
56 print >>hyperdb.DEBUG, self, 'opendb', (self, name, mode)
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
57 # determine which DB wrote the class file
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
58 path = os.path.join(os.getcwd(), self.dir, name)
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
59 if not os.path.exists(path):
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
60 if __debug__:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
61 print >>hyperdb.DEBUG, "opendb bsddb3.open(%r, 'c')"%path
815
f4ead46205ac Was reopening the database with 'n'.
Richard Jones <richard@users.sourceforge.net>
parents: 726
diff changeset
62 return bsddb3.btopen(path, 'c')
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
63
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
64 # open the database with the correct module
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
65 if __debug__:
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
66 print >>hyperdb.DEBUG, "opendb bsddb3.open(%r, %r)"%(path, mode)
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
67 return bsddb3.btopen(path, mode)
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
68
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
69 #
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
70 # Journal
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
71 #
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
72 def getjournal(self, classname, nodeid):
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
73 ''' get the journal for id
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
74 '''
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
75 # attempt to open the journal - in some rare cases, the journal may
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
76 # not exist
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
77 try:
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
78 db = bsddb3.btopen(os.path.join(self.dir, 'journals.%s'%classname),
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
79 'r')
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
80 except bsddb3._db.DBNoSuchFileError:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
81 raise IndexError, 'no such %s %s'%(classname, nodeid)
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
82 # more handling of bad journals
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
83 if not db.has_key(nodeid):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
84 raise IndexError, 'no such %s %s'%(classname, nodeid)
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
85 journal = marshal.loads(db[nodeid])
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
86 res = []
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
87 for entry in journal:
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
88 (nodeid, date_stamp, user, action, params) = entry
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
89 date_obj = date.Date(date_stamp)
726
e3d742c75ac3 Well whadya know, bsddb3 works again.
Richard Jones <richard@users.sourceforge.net>
parents: 524
diff changeset
90 res.append((nodeid, date_obj, user, action, params))
64
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
91 db.close()
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
92 return res
8bce2210ef81 bsddb3 implementation.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
diff changeset
93
891
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
94 def getCachedJournalDB(self, classname):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
95 ''' get the journal db, looking in our cache of databases for commit
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
96 '''
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
97 # get the database handle
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
98 db_name = 'journals.%s'%classname
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
99 if self.databases.has_key(db_name):
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
100 return self.databases[db_name]
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
101 else:
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
102 db = bsddb3.btopen(os.path.join(self.dir, db_name), 'c')
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
103 self.databases[db_name] = db
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
104 return db
974a4b94c5e3 Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents: 858
diff changeset
105
Roundup Issue Tracker: http://roundup-tracker.org/