Mercurial > p > roundup > code

annotate roundup/test/memorydb.py @ 6917:fba76e0bba98

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
set journal mode to wal when creating db. Trying to work around sqlite3.OperationalError: disk I/O error errors in CI that I can't reproduce locally. Also this allows reads to happen while writes are occurring. This should permit using the web interface for browsing while a large write load (e.g. bulk data loading from another tracker) is happening. Right now autogenerating issues locks up the tracker for reading with an error that the database is locked.
author John Rouillard <rouilj@ieee.org>
date Tue, 06 Sep 2022 11:32:13 -0400
parents 2ce855803633
children b1130680d4ed
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 '''Implement an in-memory hyperdb for testing purposes.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 import shutil
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
5 import os
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
6 import time
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
8 from roundup import date
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 from roundup import hyperdb
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 from roundup import roundupdb
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 from roundup import security
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 from roundup import password
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 from roundup import configuration
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 from roundup.backends import back_anydbm
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 from roundup.backends import indexer_dbm
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
16 from roundup.backends import sessions_dbm
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 from roundup.backends import indexer_common
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
18 from roundup.support import ensureParentsExist
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
19 from roundup.anypy.strings import s2b
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
20 from roundup.test.tx_Source_detector import init as tx_Source_init
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
22 default_prefix = '../../share/roundup/templates/classic'
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
23
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
24 def new_config(debug=False, prefix=default_prefix):
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
25 if not prefix.startswith('/'):
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
26 prefix = os.path.join (os.path.dirname(__file__), prefix)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 config = configuration.CoreConfig()
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
28 config.detectors = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
29 os.path.join(prefix, "detectors/config.ini"))
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
30 config.ext = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
31 os.path.join(prefix, "extensions/config.ini"))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 config.DATABASE = "db"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 #config.logging = MockNull()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34 # these TRACKER_WEB and MAIL_DOMAIN values are used in mailgw tests
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
35 if debug:
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
36 config.LOGGING_LEVEL = "DEBUG"
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
37 config.MAIL_DOMAIN = "your.tracker.email.domain.example"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38 config.TRACKER_WEB = "http://tracker.example/cgi-bin/roundup.cgi/bugs/"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 return config
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
41 def create(journaltag, create=True, debug=False, prefix=default_prefix):
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
42 # "Nuke" in-memory db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
43 db_nuke('')
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
44
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
45 db = Database(new_config(debug), journaltag)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47 # load standard schema
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
48 if not prefix.startswith('/'):
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
49 prefix = os.path.join (os.path.dirname(__file__), prefix)
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
50
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
51 schema = os.path.join(prefix, 'schema.py')
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
52 vars = hyperdb.__dict__
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
53 vars['Class'] = Class
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
54 vars['FileClass'] = FileClass
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
55 vars['IssueClass'] = IssueClass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 vars['db'] = db
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
57 fd = open(schema)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
58 exec(compile(fd.read(), schema, 'exec'), vars)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
59 fd.close()
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
60
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
61 initial_data = os.path.join(prefix, 'initial_data.py')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 vars = dict(db=db, admin_email='admin@test.com',
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 adminpw=password.Password('sekrit'))
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
64 fd = open(initial_data)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
65 exec(compile(fd.read(), initial_data, 'exec'), vars)
6493
a7072cf4ad7e Actually call close().
John Rouillard <rouilj@ieee.org>
parents: 6492
diff changeset
66 fd.close()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 # load standard detectors
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
69 dirname = os.path.join(prefix, 'detectors')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 for fn in os.listdir(dirname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 if not fn.endswith('.py'): continue
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 vars = {}
6491
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
73 with open(os.path.join(dirname, fn)) as fd:
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
74 exec(compile(fd.read(),
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
75 os.path.join(dirname, fn), 'exec'), vars)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 vars['init'](db)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
78 tx_Source_init(db)
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
79
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 status = Class(db, "status", name=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 status.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 priority = Class(db, "priority", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 priority.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 keyword = Class(db, "keyword", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 keyword.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 user = Class(db, "user", username=String(), password=Password(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 assignable=Boolean(), age=Number(), roles=String(), address=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 supervisor=Link('user'),realname=String(),alternate_addresses=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 user.setkey("username")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 file = FileClass(db, "file", name=String(), type=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 comment=String(indexme="yes"), fooz=Password())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 file_nidx = FileClass(db, "file_nidx", content=String(indexme='no'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 issue = IssueClass(db, "issue", title=String(indexme="yes"),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 status=Link("status"), nosy=Multilink("user"), deadline=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 foo=Interval(), files=Multilink("file"), assignedto=Link('user'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 priority=Link('priority'), spam=Multilink('msg'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 feedback=Link('msg'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 stuff = Class(db, "stuff", stuff=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 session = Class(db, 'session', title=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 msg = FileClass(db, "msg", date=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102 author=Link("user", do_journal='no'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103 files=Multilink('file'), inreplyto=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
104 messageid=String(), summary=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 content=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 recipients=Multilink("user", do_journal='no')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 )
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 if create:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 db.user.create(username="fred", roles='User',
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 password=password.Password('sekrit'), address='fred@example.com')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 db.security.addPermissionToRole('User', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 db.security.addPermission(name='Register', klass='user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 db.security.addPermissionToRole('User', 'Web Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 db.security.addPermissionToRole('Anonymous', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 db.security.addPermissionToRole('Anonymous', 'Register', 'user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 for cl in 'issue', 'file', 'msg', 'keyword':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 db.security.addPermissionToRole('User', 'Edit', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 db.security.addPermissionToRole('User', 'Create', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 for cl in 'priority', 'status':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 return db
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 class cldb(dict):
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
129 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
130 super(cldb, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
131 for key, value in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
132 super(cldb, self).__setitem__(s2b(key), value)
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
133 def __getitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
134 return super(cldb, self).__getitem__(s2b(key))
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
135 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
136 return super(cldb, self).__setitem__(s2b(key), value)
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
137 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
138 return super(cldb, self).__delitem__(s2b(key))
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
139 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
140 return super(cldb, self).__contains__(s2b(key))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 class BasicDatabase(dict):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 ''' Provide a nice encapsulation of an anydbm store.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 Keys are id strings, values are automatically marshalled data.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 '''
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
149 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
150 super(BasicDatabase, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
151 for k, v in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
152 super(BasicDatabase, self).__setitem__(s2b(k), v)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153 def __getitem__(self, key):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 if key not in self:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 d = self[key] = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156 return d
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
157 return super(BasicDatabase, self).__getitem__(s2b(key))
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
158 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
159 return super(BasicDatabase, self).__setitem__(s2b(key), value)
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
160 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
161 return super(BasicDatabase, self).__delitem__(s2b(key))
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
162 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
163 return super(BasicDatabase, self).__contains__(s2b(key))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164 def exists(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 return infoid in self
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
166 _marker = []
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
167 def get(self, infoid, value, default=_marker):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
168 if infoid not in self:
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
169 if default is self._marker:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
170 raise KeyError
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
171 else:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
172 return default
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 return self[infoid].get(value, default)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174 def getall(self, infoid):
4389
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
175 if infoid not in self:
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
176 raise KeyError(infoid)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
177 return self[infoid]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
178 def set(self, infoid, **newvalues):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
179 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
180 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
181 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
182 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
183 if infoid in self:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
184 del(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
185 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
186 newvalues['__timestamp'] = time.time()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
187 self[infoid].update(newvalues)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
188 def list(self):
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
189 return list(self.keys())
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 def destroy(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
191 del self[infoid]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 def commit(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
193 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 def updateTimestamp(self, sessid):
6828
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
197 sess = self.get(sessid, '__timestamp', None)
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
198 now = time.time()
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
199 if sess is None or now > sess + 60:
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
200 self.set(sessid, __timestamp=now)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201 def clean(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
204 class Sessions(BasicDatabase, sessions_dbm.Sessions):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 name = 'sessions'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206
6822
5053ee6c846b memorydb fixes for otks tests.
John Rouillard <rouilj@ieee.org>
parents: 6816
diff changeset
207 class OneTimeKeys(BasicDatabase, sessions_dbm.OneTimeKeys):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 name = 'otks'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
210 class Indexer(indexer_dbm.Indexer):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 def __init__(self, db):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 indexer_common.Indexer.__init__(self, db)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 self.reindex = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214 self.quiet = 9
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
216
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 def load_index(self, reload=0, wordlist=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 # Unless reload is indicated, do not load twice
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
219 if self.index_loaded() and not reload:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 return 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221 self.words = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222 self.files = {'_TOP':(0,None)}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223 self.fileids = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
224 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 def save_index(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 pass
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
228 def force_reindex(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
229 # TODO I'm concerned that force_reindex may not be tested by
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
230 # testForcedReindexing if the functionality can just be removed
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
231 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
233 class Database(back_anydbm.Database):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 """A database for storing records containing flexible data types.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236 Transaction stuff TODO:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 - check the timestamp of the class file and nuke the cache if it's
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 modified. Do some sort of conflict checking on the dirty stuff.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 - perhaps detect write collisions (related to above)?
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241 """
5317
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
242
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
243 dbtype = "memorydb"
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
244
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
245 # Make it a little more persistent across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
246 memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
247
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
248 def __init__(self, config, journaltag=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
249 self.config, self.journaltag = config, journaltag
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
250 self.classes = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251 self.files = {}
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
252 self.tx_files = {}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
253 self.security = security.Security(self)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
254 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
255 'filtering': 0}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
256 self.sessions = Sessions()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
257 self.otks = OneTimeKeys()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
258 self.indexer = Indexer(self)
6658
408fd477761f Add i18n object to roundupdb.Database
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6493
diff changeset
259 roundupdb.Database.__init__(self)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
260
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
261 # anydbm bits
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
262 self.cache = {} # cache of nodes loaded or created
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
263 self.dirtynodes = {} # keep track of the dirty nodes by class
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
264 self.newnodes = {} # keep track of the new nodes by class
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
265 self.destroyednodes = {}# keep track of the destroyed nodes by class
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
266 self.transactions = []
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
267 self.tx_Source = None
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
268 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
269 self.items = self.__class__.memdb.get('items', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
270 self.ids = self.__class__.memdb.get('ids', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
271 self.journals = self.__class__.memdb.get('journals', {})
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
272
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
273 def filename(self, classname, nodeid, property=None, create=0):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
274 shutil.copyfile(__file__, __file__+'.dummy')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
275 return __file__+'.dummy'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
276
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
277 def filesize(self, classname, nodeid, property=None, create=0):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
278 return len(self.getfile(classname, nodeid, property))
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
279
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
280 def post_init(self):
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5972
diff changeset
281 super(Database, self).post_init()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
282
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
283 def refresh_database(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
284 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
285
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
286 def getSessionManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
287 return self.sessions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
288
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
289 def getOTKManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
290 return self.otks
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
291
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
292 def reindex(self, classname=None, show_progress=False):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
293 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
294
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
295 def __repr__(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
296 return '<memorydb instance at %x>'%id(self)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
297
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
298 def storefile(self, classname, nodeid, property, content):
5465
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
299 if isinstance(content, str):
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
300 content = s2b(content)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
301 self.tx_files[classname, nodeid, property] = content
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
302 self.transactions.append((self.doStoreFile, (classname, nodeid,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
303 property)))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 def getfile(self, classname, nodeid, property):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
306 if (classname, nodeid, property) in self.tx_files:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
307 return self.tx_files[classname, nodeid, property]
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
308 return self.files[classname, nodeid, property]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
309
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
310 def doStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
311 self.files[classname, nodeid, property] = self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
312 return (classname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
313
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
314 def rollbackStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
315 del self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
316
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
317 def numfiles(self):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
318 return len(self.files) + len(self.tx_files)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
319
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
320 def close(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
321 self.clearCache()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
322 self.tx_files = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
323 # kill the schema too
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
324 self.classes = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
325 # just keep the .items
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
326 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
327 self.__class__.memdb['items'] = self.items
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
328 self.__class__.memdb['ids'] = self.ids
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
329 self.__class__.memdb['journals'] = self.journals
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
330
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
331 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
332 # Classes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
333 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
334 def __getattr__(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
335 """A convenient way of calling self.getclass(classname)."""
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
336 if classname in self.classes:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
337 return self.classes[classname]
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
338 raise AttributeError(classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
339
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
340 def addclass(self, cl):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
341 cn = cl.classname
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
342 if cn in self.classes:
6243
1e53e3ad9e5c Remove translation markings. Not needed.
John Rouillard <rouilj@ieee.org>
parents: 6241
diff changeset
343 raise ValueError('Class "%s" already defined.'%cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
344 self.classes[cn] = cl
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
345 if cn not in self.items:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
346 self.items[cn] = cldb()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
347 self.ids[cn] = 0
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
348
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
349 # add default Edit and View permissions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
350 self.security.addPermission(name="Create", klass=cn,
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
351 description="User is allowed to create "+cn)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
352 self.security.addPermission(name="Edit", klass=cn,
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
353 description="User is allowed to edit "+cn)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
354 self.security.addPermission(name="View", klass=cn,
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
355 description="User is allowed to access "+cn)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
356
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
357 def getclasses(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
358 """Return a list of the names of all existing classes."""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
359 return sorted(self.classes.keys())
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
360
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 def getclass(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362 """Get the Class object representing a particular class.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
363
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
364 If 'classname' is not a valid class name, a KeyError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
366 try:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367 return self.classes[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
368 except KeyError:
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
369 raise KeyError('There is no class called "%s"'%classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
371 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
372 # Class DBs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
373 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374 def clear(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
375 self.items = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
376
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
377 def getclassdb(self, classname, mode='r'):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378 """ grab a connection to the class db that will be used for
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 multiple actions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
381 return self.items[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
382
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
383 def getCachedJournalDB(self, classname):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
384 return self.journals.setdefault(classname, {})
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
385
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
386 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
387 # Node IDs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
388 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
389 def newid(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
390 self.ids[classname] += 1
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
391 return str(self.ids[classname])
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
392 def setid(self, classname, id):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
393 self.ids[classname] = int(id)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
394
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396 # Journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 #
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
398 def doSaveJournal(self, classname, nodeid, action, params, creator,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
399 creation):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400 if creator is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
401 creator = self.getuid()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
402 if creation is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 creation = date.Date()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404 self.journals.setdefault(classname, {}).setdefault(nodeid,
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
405 []).append((nodeid, creation, creator, action, params))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
406
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
407 def doSetJournal(self, classname, nodeid, journal):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
408 self.journals.setdefault(classname, {})[nodeid] = journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
409
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
410 def getjournal(self, classname, nodeid):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
411 # our journal result
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
412 res = []
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
413
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
414 # add any journal entries for transactions not committed to the
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
415 # database
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
416 for method, args in self.transactions:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
417 if method != self.doSaveJournal:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
418 continue
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
419 (cache_classname, cache_nodeid, cache_action, cache_params,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
420 cache_creator, cache_creation) = args
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
421 if cache_classname == classname and cache_nodeid == nodeid:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
422 if not cache_creator:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
423 cache_creator = self.getuid()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
424 if not cache_creation:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
425 cache_creation = date.Date()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
426 res.append((cache_nodeid, cache_creation, cache_creator,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
427 cache_action, cache_params))
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
428 try:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
429 res += self.journals.get(classname, {})[nodeid]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
430 except KeyError:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
431 if res: return res
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
432 raise IndexError(nodeid)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
433 return res
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
434
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
435 def pack(self, pack_before):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
436 """ Delete all journal entries except "create" before 'pack_before'.
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
437 """
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
438 pack_before = pack_before.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
439 for classname in self.journals:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
440 db = self.journals[classname]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
441 for key in db:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
442 # get the journal for this db entry
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
443 l = []
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
444 last_set_entry = None
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
445 for entry in db[key]:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
446 # unpack the entry
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
447 (nodeid, date_stamp, self.journaltag, action,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
448 params) = entry
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
449 date_stamp = date_stamp.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
450 # if the entry is after the pack date, _or_ the initial
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
451 # create entry, then it stays
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
452 if date_stamp > pack_before or action == 'create':
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
453 l.append(entry)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
454 db[key] = l
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
455
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
456 class Class(back_anydbm.Class):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
457 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
458
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
459 class FileClass(back_anydbm.FileClass):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
460 def __init__(self, db, classname, **properties):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
461 if 'content' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
462 properties['content'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
463 if 'type' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
464 properties['type'] = hyperdb.String()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
465 back_anydbm.Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
466
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
467 def export_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
468 dest = self.exportFilename(dirname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
469 ensureParentsExist(dest)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
470 f = open(dest, 'wb')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
471 f.write(self.db.files[self.classname, nodeid, None])
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
472 f.close()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
473
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
474 def import_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
475 source = self.exportFilename(dirname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
476 f = open(source, 'rb')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
477 self.db.files[self.classname, nodeid, None] = f.read()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
478 f.close()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
479 mime_type = None
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
480 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
481 if 'type' in props:
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
482 mime_type = self.get(nodeid, 'type')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
483 if not mime_type:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
484 mime_type = self.default_mime_type
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
485 if props['content'].indexme:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
486 self.db.indexer.add_text((self.classname, nodeid, 'content'),
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
487 self.get(nodeid, 'content'), mime_type)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
488
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
489 # deviation from spec - was called ItemClass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
490 class IssueClass(Class, roundupdb.IssueClass):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
491 # Overridden methods:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
492 def __init__(self, db, classname, **properties):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
493 """The newly-created class automatically includes the "messages",
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
494 "files", "nosy", and "superseder" properties. If the 'properties'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
495 dictionary attempts to specify any of these properties or a
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
496 "creation" or "activity" property, a ValueError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
497 """
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
498 if 'title' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
499 properties['title'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
500 if 'messages' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
501 properties['messages'] = hyperdb.Multilink("msg")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
502 if 'files' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
503 properties['files'] = hyperdb.Multilink("file")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
504 if 'nosy' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
505 # note: journalling is turned off as it really just wastes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
506 # space. this behaviour may be overridden in an instance
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
507 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
508 if 'superseder' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
509 properties['superseder'] = hyperdb.Multilink(classname)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
510 Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
511
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
512 # Methods to check for existence and nuke the db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
513 # We don't support multiple named databases
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
514
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
515 def db_exists(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
516 return bool(Database.memdb)
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
517
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
518 def db_nuke(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
519 Database.memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
520
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
521 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/