Mercurial > p > roundup > code

annotate roundup/test/memorydb.py @ 8166:53da2c697fab

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
update instruction on care and feeding for tracker.
author John Rouillard <rouilj@ieee.org>
date Wed, 20 Nov 2024 16:09:17 -0500
parents 506c86823abb
children 617d85ce4ac3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 '''Implement an in-memory hyperdb for testing purposes.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 import shutil
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
5 import os
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
6 import time
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
8 from roundup import date
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 from roundup import hyperdb
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 from roundup import roundupdb
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 from roundup import security
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 from roundup import password
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 from roundup import configuration
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 from roundup.backends import back_anydbm
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 from roundup.backends import indexer_dbm
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
16 from roundup.backends import sessions_dbm
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 from roundup.backends import indexer_common
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
18 from roundup.support import ensureParentsExist
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
19 from roundup.anypy.strings import s2b
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
20 from roundup.test.tx_Source_detector import init as tx_Source_init
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
22 default_prefix = '../../share/roundup/templates/classic'
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
23
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
24
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
25 def new_config(debug=False, prefix=default_prefix):
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
26 if not prefix.startswith('/'):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
27 prefix = os.path.join(os.path.dirname(__file__), prefix)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 config = configuration.CoreConfig()
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
29 config.detectors = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
30 os.path.join(prefix, "detectors/config.ini"))
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
31 config.ext = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
32 os.path.join(prefix, "extensions/config.ini"))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 config.DATABASE = "db"
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
34 # config.logging = MockNull()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
35 # these TRACKER_WEB and MAIL_DOMAIN values are used in mailgw tests
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
36 if debug:
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
37 config.LOGGING_LEVEL = "DEBUG"
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38 config.MAIL_DOMAIN = "your.tracker.email.domain.example"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 config.TRACKER_WEB = "http://tracker.example/cgi-bin/roundup.cgi/bugs/"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 return config
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
42
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
43 def create(journaltag, create=True, debug=False, prefix=default_prefix):
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
44 # "Nuke" in-memory db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
45 db_nuke('')
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
46
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
47 db = Database(new_config(debug), journaltag)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 # load standard schema
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
50 if not prefix.startswith('/'):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
51 prefix = os.path.join(os.path.dirname(__file__), prefix)
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
52
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
53 schema = os.path.join(prefix, 'schema.py')
5077
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
54 vars = hyperdb.__dict__
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
55 vars['Class'] = Class
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
56 vars['FileClass'] = FileClass
88fc7202f87d Remove 'import *' statement from test/memorydb.py
John Kristensen <john@jerrykan.com>
parents: 4795
diff changeset
57 vars['IssueClass'] = IssueClass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 vars['db'] = db
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
59 fd = open(schema)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
60 exec(compile(fd.read(), schema, 'exec'), vars)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
61 fd.close()
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
62
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
63 initial_data = os.path.join(prefix, 'initial_data.py')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 vars = dict(db=db, admin_email='admin@test.com',
7211
506c86823abb Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents: 6937
diff changeset
65 adminpw=password.Password('sekrit', config=db.config))
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
66 fd = open(initial_data)
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
67 exec(compile(fd.read(), initial_data, 'exec'), vars)
6493
a7072cf4ad7e Actually call close().
John Rouillard <rouilj@ieee.org>
parents: 6492
diff changeset
68 fd.close()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 # load standard detectors
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
71 dirname = os.path.join(prefix, 'detectors')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 for fn in os.listdir(dirname):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
73 if not fn.endswith('.py'): continue # noqa: E701
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 vars = {}
6491
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
75 with open(os.path.join(dirname, fn)) as fd:
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
76 exec(compile(fd.read(),
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
77 os.path.join(dirname, fn), 'exec'), vars)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 vars['init'](db)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
80 tx_Source_init(db)
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
81
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 status = Class(db, "status", name=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 status.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 priority = Class(db, "priority", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 priority.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 keyword = Class(db, "keyword", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 keyword.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 user = Class(db, "user", username=String(), password=Password(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 assignable=Boolean(), age=Number(), roles=String(), address=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 supervisor=Link('user'),realname=String(),alternate_addresses=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 user.setkey("username")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 file = FileClass(db, "file", name=String(), type=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 comment=String(indexme="yes"), fooz=Password())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 file_nidx = FileClass(db, "file_nidx", content=String(indexme='no'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 issue = IssueClass(db, "issue", title=String(indexme="yes"),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 status=Link("status"), nosy=Multilink("user"), deadline=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 foo=Interval(), files=Multilink("file"), assignedto=Link('user'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 priority=Link('priority'), spam=Multilink('msg'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 feedback=Link('msg'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 stuff = Class(db, "stuff", stuff=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102 session = Class(db, 'session', title=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103 msg = FileClass(db, "msg", date=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
104 author=Link("user", do_journal='no'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 files=Multilink('file'), inreplyto=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 messageid=String(), summary=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 content=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 recipients=Multilink("user", do_journal='no')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 )
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 if create:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 db.user.create(username="fred", roles='User',
7211
506c86823abb Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents: 6937
diff changeset
113 password=password.Password('sekrit',
506c86823abb Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents: 6937
diff changeset
114 config=db.config),
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
115 address='fred@example.com')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 db.security.addPermissionToRole('User', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 db.security.addPermission(name='Register', klass='user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 db.security.addPermissionToRole('User', 'Web Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 db.security.addPermissionToRole('Anonymous', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 db.security.addPermissionToRole('Anonymous', 'Register', 'user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 for cl in 'issue', 'file', 'msg', 'keyword':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 db.security.addPermissionToRole('User', 'Edit', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 db.security.addPermissionToRole('User', 'Create', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 for cl in 'priority', 'status':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 return db
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
132
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 class cldb(dict):
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
134 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
135 super(cldb, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
136 for key, value in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
137 super(cldb, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
138
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
139 def __getitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
140 return super(cldb, self).__getitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
141
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
142 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
143 return super(cldb, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
144
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
145 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
146 return super(cldb, self).__delitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
147
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
148 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
149 return super(cldb, self).__contains__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
150
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
151 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
152 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
154
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 class BasicDatabase(dict):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156 ''' Provide a nice encapsulation of an anydbm store.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 Keys are id strings, values are automatically marshalled data.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 '''
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
160 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
161 super(BasicDatabase, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
162 for k, v in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
163 super(BasicDatabase, self).__setitem__(s2b(k), v)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
164
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 def __getitem__(self, key):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
166 if key not in self:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
167 d = self[key] = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
168 return d
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
169 return super(BasicDatabase, self).__getitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
170
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
171 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
172 return super(BasicDatabase, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
173
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
174 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
175 return super(BasicDatabase, self).__delitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
176
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
177 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
178 return super(BasicDatabase, self).__contains__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
179
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
180 def exists(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 return infoid in self
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
182 _marker = []
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
183
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
184 def get(self, infoid, value, default=_marker):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
185 if infoid not in self:
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
186 if default is self._marker:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
187 raise KeyError
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
188 else:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
189 return default
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 return self[infoid].get(value, default)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
191
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 def getall(self, infoid):
4389
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
193 if infoid not in self:
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
194 raise KeyError(infoid)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 return self[infoid]
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
196
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 def set(self, infoid, **newvalues):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
198 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
199 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
200 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
201 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
202 if infoid in self:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
203 del(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
204 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
205 newvalues['__timestamp'] = time.time()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 self[infoid].update(newvalues)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
207
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 def list(self):
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
209 return list(self.keys())
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
210
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 def destroy(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 del self[infoid]
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
213
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214 def commit(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
216
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
219
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 def updateTimestamp(self, sessid):
6828
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
221 sess = self.get(sessid, '__timestamp', None)
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
222 now = time.time()
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
223 if sess is None or now > sess + 60:
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
224 self.set(sessid, __timestamp=now)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
225
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 def clean(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
228
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
229
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
230 class Sessions(BasicDatabase, sessions_dbm.Sessions):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 name = 'sessions'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
233
6822
5053ee6c846b memorydb fixes for otks tests.
John Rouillard <rouilj@ieee.org>
parents: 6816
diff changeset
234 class OneTimeKeys(BasicDatabase, sessions_dbm.OneTimeKeys):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235 name = 'otks'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
237
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 class Indexer(indexer_dbm.Indexer):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 def __init__(self, db):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 indexer_common.Indexer.__init__(self, db)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241 self.reindex = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242 self.quiet = 9
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
243 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
244
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
245 def load_index(self, reload=0, wordlist=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246 # Unless reload is indicated, do not load twice
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
247 if self.index_loaded() and not reload:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
248 return 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
249 self.words = {}
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
250 self.files = {'_TOP': (0, None)}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251 self.fileids = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
252 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
253
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
254 def save_index(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
255 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
256
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
257 def force_reindex(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
258 # TODO I'm concerned that force_reindex may not be tested by
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
259 # testForcedReindexing if the functionality can just be removed
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
260 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
261
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
262
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
263 class Database(back_anydbm.Database):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
264 """A database for storing records containing flexible data types.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
265
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
266 Transaction stuff TODO:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
267
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
268 - check the timestamp of the class file and nuke the cache if it's
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269 modified. Do some sort of conflict checking on the dirty stuff.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270 - perhaps detect write collisions (related to above)?
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 """
5317
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
272
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
273 dbtype = "memorydb"
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
274
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
275 # Make it a little more persistent across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
276 memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
277
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
278 def __init__(self, config, journaltag=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
279 self.config, self.journaltag = config, journaltag
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
280 self.classes = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
281 self.files = {}
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
282 self.tx_files = {}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
283 self.security = security.Security(self)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
284 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
285 'filtering': 0}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
286 self.sessions = Sessions()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
287 self.otks = OneTimeKeys()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
288 self.indexer = Indexer(self)
6658
408fd477761f Add i18n object to roundupdb.Database
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6493
diff changeset
289 roundupdb.Database.__init__(self)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
290
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
291 # anydbm bits
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
292 self.cache = {} # cache of nodes loaded or created
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
293 self.dirtynodes = {} # keep track of the dirty nodes by class
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
294 self.newnodes = {} # keep track of the new nodes by class
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
295 self.destroyednodes = {} # keep track of the destroyed nodes by class
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
296 self.transactions = []
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
297 self.tx_Source = None
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
298 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
299 self.items = self.__class__.memdb.get('items', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
300 self.ids = self.__class__.memdb.get('ids', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
301 self.journals = self.__class__.memdb.get('journals', {})
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
302
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
303 def filename(self, classname, nodeid, property=None, create=0):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304 shutil.copyfile(__file__, __file__+'.dummy')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 return __file__+'.dummy'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
306
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
307 def filesize(self, classname, nodeid, property=None, create=0):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
308 return len(self.getfile(classname, nodeid, property))
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
309
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
310 def post_init(self):
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5972
diff changeset
311 super(Database, self).post_init()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
312
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
313 def refresh_database(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
315
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
316 def getSessionManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
317 return self.sessions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
318
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
319 def getOTKManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
320 return self.otks
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
321
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
322 def reindex(self, classname=None, show_progress=False):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
323 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
324
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325 def __repr__(self):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
326 return '<memorydb instance at %x>' % id(self)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
327
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
328 def storefile(self, classname, nodeid, property, content):
5465
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
329 if isinstance(content, str):
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
330 content = s2b(content)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
331 self.tx_files[classname, nodeid, property] = content
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
332 self.transactions.append((self.doStoreFile, (classname, nodeid,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
333 property)))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
334
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
335 def getfile(self, classname, nodeid, property):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
336 if (classname, nodeid, property) in self.tx_files:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
337 return self.tx_files[classname, nodeid, property]
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
338 return self.files[classname, nodeid, property]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
339
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
340 def doStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
341 self.files[classname, nodeid, property] = self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
342 return (classname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
343
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
344 def rollbackStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
345 del self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
346
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
347 def numfiles(self):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
348 return len(self.files) + len(self.tx_files)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
350 def close(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
351 self.clearCache()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
352 self.tx_files = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
353 # kill the schema too
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
354 self.classes = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
355 # just keep the .items
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
356 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
357 self.__class__.memdb['items'] = self.items
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
358 self.__class__.memdb['ids'] = self.ids
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
359 self.__class__.memdb['journals'] = self.journals
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
360
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362 # Classes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
363 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
364 def __getattr__(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 """A convenient way of calling self.getclass(classname)."""
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
366 if classname in self.classes:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367 return self.classes[classname]
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
368 raise AttributeError(classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370 def addclass(self, cl):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
371 cn = cl.classname
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
372 if cn in self.classes:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
373 raise ValueError('Class "%s" already defined.' % cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374 self.classes[cn] = cl
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
375 if cn not in self.items:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
376 self.items[cn] = cldb()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
377 self.ids[cn] = 0
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 # add default Edit and View permissions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 self.security.addPermission(name="Create", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
381 description="User is allowed to create " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
382 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
383 self.security.addPermission(name="Edit", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
384 description="User is allowed to edit " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
385 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
386 self.security.addPermission(name="View", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
387 description="User is allowed to access " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
388 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
389
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
390 def getclasses(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
391 """Return a list of the names of all existing classes."""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
392 return sorted(self.classes.keys())
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
393
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
394 def getclass(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 """Get the Class object representing a particular class.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 If 'classname' is not a valid class name, a KeyError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
398 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
399 try:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400 return self.classes[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
401 except KeyError:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
402 raise KeyError('There is no class called "%s"' % classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
405 # Class DBs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
406 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
407 def clear(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
408 self.items = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
409
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
410 def getclassdb(self, classname, mode='r'):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
411 """ grab a connection to the class db that will be used for
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
412 multiple actions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
413 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
414 return self.items[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
415
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
416 def getCachedJournalDB(self, classname):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
417 return self.journals.setdefault(classname, {})
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
418
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
419 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
420 # Node IDs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
421 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
422 def newid(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
423 self.ids[classname] += 1
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
424 return str(self.ids[classname])
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
425
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
426 def setid(self, classname, id):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
427 self.ids[classname] = int(id)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
428
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
429 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
430 # Journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
431 #
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
432 def doSaveJournal(self, classname, nodeid, action, params, creator,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
433 creation):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
434 if creator is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
435 creator = self.getuid()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
436 if creation is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
437 creation = date.Date()
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
438 self.journals.setdefault(classname, {}).setdefault(
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
439 nodeid, []).append((nodeid, creation, creator, action, params))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
440
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
441 def doSetJournal(self, classname, nodeid, journal):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
442 self.journals.setdefault(classname, {})[nodeid] = journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
443
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
444 def getjournal(self, classname, nodeid):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
445 # our journal result
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
446 res = []
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
447
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
448 # add any journal entries for transactions not committed to the
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
449 # database
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
450 for method, args in self.transactions:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
451 if method != self.doSaveJournal:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
452 continue
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
453 (cache_classname, cache_nodeid, cache_action, cache_params,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
454 cache_creator, cache_creation) = args
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
455 if cache_classname == classname and cache_nodeid == nodeid:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
456 if not cache_creator:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
457 cache_creator = self.getuid()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
458 if not cache_creation:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
459 cache_creation = date.Date()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
460 res.append((cache_nodeid, cache_creation, cache_creator,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
461 cache_action, cache_params))
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
462 try:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
463 res += self.journals.get(classname, {})[nodeid]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
464 except KeyError:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
465 if res: return res # noqa: E701
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
466 raise IndexError(nodeid)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
467 return res
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
468
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
469 def pack(self, pack_before):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
470 """ Delete all journal entries except "create" before 'pack_before'.
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
471 """
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
472 pack_before = pack_before.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
473 for classname in self.journals:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
474 db = self.journals[classname]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
475 for key in db:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
476 # get the journal for this db entry
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
477 kept_journals = []
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
478 for entry in db[key]:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
479 # unpack the entry
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
480 (nodeid, date_stamp, self.journaltag, action,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
481 params) = entry
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
482 date_stamp = date_stamp.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
483 # if the entry is after the pack date, _or_ the initial
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
484 # create entry, then it stays
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
485 if date_stamp > pack_before or action == 'create':
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
486 kept_journals.append(entry)
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
487 db[key] = kept_journals
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
488
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
489
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
490 class Class(back_anydbm.Class):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
491 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
492
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
493
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
494 class FileClass(back_anydbm.FileClass):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
495 def __init__(self, db, classname, **properties):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
496 if 'content' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
497 properties['content'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
498 if 'type' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
499 properties['type'] = hyperdb.String()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
500 back_anydbm.Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
501
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
502 def export_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
503 dest = self.exportFilename(dirname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
504 ensureParentsExist(dest)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
505 f = open(dest, 'wb')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
506 f.write(self.db.files[self.classname, nodeid, None])
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
507 f.close()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
508
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
509 def import_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
510 source = self.exportFilename(dirname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
511 f = open(source, 'rb')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
512 self.db.files[self.classname, nodeid, None] = f.read()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
513 f.close()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
514 mime_type = None
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
515 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
516 if 'type' in props:
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
517 mime_type = self.get(nodeid, 'type')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
518 if not mime_type:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
519 mime_type = self.default_mime_type
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
520 if props['content'].indexme:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
521 self.db.indexer.add_text((self.classname, nodeid, 'content'),
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
522 self.get(nodeid, 'content'), mime_type)
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
523
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
524
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
525 # deviation from spec - was called ItemClass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
526 class IssueClass(Class, roundupdb.IssueClass):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
527 # Overridden methods:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
528 def __init__(self, db, classname, **properties):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
529 """The newly-created class automatically includes the "messages",
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
530 "files", "nosy", and "superseder" properties. If the 'properties'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
531 dictionary attempts to specify any of these properties or a
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
532 "creation" or "activity" property, a ValueError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
533 """
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
534 if 'title' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
535 properties['title'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
536 if 'messages' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
537 properties['messages'] = hyperdb.Multilink("msg")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
538 if 'files' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
539 properties['files'] = hyperdb.Multilink("file")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
540 if 'nosy' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
541 # note: journalling is turned off as it really just wastes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
542 # space. this behaviour may be overridden in an instance
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
543 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
544 if 'superseder' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
545 properties['superseder'] = hyperdb.Multilink(classname)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
546 Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
547
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
548 # Methods to check for existence and nuke the db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
549 # We don't support multiple named databases
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
550
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
551
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
552 def db_exists(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
553 return bool(Database.memdb)
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
554
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
555
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
556 def db_nuke(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
557 Database.memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
558
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
559 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/