Mercurial > p > roundup > code

annotate roundup/test/memorydb.py @ 8478:ed4ef394d5d6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc: initial attempt to document setup of pgp support for email. Used an AI assistant to help write this. Basic gpg commands seem to work, but I have not tested this totally. Docs basically follow the setup used for pgp testing in the test suite. It looks like roundup accepts signed emails as well as encrypted and signed emails. But it does not generate signed emails. Also it looks like there is no PGP support for alternate email addresses. Only primary addresses can do PGP emails.
author John Rouillard <rouilj@ieee.org>
date Sat, 15 Nov 2025 16:59:24 -0500
parents 617d85ce4ac3
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 '''Implement an in-memory hyperdb for testing purposes.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
4 import os
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 import shutil
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
6 import time
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
8 from roundup import configuration, date, hyperdb, password, roundupdb, security
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
9 from roundup.anypy.strings import s2b
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
10 from roundup.backends import back_anydbm, indexer_common, indexer_dbm, sessions_dbm
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
11 from roundup.support import ensureParentsExist
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
12 from roundup.test.tx_Source_detector import init as tx_Source_init
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
14 default_prefix = '../../share/roundup/templates/classic'
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
15
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
16
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
17 def new_config(debug=False, prefix=default_prefix):
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
18 if not prefix.startswith('/'):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
19 prefix = os.path.join(os.path.dirname(__file__), prefix)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 config = configuration.CoreConfig()
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
21 config.detectors = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
22 os.path.join(prefix, "detectors/config.ini"))
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
23 config.ext = configuration.UserConfig(
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
24 os.path.join(prefix, "extensions/config.ini"))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25 config.DATABASE = "db"
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
26 # config.logging = MockNull()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 # these TRACKER_WEB and MAIL_DOMAIN values are used in mailgw tests
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
28 if debug:
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
29 config.LOGGING_LEVEL = "DEBUG"
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30 config.MAIL_DOMAIN = "your.tracker.email.domain.example"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31 config.TRACKER_WEB = "http://tracker.example/cgi-bin/roundup.cgi/bugs/"
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 return config
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
34
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
35 def create(journaltag, create=True, debug=False, prefix=default_prefix):
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
36 # "Nuke" in-memory db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
37 db_nuke('')
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
38
4358
887a153cedec The email for the first message on an issue...
Richard Jones <richard@users.sourceforge.net>
parents: 4350
diff changeset
39 db = Database(new_config(debug), journaltag)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 # load standard schema
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
42 if not prefix.startswith('/'):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
43 prefix = os.path.join(os.path.dirname(__file__), prefix)
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
44
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
45 schema = os.path.join(prefix, 'schema.py')
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
46 hyperdb_vars = hyperdb.__dict__
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
47 hyperdb_vars['Class'] = Class
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
48 hyperdb_vars['FileClass'] = FileClass
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
49 hyperdb_vars['IssueClass'] = IssueClass
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
50 hyperdb_vars['db'] = db
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
51
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
52 with open(schema) as fd:
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
53 exec(compile(fd.read(), schema, 'exec'), hyperdb_vars)
6492
4351c95faace Fix more opens that aren't closed.
John Rouillard <rouilj@ieee.org>
parents: 6491
diff changeset
54
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
55 initial_data = os.path.join(prefix, 'initial_data.py')
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
56 admin_vars = {"db": db, "admin_email": "admin@test.com",
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
57 "adminpw": password.Password('sekrit', config=db.config)}
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
58 with open(initial_data) as fd:
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
59 exec(compile(fd.read(), initial_data, 'exec'), admin_vars)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61 # load standard detectors
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
62 dirname = os.path.join(prefix, 'detectors')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 for fn in os.listdir(dirname):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
64 if not fn.endswith('.py'): continue # noqa: E701
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
65 exec_vars = {}
6491
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
66 with open(os.path.join(dirname, fn)) as fd:
087cae2fbcea Handle more ResourceWarning issues.
John Rouillard <rouilj@ieee.org>
parents: 6365
diff changeset
67 exec(compile(fd.read(),
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
68 os.path.join(dirname, fn), 'exec'), exec_vars)
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
69 exec_vars['init'](db)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70
6361
58817c3bf471 Fix roundup/test
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6360
diff changeset
71 tx_Source_init(db)
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
72
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 status = Class(db, "status", name=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 status.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 priority = Class(db, "priority", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 priority.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 keyword = Class(db, "keyword", name=String(), order=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 keyword.setkey("name")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 user = Class(db, "user", username=String(), password=Password(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 assignable=Boolean(), age=Number(), roles=String(), address=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 supervisor=Link('user'),realname=String(),alternate_addresses=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 user.setkey("username")
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 file = FileClass(db, "file", name=String(), type=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 comment=String(indexme="yes"), fooz=Password())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 file_nidx = FileClass(db, "file_nidx", content=String(indexme='no'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 issue = IssueClass(db, "issue", title=String(indexme="yes"),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 status=Link("status"), nosy=Multilink("user"), deadline=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 foo=Interval(), files=Multilink("file"), assignedto=Link('user'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 priority=Link('priority'), spam=Multilink('msg'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 feedback=Link('msg'))
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 stuff = Class(db, "stuff", stuff=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 session = Class(db, 'session', title=String())
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 msg = FileClass(db, "msg", date=Date(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 author=Link("user", do_journal='no'),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 files=Multilink('file'), inreplyto=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 messageid=String(), summary=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 content=String(),
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 recipients=Multilink("user", do_journal='no')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 )
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102 if create:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103 db.user.create(username="fred", roles='User',
7211
506c86823abb Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents: 6937
diff changeset
104 password=password.Password('sekrit',
506c86823abb Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents: 6937
diff changeset
105 config=db.config),
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
106 address='fred@example.com')
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108 db.security.addPermissionToRole('User', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 db.security.addPermission(name='Register', klass='user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 db.security.addPermissionToRole('User', 'Web Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 db.security.addPermissionToRole('Anonymous', 'Email Access')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 db.security.addPermissionToRole('Anonymous', 'Register', 'user')
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 for cl in 'issue', 'file', 'msg', 'keyword':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 db.security.addPermissionToRole('User', 'Edit', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 db.security.addPermissionToRole('User', 'Create', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 for cl in 'priority', 'status':
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 db.security.addPermissionToRole('User', 'View', cl)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 '''
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 return db
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
123
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 class cldb(dict):
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
125 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
126 super(cldb, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
127 for key, value in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
128 super(cldb, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
129
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
130 def __getitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
131 return super(cldb, self).__getitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
132
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
133 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
134 return super(cldb, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
135
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
136 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
137 return super(cldb, self).__delitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
138
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
139 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
140 return super(cldb, self).__contains__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
141
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
145
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 class BasicDatabase(dict):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 ''' Provide a nice encapsulation of an anydbm store.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 Keys are id strings, values are automatically marshalled data.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
150 '''
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
151 def __init__(self, **values):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
152 super(BasicDatabase, self).__init__()
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
153 for k, v in values.items():
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
154 super(BasicDatabase, self).__setitem__(s2b(k), v)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
155
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156 def __getitem__(self, key):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157 if key not in self:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 d = self[key] = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 return d
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
160 return super(BasicDatabase, self).__getitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
161
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
162 def __setitem__(self, key, value):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
163 return super(BasicDatabase, self).__setitem__(s2b(key), value)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
164
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
165 def __delitem__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
166 return super(BasicDatabase, self).__delitem__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
167
5462
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
168 def __contains__(self, key):
d0fd8bb64c3f fixed memorydb string encoding
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5395
diff changeset
169 return super(BasicDatabase, self).__contains__(s2b(key))
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
170
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171 def exists(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
172 return infoid in self
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
173 _marker = []
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
174
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
175 def get(self, infoid, value, default=_marker):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
176 if infoid not in self:
6816
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
177 if default is self._marker:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
178 raise KeyError
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
179 else:
a4652d809ec5 implement default argument return in BasicDatabase.get()
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
180 return default
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 return self[infoid].get(value, default)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
182
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183 def getall(self, infoid):
4389
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
184 if infoid not in self:
355cd8fdeb32 memorydb API consistency fix
Richard Jones <richard@users.sourceforge.net>
parents: 4358
diff changeset
185 raise KeyError(infoid)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
186 return self[infoid]
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
187
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
188 def set(self, infoid, **newvalues):
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
189 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
190 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
191 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
192 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
193 if infoid in self:
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
194 del (newvalues['__timestamp'])
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
195 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6658
diff changeset
196 newvalues['__timestamp'] = time.time()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 self[infoid].update(newvalues)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
198
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 def list(self):
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
200 return list(self.keys())
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
201
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202 def destroy(self, infoid):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203 del self[infoid]
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
204
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 def commit(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
207
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 def close(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
210
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 def updateTimestamp(self, sessid):
6828
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
212 sess = self.get(sessid, '__timestamp', None)
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
213 now = time.time()
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
214 if sess is None or now > sess + 60:
2ce855803633 Implement UpdateTimestamp for memorydb and test.
John Rouillard <rouilj@ieee.org>
parents: 6822
diff changeset
215 self.set(sessid, __timestamp=now)
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
216
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 def clean(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
219
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
220
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
221 class Sessions(BasicDatabase, sessions_dbm.Sessions):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222 name = 'sessions'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
224
6822
5053ee6c846b memorydb fixes for otks tests.
John Rouillard <rouilj@ieee.org>
parents: 6816
diff changeset
225 class OneTimeKeys(BasicDatabase, sessions_dbm.OneTimeKeys):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 name = 'otks'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
228
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
229 class Indexer(indexer_dbm.Indexer):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230 def __init__(self, db):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 indexer_common.Indexer.__init__(self, db)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232 self.reindex = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
233 self.quiet = 9
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236 def load_index(self, reload=0, wordlist=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237 # Unless reload is indicated, do not load twice
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 if self.index_loaded() and not reload:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 return 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 self.words = {}
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
241 self.files = {'_TOP': (0, None)}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242 self.fileids = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
243 self.changed = 0
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
244
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
245 def save_index(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246 pass
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
247
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
248 def force_reindex(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
249 # TODO I'm concerned that force_reindex may not be tested by
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
250 # testForcedReindexing if the functionality can just be removed
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
251 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
252
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
253
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
254 class Database(back_anydbm.Database):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
255 """A database for storing records containing flexible data types.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
256
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
257 Transaction stuff TODO:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
258
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
259 - check the timestamp of the class file and nuke the cache if it's
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
260 modified. Do some sort of conflict checking on the dirty stuff.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
261 - perhaps detect write collisions (related to above)?
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
262 """
5317
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
263
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
264 dbtype = "memorydb"
c0cda00af479 Fix tests for memorydb and anydbm backends
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5077
diff changeset
265
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
266 # Make it a little more persistent across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
267 memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
268
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269 def __init__(self, config, journaltag=None):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270 self.config, self.journaltag = config, journaltag
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 self.classes = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
272 self.files = {}
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
273 self.tx_files = {}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
274 self.security = security.Security(self)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
275 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
276 'filtering': 0}
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
277 self.sessions = Sessions()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
278 self.otks = OneTimeKeys()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
279 self.indexer = Indexer(self)
6658
408fd477761f Add i18n object to roundupdb.Database
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6493
diff changeset
280 roundupdb.Database.__init__(self)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
281
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
282 # anydbm bits
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
283 self.cache = {} # cache of nodes loaded or created
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
284 self.dirtynodes = {} # keep track of the dirty nodes by class
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
285 self.newnodes = {} # keep track of the new nodes by class
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
286 self.destroyednodes = {} # keep track of the destroyed nodes by class
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
287 self.transactions = []
4781
6e9b9743de89 Implementation for:
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
288 self.tx_Source = None
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
289 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
290 self.items = self.__class__.memdb.get('items', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
291 self.ids = self.__class__.memdb.get('ids', {})
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
292 self.journals = self.__class__.memdb.get('journals', {})
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
293
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
294 def filename(self, classname, nodeid, property=None, create=0):
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
295 shutil.copyfile(__file__, __file__ + '.dummy')
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
296 return __file__ + '.dummy'
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
297
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
298 def filesize(self, classname, nodeid, property=None, create=0):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
299 return len(self.getfile(classname, nodeid, property))
4347
0e33bf5571dc make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents: 4346
diff changeset
300
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
301 def post_init(self):
6148
8497bf3f23a1 Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5972
diff changeset
302 super(Database, self).post_init()
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
303
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304 def refresh_database(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
306
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
307 def getSessionManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
308 return self.sessions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
309
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
310 def getOTKManager(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
311 return self.otks
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
312
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
313 def reindex(self, classname=None, show_progress=False):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314 pass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
315
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
316 def __repr__(self):
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
317 return '<memorydb instance at %x>' % id(self)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
318
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
319 def storefile(self, classname, nodeid, property, content):
5465
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
320 if isinstance(content, str):
c4f9a152258d keep test code in sync with real implementation
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5462
diff changeset
321 content = s2b(content)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
322 self.tx_files[classname, nodeid, property] = content
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
323 self.transactions.append((self.doStoreFile, (classname, nodeid,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
324 property)))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
326 def getfile(self, classname, nodeid, property):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
327 if (classname, nodeid, property) in self.tx_files:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
328 return self.tx_files[classname, nodeid, property]
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
329 return self.files[classname, nodeid, property]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
330
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
331 def doStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
332 self.files[classname, nodeid, property] = self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
333 return (classname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
334
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
335 def rollbackStoreFile(self, classname, nodeid, property, **databases):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
336 del self.tx_files[classname, nodeid, property]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
337
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
338 def numfiles(self):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
339 return len(self.files) + len(self.tx_files)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
340
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
341 def close(self):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
342 self.clearCache()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
343 self.tx_files = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
344 # kill the schema too
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
345 self.classes = {}
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
346 # just keep the .items
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
347 # persistence across re-open
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
348 self.__class__.memdb['items'] = self.items
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
349 self.__class__.memdb['ids'] = self.ids
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
350 self.__class__.memdb['journals'] = self.journals
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
351
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
352 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
353 # Classes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
354 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
355 def __getattr__(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
356 """A convenient way of calling self.getclass(classname)."""
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
357 if classname in self.classes:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
358 return self.classes[classname]
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
359 raise AttributeError(classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
360
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 def addclass(self, cl):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362 cn = cl.classname
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
363 if cn in self.classes:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
364 raise ValueError('Class "%s" already defined.' % cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 self.classes[cn] = cl
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
366 if cn not in self.items:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
367 self.items[cn] = cldb()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
368 self.ids[cn] = 0
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370 # add default Edit and View permissions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
371 self.security.addPermission(name="Create", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
372 description="User is allowed to create " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
373 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374 self.security.addPermission(name="Edit", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
375 description="User is allowed to edit " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
376 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
377 self.security.addPermission(name="View", klass=cn,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
378 description="User is allowed to access " +
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
379 cn)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
381 def getclasses(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
382 """Return a list of the names of all existing classes."""
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5385
diff changeset
383 return sorted(self.classes.keys())
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
384
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
385 def getclass(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
386 """Get the Class object representing a particular class.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
387
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
388 If 'classname' is not a valid class name, a KeyError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
389 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
390 try:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
391 return self.classes[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
392 except KeyError:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
393 raise KeyError('There is no class called "%s"' % classname)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
394
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396 # Class DBs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
398 def clear(self):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
399 self.items = {}
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
401 def getclassdb(self, classname, mode='r'):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
402 """ grab a connection to the class db that will be used for
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 multiple actions
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404 """
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
405 return self.items[classname]
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
406
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
407 def getCachedJournalDB(self, classname):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
408 return self.journals.setdefault(classname, {})
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
409
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
410 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
411 # Node IDs
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
412 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
413 def newid(self, classname):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
414 self.ids[classname] += 1
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
415 return str(self.ids[classname])
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
416
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
417 def setid(self, classname, nodeid):
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
418 self.ids[classname] = int(nodeid)
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
419
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
420 #
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
421 # Journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
422 #
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
423 def doSaveJournal(self, classname, nodeid, action, params, creator,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
424 creation):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
425 if creator is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
426 creator = self.getuid()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
427 if creation is None:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
428 creation = date.Date()
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
429 self.journals.setdefault(classname, {}).setdefault(
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
430 nodeid, []).append((nodeid, creation, creator, action, params))
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
431
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
432 def doSetJournal(self, classname, nodeid, journal):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
433 self.journals.setdefault(classname, {})[nodeid] = journal
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
434
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
435 def getjournal(self, classname, nodeid):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
436 # our journal result
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
437 res = []
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
438
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
439 # add any journal entries for transactions not committed to the
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
440 # database
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
441 for method, args in self.transactions:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
442 if method != self.doSaveJournal:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
443 continue
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
444 (cache_classname, cache_nodeid, cache_action, cache_params,
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
445 cache_creator, cache_creation) = args
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
446 if cache_classname == classname and cache_nodeid == nodeid:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
447 if not cache_creator:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
448 cache_creator = self.getuid()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
449 if not cache_creation:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
450 cache_creation = date.Date()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
451 res.append((cache_nodeid, cache_creation, cache_creator,
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
452 cache_action, cache_params))
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
453 try:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
454 res += self.journals.get(classname, {})[nodeid]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
455 except KeyError:
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
456 if res: return res # noqa: E701
5378
35ea9b1efc14 Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5317
diff changeset
457 raise IndexError(nodeid)
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
458 return res
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
459
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
460 def pack(self, pack_before):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
461 """ Delete all journal entries except "create" before 'pack_before'.
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
462 """
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
463 pack_before = pack_before.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
464 for classname in self.journals:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
465 db = self.journals[classname]
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
466 for key in db:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
467 # get the journal for this db entry
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
468 kept_journals = []
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
469 for entry in db[key]:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
470 # unpack the entry
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
471 (_nodeid, date_stamp, self.journaltag, action,
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
472 _params) = entry
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
473 date_stamp = date_stamp.serialise()
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
474 # if the entry is after the pack date, _or_ the initial
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
475 # create entry, then it stays
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
476 if date_stamp > pack_before or action == 'create':
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
477 kept_journals.append(entry)
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
478 db[key] = kept_journals
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
479
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
480
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
481 class Class(back_anydbm.Class):
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
482 pass
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
483
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
484
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
485 class FileClass(back_anydbm.FileClass):
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
486 def __init__(self, db, classname, **properties):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
487 if 'content' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
488 properties['content'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
489 if 'type' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
490 properties['type'] = hyperdb.String()
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
491 back_anydbm.Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
492
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
493 def export_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
494 dest = self.exportFilename(dirname, nodeid)
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
495 ensureParentsExist(dest)
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
496 with open(dest, 'wb') as f:
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
497 f.write(self.db.files[self.classname, nodeid, None])
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
498
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
499 def import_files(self, dirname, nodeid):
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
500 source = self.exportFilename(dirname, nodeid)
8260
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
501 with open(source, 'rb') as f:
617d85ce4ac3 chore(ruff): variable renames, formatting, sort imports, use with open
John Rouillard <rouilj@ieee.org>
parents: 7211
diff changeset
502 self.db.files[self.classname, nodeid, None] = f.read()
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
503 mime_type = None
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
504 props = self.getprops()
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
505 if 'type' in props:
4349
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
506 mime_type = self.get(nodeid, 'type')
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
507 if not mime_type:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
508 mime_type = self.default_mime_type
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
509 if props['content'].indexme:
f0faef4dd023 Make memorydb pass all tests;
Richard Jones <richard@users.sourceforge.net>
parents: 4347
diff changeset
510 self.db.indexer.add_text((self.classname, nodeid, 'content'),
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
511 self.get(nodeid, 'content'), mime_type)
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
512
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
513
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
514 # deviation from spec - was called ItemClass
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
515 class IssueClass(Class, roundupdb.IssueClass):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
516 # Overridden methods:
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
517 def __init__(self, db, classname, **properties):
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
518 """The newly-created class automatically includes the "messages",
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
519 "files", "nosy", and "superseder" properties. If the 'properties'
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
520 dictionary attempts to specify any of these properties or a
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
521 "creation" or "activity" property, a ValueError is raised.
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
522 """
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
523 if 'title' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
524 properties['title'] = hyperdb.String(indexme='yes')
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
525 if 'messages' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
526 properties['messages'] = hyperdb.Multilink("msg")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
527 if 'files' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
528 properties['files'] = hyperdb.Multilink("file")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
529 if 'nosy' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
530 # note: journalling is turned off as it really just wastes
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
531 # space. this behaviour may be overridden in an instance
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
532 properties['nosy'] = hyperdb.Multilink("user", do_journal="no")
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5378
diff changeset
533 if 'superseder' not in properties:
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
534 properties['superseder'] = hyperdb.Multilink(classname)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
535 Class.__init__(self, db, classname, **properties)
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
536
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
537 # Methods to check for existence and nuke the db
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
538 # We don't support multiple named databases
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
539
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
540
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
541 def db_exists(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
542 return bool(Database.memdb)
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
543
6937
b1130680d4ed flake8 fixes
John Rouillard <rouilj@ieee.org>
parents: 6828
diff changeset
544
6365
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
545 def db_nuke(name):
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
546 Database.memdb = {}
7f00fc5958ca Make memorydb persistent across re-open
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6361
diff changeset
547
4342
94c992852f12 add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
548 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/