Mercurial > p > roundup > code
annotate roundup/backends/back_anydbm.py @ 8241:741ea8a86012
fix: issue2551374. Error handling for filter expressions.
Errors in filter expressions are now reported. The UI needs some work
but even the current code is helpful when debugging filter
expressions.
mlink_expr:
defines/raises ExpressionError(error string template,
context=dict())
raises ExpressionError when it detects errors when popping arguments
off stack
raises ExpressionError when more than one element left on the stack
before returning
also ruff fix to group boolean expression with parens
back_anydbm.py, rdbms_common.py:
catches ExpressionError, augments context with class and
attribute being searched. raises the exception
for both link and multilink relations
client.py
catches ExpressionError returning a basic error page. The page is a
dead end. There are no links or anything for the user to move
forward. The user has to go back, possibly refresh the page (because
the submit button may be disalbled) re-enter the query and try
again.
This needs to be improved.
test_liveserver.py
test the error page generated by client.py
db_test_base
unit tests for filter with too few arguments, too many arguments,
check all repr and str formats.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 30 Dec 2024 20:22:55 -0500 |
| parents | 9056b0c8fcd6 |
| children | 24549122f9b1 |
| rev | line source |
|---|---|
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
1 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
6 # |
| 214 | 7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
11 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
17 # |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
18 """This module defines a backend that saves the hyperdatabase in a |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
19 database chosen by anydbm. It is guaranteed to always be available in python |
|
440
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
20 versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several |
|
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
21 serious bugs, and is not available) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
22 """ |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
23 __docformat__ = 'restructuredtext' |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
24 |
|
6998
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
25 import copy |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
26 import logging |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
27 import marshal |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
28 import os |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
29 import re |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
30 import shutil |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
31 import time |
|
1809
bd127cafe3a8
Simplify backend importing, by moving the imports into the backend modules.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1800
diff
changeset
|
32 |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
33 from roundup.anypy.dbm_ import anydbm, whichdb |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
34 from roundup.anypy.strings import b2s, bs2b, repr_export, eval_import, is_us |
|
2908
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
35 |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
36 from roundup import hyperdb, date, password, roundupdb, security, support |
|
8241
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
37 from roundup.mlink_expr import Expression, ExpressionError |
|
2908
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
38 from roundup.backends import locking |
|
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
39 from roundup.i18n import _ |
|
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
40 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
41 from roundup.backends.blobfiles import FileStorage |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
42 from roundup.backends import sessions_dbm |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
43 |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
44 try: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
45 from roundup.backends import sessions_redis |
|
6820
feb175271f3e
redis import failure on python2 causes crash
John Rouillard <rouilj@ieee.org>
parents:
6814
diff
changeset
|
46 except ImportError: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
47 sessions_redis = None |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
48 |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
49 from roundup.backends.indexer_common import get_indexer |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
50 |
|
5725
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
51 from hashlib import md5 |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
52 |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
53 |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
54 def db_exists(config): |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
55 # check for the user db |
|
7884
88a2a6079a89
fix: support dumbdbm on windows.
John Rouillard <rouilj@ieee.org>
parents:
7750
diff
changeset
|
56 for db in 'nodes.user nodes.user.db nodes.user.dat'.split(): |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
57 if os.path.exists(os.path.join(config.DATABASE, db)): |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
58 return 1 |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
59 return 0 |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
60 |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
61 |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
62 def db_nuke(config): |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
63 shutil.rmtree(config.DATABASE) |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
64 |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
65 |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
66 # python 3 doesn't have a unicode type |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
67 try: |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
68 unicode # noqa: F821 |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
69 except NameError: |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
70 unicode = str |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
71 |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
72 |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
73 # marker used for an unspecified keyword argument |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
74 _marker = [] |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
75 |
|
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
76 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
77 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
78 # Now the database |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
79 # |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
80 |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
81 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
82 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
83 """A database for storing records containing flexible data types. |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
84 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
85 Transaction stuff TODO: |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
86 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
87 - check the timestamp of the class file and nuke the cache if it's |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
88 modified. Do some sort of conflict checking on the dirty stuff. |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
89 - perhaps detect write collisions (related to above)? |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
90 |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
91 attributes: |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
92 dbtype: |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
93 holds the value for the type of db. It is used by indexer to |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
94 identify the database type so it can import the correct indexer |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
95 module when using native text search mode. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
96 """ |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
97 |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
98 dbtype = "anydbm" |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
99 |
|
5100
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
100 # used by migrate roundup_admin command. Is a no-op for anydbm. |
|
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
101 # but needed to stop traceback in admin. |
|
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
102 db_version_updated = False |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
103 |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
104 def __init__(self, config, journaltag=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
105 """Open a hyperdatabase given a specifier to some storage. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
106 |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
107 The 'storagelocator' is obtained from config.DATABASE. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
108 The meaning of 'storagelocator' depends on the particular |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
109 implementation of the hyperdatabase. It could be a file name, |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
110 a directory path, a socket descriptor for a connection to a |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
111 database over the network, etc. |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
112 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
113 The 'journaltag' is a token that will be attached to the journal |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
114 entries for any edits done on the database. If 'journaltag' is |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
115 None, the database is opened in read-only mode: the Class.create(), |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
116 Class.set(), Class.retire(), and Class.restore() methods are |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
117 disabled. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
118 """ |
|
3897
e2c2d91932ad
respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3872
diff
changeset
|
119 FileStorage.__init__(self, config.UMASK) |
|
6658
408fd477761f
Add i18n object to roundupdb.Database
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6427
diff
changeset
|
120 roundupdb.Database.__init__(self) |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
121 self.config, self.journaltag = config, journaltag |
|
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
122 self.dir = config.DATABASE |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
123 self.classes = {} |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
124 self.cache = {} # cache of nodes loaded or created |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
125 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
126 'filtering': 0} |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
127 self.dirtynodes = {} # keep track of the dirty nodes by class |
|
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
128 self.newnodes = {} # keep track of the new nodes by class |
|
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
129 self.destroyednodes = {} # keep track of the destroyed nodes by class |
|
252
76c6994aa4e8
CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents:
224
diff
changeset
|
130 self.transactions = [] |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
131 self.indexer = get_indexer(config, self) |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
132 self.security = security.Security(self) |
|
3609
f2fda3e6fc8b
umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents:
3601
diff
changeset
|
133 os.umask(config.UMASK) |
|
440
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
134 |
|
5041
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
135 # make sure the database directory exists |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
136 if not os.path.isdir(self.config.DATABASE): |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
137 os.makedirs(self.config.DATABASE) |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
138 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
139 # lock it |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
140 lockfilenm = os.path.join(self.dir, 'lock') |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
141 self.lockfile = locking.acquire_lock(lockfilenm) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
142 self.lockfile.write(str(os.getpid())) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
143 self.lockfile.flush() |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
144 |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
145 self.Session = None |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
146 self.Otk = None |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
147 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
148 def post_init(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
149 """Called once the schema initialisation has finished. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
150 """ |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
151 super(Database, self).post_init() |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
152 # reindex the db if necessary |
|
826
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
153 if self.indexer.should_reindex(): |
|
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
154 self.reindex() |
|
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
155 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
156 def refresh_database(self): |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
157 """Rebuild the database |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
158 """ |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
159 self.reindex() |
|
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
160 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
161 def getSessionManager(self): |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
162 if not self.Session: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
163 if self.config.SESSIONDB_BACKEND == "redis": |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
164 if sessions_redis is None: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
165 self.Session = sessions_dbm.Sessions(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
166 raise ValueError("[redis] session is set, but " |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
167 "redis is not found") |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
168 self.Session = sessions_redis.Sessions(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
169 else: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
170 self.Session = sessions_dbm.Sessions(self) |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
171 return self.Session |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
172 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
173 def getOTKManager(self): |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
174 if not self.Otk: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
175 if self.config.SESSIONDB_BACKEND == "redis": |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
176 if sessions_redis is None: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
177 self.Session = sessions_dbm.OneTimeKeys(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
178 raise ValueError("[redis] session is set, but " |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
179 "redis is not found") |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
180 self.Otk = sessions_redis.OneTimeKeys(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
181 else: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
182 self.Otk = sessions_dbm.OneTimeKeys(self) |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
183 return self.Otk |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
184 |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
185 def reindex(self, classname=None, show_progress=False): |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
186 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
187 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
188 else: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
189 classes = self.classes.values() |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
190 for klass in classes: |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
191 if show_progress: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
192 for nodeid in support.Progress('Reindex %s' % |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
193 klass.classname, klass.list()): |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
194 klass.index(nodeid) |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
195 else: |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
196 for nodeid in klass.list(): |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
197 klass.index(nodeid) |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
198 self.indexer.save_index() |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
199 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
200 def __repr__(self): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
201 return '<back_anydbm instance at %x>' % id(self) |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
202 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
203 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
204 # Classes |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
205 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
206 def __getattr__(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
207 """A convenient way of calling self.getclass(classname).""" |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
208 if classname in self.classes: |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
209 return self.classes[classname] |
|
5378
35ea9b1efc14
Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5319
diff
changeset
|
210 raise AttributeError(classname) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
211 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
212 def addclass(self, cl): |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
213 cn = cl.classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
214 if cn in self.classes: |
|
7750
216662fbaaee
fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents:
7547
diff
changeset
|
215 raise ValueError(_('Class "%s" already defined.') % cn) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
216 self.classes[cn] = cl |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
217 |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
218 # add default Edit and View permissions |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
219 self.security.addPermission( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
220 name="Create", klass=cn, |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
221 description="User is allowed to create "+cn) |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
222 self.security.addPermission( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
223 name="Edit", klass=cn, |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
224 description="User is allowed to edit "+cn) |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
225 self.security.addPermission( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
226 name="View", klass=cn, |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
227 description="User is allowed to access "+cn) |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
228 self.security.addPermission( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
229 name="Retire", klass=cn, |
|
4517
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4483
diff
changeset
|
230 description="User is allowed to retire "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
231 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
232 def getclasses(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
233 """Return a list of the names of all existing classes.""" |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
234 return sorted(self.classes.keys()) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
235 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
236 def getclass(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
237 """Get the Class object representing a particular class. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
238 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
239 If 'classname' is not a valid class name, a KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
240 """ |
|
1145
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
241 try: |
|
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
242 return self.classes[classname] |
|
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
243 except KeyError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
244 raise KeyError('There is no class called "%s"' % classname) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
245 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
246 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
247 # Class DBs |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
248 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
249 def clear(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
250 """Delete all database contents |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
251 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
252 logging.getLogger('roundup.hyperdb.backend').info('clear') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
253 for cn in self.classes: |
|
7008
f92c56e74f82
in clear() clear journals and node files.
John Rouillard <rouilj@ieee.org>
parents:
7006
diff
changeset
|
254 for data_type in 'nodes', 'journals': |
|
f92c56e74f82
in clear() clear journals and node files.
John Rouillard <rouilj@ieee.org>
parents:
7006
diff
changeset
|
255 path = os.path.join(self.dir, '%s.%s' % (data_type, cn)) |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
256 if os.path.exists(path): |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
257 os.remove(path) |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
258 elif os.path.exists(path+'.db'): # dbm appends .db |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
259 os.remove(path+'.db') |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
260 # reset id sequences |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
261 path = os.path.join(os.getcwd(), self.dir, '_ids') |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
262 if os.path.exists(path): |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
263 os.remove(path) |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
264 elif os.path.exists(path+'.db'): # dbm appends .db |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
265 os.remove(path+'.db') |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
266 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
267 def getclassdb(self, classname, mode='r'): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
268 """ grab a connection to the class db that will be used for |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
269 multiple actions |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
270 """ |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
271 return self.opendb('nodes.%s' % classname, mode) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
272 |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
273 def determine_db_type(self, path): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
274 """ determine which DB wrote the class file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
275 """ |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
276 db_type = '' |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
277 if os.path.exists(path): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
278 db_type = whichdb(path) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
279 if not db_type: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
280 raise hyperdb.DatabaseError(_("Couldn't identify database type")) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
281 elif os.path.exists(path+'.db'): |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
282 # if the path ends in '.db', it's a dbm database, whether |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
283 # anydbm says it's dbhash or not! |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
284 db_type = 'dbm' |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
285 return db_type |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
286 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
287 def opendb(self, name, mode): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
288 """Low-level database opener that gets around anydbm/dbm |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
289 eccentricities. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
290 """ |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
291 # figure the class db type |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
292 path = os.path.join(os.getcwd(), self.dir, name) |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
293 db_type = self.determine_db_type(path) |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
294 |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
295 # new database? let anydbm pick the best dbm |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
296 # in Python 3+ the "dbm" ("anydbm" to us) module already uses the |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
297 # whichdb() function to do this |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
298 if not db_type or hasattr(anydbm, 'whichdb'): |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
299 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
300 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
301 "opendb anydbm.open(%r, 'c')" % path) |
|
1028
16498e77e3ff
allow overiding of the index args roundup/cgi/templating.py
Richard Jones <richard@users.sourceforge.net>
parents:
1022
diff
changeset
|
302 return anydbm.open(path, 'c') |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
303 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
304 # in Python <3 it anydbm was a little dumb so manually open the |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
305 # database with the correct module |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
306 try: |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
307 dbm = __import__(db_type) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
308 except ImportError: |
|
6209
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
309 if db_type == 'gdbm': |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
310 try: |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
311 dbm = __import__('dbm.gnu') |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
312 except ImportError: |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
313 raise hyperdb.DatabaseError(_( |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
314 "Couldn't open database - the required module '%s' " |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
315 "(as dbm.gnu) is not available") % db_type) |
|
6209
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
316 else: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
317 raise hyperdb.DatabaseError(_( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
318 "Couldn't open database - the " |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
319 "required module '%s' is not available") % db_type) |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
320 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
321 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
322 "opendb %r.open(%r, %r)" % (db_type, path, mode)) |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
323 return dbm.open(path, mode) |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
324 |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
325 # |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
326 # Node IDs |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
327 # |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
328 def newid(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
329 """ Generate a new id for the given class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
330 """ |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
331 # open the ids DB - create if if doesn't exist |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
332 db = self.opendb('_ids', 'c') |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
333 if classname in db: |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
334 newid = db[classname] = str(int(db[classname]) + 1) |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
335 else: |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
336 # the count() bit is transitional - older dbs won't start at 1 |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
337 newid = str(self.getclass(classname).count()+1) |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
338 db[classname] = newid |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
339 db.close() |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
340 return newid |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
341 |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
342 def setid(self, classname, setid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
343 """ Set the id counter: used during import of database |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
344 """ |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
345 # open the ids DB - create if if doesn't exist |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
346 db = self.opendb('_ids', 'c') |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
347 db[classname] = str(setid) |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
348 db.close() |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
349 |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
350 # |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
351 # Nodes |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
352 # |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
353 def addnode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
354 """ add the specified node to its class's db |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
355 """ |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
356 # we'll be supplied these props if we're doing an import |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
357 if 'creator' not in node: |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
358 # add in the "calculated" properties (dupe so we don't affect |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
359 # calling code's node assumptions) |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
360 node = node.copy() |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
361 node['creator'] = self.getuid() |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
362 node['actor'] = self.getuid() |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
363 node['creation'] = node['activity'] = date.Date() |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
364 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
365 self.newnodes.setdefault(classname, {})[nodeid] = 1 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
366 self.cache.setdefault(classname, {})[nodeid] = node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
367 self.savenode(classname, nodeid, node) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
368 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
369 def setnode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
370 """ change the specified node |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
371 """ |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
372 self.dirtynodes.setdefault(classname, {})[nodeid] = 1 |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
373 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
374 # can't set without having already loaded the node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
375 self.cache[classname][nodeid] = node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
376 self.savenode(classname, nodeid, node) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
377 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
378 def savenode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
379 """ perform the saving of data specified by the set/addnode |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
380 """ |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
381 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
382 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
383 'save %s%s %r' % (classname, nodeid, node)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
384 self.transactions.append((self.doSaveNode, (classname, nodeid, node))) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
385 |
|
475
a1a44636bace
Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents:
464
diff
changeset
|
386 def getnode(self, classname, nodeid, db=None, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
387 """ get a node from the database |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
388 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
389 Note the "cache" parameter is not used, and exists purely for |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
390 backward compatibility! |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
391 """ |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
392 # try the cache |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
393 cache_dict = self.cache.setdefault(classname, {}) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
394 if nodeid in cache_dict: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
395 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
396 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
397 'get %s%s cached' % (classname, nodeid)) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
398 self.stats['cache_hits'] += 1 |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
399 return cache_dict[nodeid] |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
400 |
|
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
401 if __debug__: |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
402 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
403 start_t = time.time() |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
404 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
405 'get %s%s' % (classname, nodeid)) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
406 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
407 # get from the database and save in the cache |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
408 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
409 db = self.getclassdb(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
410 if nodeid not in db: |
|
8130
9056b0c8fcd6
Fix file descriptor leak in anydbm
Ralf Schlatterbeck <rsc@runtux.com>
parents:
7936
diff
changeset
|
411 db.close() |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
412 raise IndexError("no such %s %s" % (classname, nodeid)) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
413 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
414 # check the uncommitted, destroyed nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
415 if (classname in self.destroyednodes and |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
416 nodeid in self.destroyednodes[classname]): |
|
8130
9056b0c8fcd6
Fix file descriptor leak in anydbm
Ralf Schlatterbeck <rsc@runtux.com>
parents:
7936
diff
changeset
|
417 db.close() |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
418 raise IndexError("no such %s %s" % (classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
419 |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
420 # decode |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
421 res = marshal.loads(db[nodeid]) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
422 |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
423 # reverse the serialisation |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
424 res = self.unserialise(classname, res) |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
425 |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
426 # store off in the cache dict |
|
475
a1a44636bace
Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents:
464
diff
changeset
|
427 if cache: |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
428 cache_dict[nodeid] = res |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
429 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
430 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
431 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
432 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
433 return res |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
434 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
435 def destroynode(self, classname, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
436 """Remove a node from the database. Called exclusively by the |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
437 destroy() method on Class. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
438 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
439 logging.getLogger('roundup.hyperdb.backend').info( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
440 'destroy %s%s' % (classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
441 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
442 # remove from cache and newnodes if it's there |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
443 if (classname in self.cache and nodeid in self.cache[classname]): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
444 del self.cache[classname][nodeid] |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
445 if (classname in self.newnodes and nodeid in self.newnodes[classname]): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
446 del self.newnodes[classname][nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
447 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
448 # see if there's any obvious commit actions that we should get rid of |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
449 for entry in self.transactions[:]: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
450 if entry[1][:2] == (classname, nodeid): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
451 self.transactions.remove(entry) |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
452 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
453 # add to the destroyednodes map |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
454 self.destroyednodes.setdefault(classname, {})[nodeid] = 1 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
455 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
456 # add the destroy commit action |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
457 self.transactions.append((self.doDestroyNode, (classname, nodeid))) |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
458 self.transactions.append((FileStorage.destroy, (self, classname, nodeid))) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
459 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
460 def serialise(self, classname, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
461 """Copy the node contents, converting non-marshallable data into |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
462 marshallable data. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
463 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
464 properties = self.getclass(classname).getprops() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
465 d = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
466 for k, v in node.items(): |
|
2964
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
467 if k == self.RETIRED_FLAG: |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
468 d[k] = v |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
469 continue |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
470 |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
471 # if the property doesn't exist then we really don't care |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
472 if k not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
473 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
474 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
475 # get the property spec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
476 prop = properties[k] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
477 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
478 if isinstance(prop, hyperdb.Password) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
479 d[k] = str(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
480 elif isinstance(prop, hyperdb.Date) and v is not None: |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
481 d[k] = v.serialise() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
482 elif isinstance(prop, hyperdb.Interval) and v is not None: |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
483 d[k] = v.serialise() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
484 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
485 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
486 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
487 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
488 def unserialise(self, classname, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
489 """Decode the marshalled node data |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
490 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
491 properties = self.getclass(classname).getprops() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
492 d = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
493 for k, v in node.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
494 # if the property doesn't exist, or is the "retired" flag then |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
495 # it won't be in the properties dict |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
496 if k not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
497 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
498 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
499 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
500 # get the property spec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
501 prop = properties[k] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
502 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
503 if isinstance(prop, hyperdb.Date) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
504 d[k] = date.Date(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
505 elif isinstance(prop, hyperdb.Interval) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
506 d[k] = date.Interval(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
507 elif isinstance(prop, hyperdb.Password) and v is not None: |
|
7211
506c86823abb
Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents:
7038
diff
changeset
|
508 d[k] = password.Password(encrypted=v, config=self.config) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
509 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
510 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
511 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
512 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
513 def hasnode(self, classname, nodeid, db=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
514 """ determine if the database has a given node |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
515 """ |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
516 # try the cache |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
517 cache = self.cache.setdefault(classname, {}) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
518 if nodeid in cache: |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
519 return 1 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
520 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
521 # not in the cache - check the database |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
522 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
523 db = self.getclassdb(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
524 return nodeid in db |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
525 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
526 def countnodes(self, classname, db=None): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
527 count = 0 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
528 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
529 # include the uncommitted nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
530 if classname in self.newnodes: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
531 count += len(self.newnodes[classname]) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
532 if classname in self.destroyednodes: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
533 count -= len(self.destroyednodes[classname]) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
534 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
535 # and count those in the DB |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
536 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
537 db = self.getclassdb(classname) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
538 return count + len(db) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
539 |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
540 # |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
541 # Files - special node properties |
|
646
07abfe8f0c01
use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
624
diff
changeset
|
542 # inherited from FileStorage |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
543 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
544 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
545 # Journal |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
546 # |
| 1143 | 547 def addjournal(self, classname, nodeid, action, params, creator=None, |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
548 creation=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
549 """ Journal the Action |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
550 'action' may be: |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
551 |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
552 'set' -- 'params' is a dictionary of property values |
|
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
553 'create' -- 'params' is an empty dictionary as of |
|
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
554 Wed Nov 06 11:38:43 2002 +0000 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
555 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
556 'retired' or 'restored' -- 'params' is None |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
557 |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
558 'creator' -- the user performing the action, which defaults to |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
559 the current user. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
560 """ |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
561 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
562 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
563 'addjournal %s%s %s %r %s %r' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
564 classname, nodeid, action, params, creator, creation)) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
565 if creator is None: |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
566 creator = self.getuid() |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
567 self.transactions.append((self.doSaveJournal, ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
568 classname, nodeid, action, params, creator, creation))) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
569 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
570 def setjournal(self, classname, nodeid, journal): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
571 """Set the journal to the "journal" list.""" |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
572 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
573 logging.getLogger('roundup.hyperdb.backend').debug( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
574 'setjournal %s%s %r' % (classname, nodeid, journal)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
575 self.transactions.append((self.doSetJournal, (classname, nodeid, |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
576 journal))) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
577 |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
578 def fix_journal(self, classname, journal): |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
579 """ fix password entries to correct type """ |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
580 pwprops = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
581 for pn, prop in self.getclass(classname).properties.items(): |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
582 if isinstance(prop, hyperdb.Password): |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
583 pwprops[pn] = 1 |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
584 if not pwprops: |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
585 return journal |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
586 for j in journal: |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
587 if j[3] == 'set': |
|
7038
f524ddc27af8
replace for a,b in x.items() with for a in x.keys()
John Rouillard <rouilj@ieee.org>
parents:
7012
diff
changeset
|
588 for k in j[4].keys(): |
|
4538
fd972e18b21a
- fix import/export regression test for anydbm for latest journal fix
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4534
diff
changeset
|
589 if k in pwprops and j[4][k]: |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
590 j[4][k] = password.JournalPassword(j[4][k]) |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
591 return journal |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
592 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
593 def getjournal(self, classname, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
594 """ get the journal for id |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
595 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
596 Raise IndexError if the node doesn't exist (as per history()'s |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
597 API) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
598 """ |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
599 # our journal result |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
600 res = [] |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
601 |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
602 # add any journal entries for transactions not committed to the |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
603 # database |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
604 for method, args in self.transactions: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
605 if method != self.doSaveJournal: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
606 continue |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
607 (cache_classname, cache_nodeid, cache_action, cache_params, |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
608 cache_creator, cache_creation) = args |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
609 if cache_classname == classname and cache_nodeid == nodeid: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
610 if not cache_creator: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
611 cache_creator = self.getuid() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
612 if not cache_creation: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
613 cache_creation = date.Date() |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
614 res.append((cache_nodeid, cache_creation, cache_creator, |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
615 cache_action, cache_params)) |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
616 |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
617 # attempt to open the journal - in some rare cases, the journal may |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
618 # not exist |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
619 try: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
620 db = self.opendb('journals.%s' % classname, 'r') |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
5232
diff
changeset
|
621 except anydbm.error as error: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
622 if str(error) == "need 'c' or 'n' flag to open new db": |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
623 raise IndexError('no such %s %s' % (classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
624 elif error.args[0] != 2: |
|
1920
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
625 # this isn't a "not found" error, be alarmed! |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
626 raise |
|
1920
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
627 if res: |
|
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
628 # we have unsaved journal entries, return them |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
629 return self.fix_journal(classname, res) |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
630 raise IndexError('no such %s %s' % (classname, nodeid)) |
|
787
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
631 try: |
|
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
632 journal = marshal.loads(db[nodeid]) |
|
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
633 except KeyError: |
|
843
01f9d02faea1
...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents:
842
diff
changeset
|
634 db.close() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
635 if res: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
636 # we have some unsaved journal entries, be happy! |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
637 return self.fix_journal(classname, res) |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
638 raise IndexError('no such %s %s' % (classname, nodeid)) |
|
843
01f9d02faea1
...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents:
842
diff
changeset
|
639 db.close() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
640 |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
641 # add all the saved journal entries for this node |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
642 for nodeid, date_stamp, user, action, params in journal: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
643 res.append((nodeid, date.Date(date_stamp), user, action, params)) |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
644 return self.fix_journal(classname, res) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
645 |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
646 def pack(self, pack_before): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
647 """ Delete all journal entries except "create" before 'pack_before'. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
648 """ |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
649 pack_before = pack_before.serialise() |
| 990 | 650 for classname in self.getclasses(): |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
651 packed = 0 |
| 990 | 652 # get the journal db |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
653 db_name = 'journals.%s' % classname |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
654 path = os.path.join(os.getcwd(), self.dir, classname) |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
655 db_type = self.determine_db_type(path) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
656 db = self.opendb(db_name, 'w') |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
657 |
|
5444
167f0d25ea8e
Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5395
diff
changeset
|
658 for key in map(b2s, db.keys()): |
| 990 | 659 # get the journal for this db entry |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
660 journal = marshal.loads(db[key]) |
|
7010
1bb7aa8414b8
change variable name l -> kept_entries
John Rouillard <rouilj@ieee.org>
parents:
7009
diff
changeset
|
661 kept_entries = [] |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
662 for entry in journal: |
| 990 | 663 # unpack the entry |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
664 (nodeid, date_stamp, self.journaltag, action, |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
665 params) = entry |
| 990 | 666 # if the entry is after the pack date, _or_ the initial |
| 667 # create entry, then it stays | |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
668 if date_stamp > pack_before or action == 'create': |
|
7010
1bb7aa8414b8
change variable name l -> kept_entries
John Rouillard <rouilj@ieee.org>
parents:
7009
diff
changeset
|
669 kept_entries.append(entry) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
670 else: |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
671 packed += 1 |
|
7010
1bb7aa8414b8
change variable name l -> kept_entries
John Rouillard <rouilj@ieee.org>
parents:
7009
diff
changeset
|
672 db[key] = marshal.dumps(kept_entries) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
673 |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
674 logging.getLogger('roundup.hyperdb.backend').info( |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
675 'packed %d %s items' % (packed, classname)) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
676 |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
677 if db_type == 'gdbm': |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
678 db.reorganize() |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
679 db.close() |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
680 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
681 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
682 # Basic transaction support |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
683 # |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
684 def commit(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
685 """ Commit the current transactions. |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
686 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
687 Save all data changed since the database was opened or since the |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
688 last commit() or rollback(). |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
689 """ |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
690 logging.getLogger('roundup.hyperdb.backend').info( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
691 'commit %s transactions' % (len(self.transactions))) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
692 |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
693 # keep a handle to all the database files opened |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
694 self.databases = {} |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
695 |
|
1904
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
696 try: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
697 # now, do all the transactions |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
698 reindex = {} |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
699 for method, args in self.transactions: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
700 reindex[method(*args)] = 1 |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
701 finally: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
702 # make sure we close all the database files |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
703 for db in self.databases.values(): |
|
1904
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
704 db.close() |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
705 del self.databases |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
706 |
|
3960
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
707 # clear the transactions list now so the blobfile implementation |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
708 # doesn't think there's still pending file commits when it tries |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
709 # to access the file data |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
710 self.transactions = [] |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
711 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
712 # reindex the nodes that request it |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
713 for classname, nodeid in [k for k in reindex if k]: |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
714 self.getclass(classname).index(nodeid) |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
715 |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
716 # save the indexer state |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
717 self.indexer.save_index() |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
718 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
719 self.clearCache() |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
720 |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
721 def clearCache(self): |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
722 # all transactions committed, back to normal |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
723 self.cache = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
724 self.dirtynodes = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
725 self.newnodes = {} |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
726 self.destroyednodes = {} |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
727 self.transactions = [] |
|
4652
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4538
diff
changeset
|
728 # upcall is necessary! |
|
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4538
diff
changeset
|
729 roundupdb.Database.clearCache(self) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
730 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
731 def getCachedClassDB(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
732 """ get the class db, looking in our cache of databases for commit |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
733 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
734 # get the database handle |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
735 db_name = 'nodes.%s' % classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
736 if db_name not in self.databases: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
737 self.databases[db_name] = self.getclassdb(classname, 'c') |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
738 return self.databases[db_name] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
739 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
740 def doSaveNode(self, classname, nodeid, node): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
741 db = self.getCachedClassDB(classname) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
742 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
743 # now save the marshalled data |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
744 db[nodeid] = marshal.dumps(self.serialise(classname, node)) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
745 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
746 # return the classname, nodeid so we reindex this content |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
747 return (classname, nodeid) |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
748 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
749 def getCachedJournalDB(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
750 """ get the journal db, looking in our cache of databases for commit |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
751 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
752 # get the database handle |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
753 db_name = 'journals.%s' % classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
754 if db_name not in self.databases: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
755 self.databases[db_name] = self.opendb(db_name, 'c') |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
756 return self.databases[db_name] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
757 |
| 1143 | 758 def doSaveJournal(self, classname, nodeid, action, params, creator, |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
759 creation): |
| 1143 | 760 # serialise the parameters now if necessary |
| 761 if isinstance(params, type({})): | |
| 762 if action in ('set', 'create'): | |
| 763 params = self.serialise(classname, params) | |
| 764 | |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
765 # handle supply of the special journalling parameters (usually |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
766 # supplied on importing an existing database) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
767 journaltag = creator |
| 1143 | 768 if creation: |
| 769 journaldate = creation.serialise() | |
| 770 else: | |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
771 journaldate = date.Date().serialise() |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
772 |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
773 # create the journal entry |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
774 entry = (nodeid, journaldate, journaltag, action, params) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
775 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
776 db = self.getCachedJournalDB(classname) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
777 |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
778 # now insert the journal entry |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
779 if nodeid in db: |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
780 # append to existing |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
781 s = db[nodeid] |
|
650
9b2575610953
Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents:
646
diff
changeset
|
782 l = marshal.loads(s) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
783 l.append(entry) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
784 else: |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
785 l = [entry] |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
786 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
787 db[nodeid] = marshal.dumps(l) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
788 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
789 def doSetJournal(self, classname, nodeid, journal): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
790 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
791 for nodeid, journaldate, journaltag, action, params in journal: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
792 # serialise the parameters now if necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
793 if isinstance(params, type({})): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
794 if action in ('set', 'create'): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
795 params = self.serialise(classname, params) |
|
2512
f5542d92307a
fix anydbm journal import [SF#983166]
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
796 journaldate = journaldate.serialise() |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
797 l.append((nodeid, journaldate, journaltag, action, params)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
798 db = self.getCachedJournalDB(classname) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
799 db[nodeid] = marshal.dumps(l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
800 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
801 def doDestroyNode(self, classname, nodeid): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
802 # delete from the class database |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
803 db = self.getCachedClassDB(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
804 if nodeid in db: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
805 del db[nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
806 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
807 # delete from the database |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
808 db = self.getCachedJournalDB(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
809 if nodeid in db: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
810 del db[nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
811 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
812 def rollback(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
813 """ Reverse all actions from the current transaction. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
814 """ |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
815 logging.getLogger('roundup.hyperdb.backend').info( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
816 'rollback %s transactions' % (len(self.transactions))) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
817 |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
818 for method, args in self.transactions: |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
819 # delete temporary files |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
820 if method == self.doStoreFile: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
821 self.rollbackStoreFile(*args) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
822 self.cache = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
823 self.dirtynodes = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
824 self.newnodes = {} |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
825 self.destroyednodes = {} |
|
252
76c6994aa4e8
CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents:
224
diff
changeset
|
826 self.transactions = [] |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
827 |
|
1131
92e92ae58494
add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents:
1127
diff
changeset
|
828 def close(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
829 """ Nothing to do |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
830 """ |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
831 if self.lockfile is not None: |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
832 locking.release_lock(self.lockfile) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
833 self.lockfile.close() |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
834 self.lockfile = None |
|
1131
92e92ae58494
add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents:
1127
diff
changeset
|
835 |
| 7002 | 836 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
837 class Class(hyperdb.Class): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
838 """The handle to a particular class of nodes in a hyperdatabase.""" |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
839 |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
840 def enableJournalling(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
841 """Turn journalling on for this class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
842 """ |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
843 self.do_journal = 1 |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
844 |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
845 def disableJournalling(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
846 """Turn journalling off for this class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
847 """ |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
848 self.do_journal = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
849 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
850 # Editing nodes: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
851 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
852 def create(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
853 """Create a new node of this class and return its id. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
854 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
855 The keyword arguments in 'propvalues' map property names to values. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
856 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
857 The values of arguments must be acceptable for the types of their |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
858 corresponding properties or a TypeError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
859 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
860 If this class has a key property, it must be present and its value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
861 must not collide with other key strings or a ValueError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
862 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
863 Any other properties on this class that are missing from the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
864 'propvalues' dictionary are set to None. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
865 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
866 If an id in a link or multilink property does not refer to a valid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
867 node, an IndexError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
868 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
869 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
870 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
871 """ |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
872 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
873 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
874 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
875 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
876 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
877 return newid |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
878 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
879 def create_inner(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
880 """ Called by create, in-between the audit and react calls. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
881 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
882 if 'id' in propvalues: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
883 raise KeyError('"id" is reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
884 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
885 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
886 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
887 |
| 7002 | 888 if ('creator' in propvalues or 'actor' in propvalues or |
| 889 'creation' in propvalues or 'activity' in propvalues): | |
|
5704
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
890 raise KeyError('"creator", "actor", "creation" and ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
891 '"activity" are reserved') |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
892 |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
893 for p in propvalues: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
894 prop = self.properties[p] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
895 if prop.computed: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
896 raise KeyError('"%s" is a computed property' % p) |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
897 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
898 # new node's id |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
899 newid = self.db.newid(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
900 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
901 # validate propvalues |
| 5809 | 902 num_re = re.compile(r'^\d+$') |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
903 for key, value in propvalues.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
904 if key == self.key: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
905 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
906 self.lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
907 except KeyError: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
908 pass |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
909 else: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
910 raise ValueError('node with key "%s" exists' % value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
911 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
912 # try to handle this property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
913 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
914 prop = self.properties[key] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
915 except KeyError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
916 raise KeyError('"%s" has no property "%s"' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
917 self.classname, key)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
918 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
919 if value is not None and isinstance(prop, hyperdb.Link): |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
920 if not isinstance(value, str): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
921 raise ValueError('link value must be String') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
922 link_class = self.properties[key].classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
923 # if it isn't a number, it's a key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
924 if not num_re.match(value): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
925 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
926 value = self.db.classes[link_class].lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
927 except (TypeError, KeyError): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
928 raise IndexError('new property "%s": %s not a %s' % ( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
929 key, value, link_class)) |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
930 elif not self.db.getclass(link_class).hasnode(value): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
931 raise IndexError('%s has no node %s' % (link_class, value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
932 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
933 # save off the value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
934 propvalues[key] = value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
935 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
936 # register the link with the newly linked node |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
937 if self.do_journal and self.properties[key].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
938 self.db.addjournal(link_class, value, 'link', |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
939 (self.classname, newid, key)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
940 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
941 elif isinstance(prop, hyperdb.Multilink): |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
942 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
943 value = [] |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
944 if not hasattr(value, '__iter__') or isinstance(value, str): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
945 raise TypeError( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
946 'new property "%s" not an iterable of ids' % key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
947 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
948 # clean up and validate the list of links |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
949 link_class = self.properties[key].classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
950 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
951 for entry in value: |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
952 if not isinstance(entry, str): |
| 7004 | 953 raise ValueError('"%s" multilink value (%r) ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
954 'must contain Strings' % (key, value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
955 # if it isn't a number, it's a key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
956 if not num_re.match(entry): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
957 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
958 entry = self.db.classes[link_class].lookup(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
959 except (TypeError, KeyError): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
960 raise IndexError( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
961 'new property "%s": %s not a %s' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
962 key, entry, |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
963 self.properties[key].classname)) |
|
6999
e5a28b8ded72
backout checkin 60b37f632601
John Rouillard <rouilj@ieee.org>
parents:
6998
diff
changeset
|
964 l.append(entry) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
965 value = l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
966 propvalues[key] = value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
967 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
968 # handle additions |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
969 for nodeid in value: |
|
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
970 if not self.db.getclass(link_class).hasnode(nodeid): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
971 raise IndexError('%s has no node %s' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
972 link_class, nodeid)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
973 # register the link with the newly linked node |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
974 if self.do_journal and self.properties[key].do_journal: |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
975 self.db.addjournal(link_class, nodeid, 'link', |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
976 (self.classname, newid, key)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
977 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
978 elif isinstance(prop, hyperdb.String): |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
979 if not isinstance(value, (str, unicode)): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
980 raise TypeError('new property "%s" not a string' % key) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
981 if prop.indexme: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
982 self.db.indexer.add_text( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
983 (self.classname, newid, key), value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
984 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
985 elif isinstance(prop, hyperdb.Password): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
986 if not isinstance(value, password.Password): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
987 raise TypeError('new property "%s" not a Password' % key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
988 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
989 elif isinstance(prop, hyperdb.Date): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
990 if value is not None and not isinstance(value, date.Date): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
991 raise TypeError('new property "%s" not a Date' % key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
992 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
993 elif isinstance(prop, hyperdb.Interval): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
994 if value is not None and not isinstance(value, date.Interval): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
995 raise TypeError('new property "%s" not an Interval' % key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
996 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
997 elif value is not None and isinstance(prop, hyperdb.Number): |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
998 try: |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
999 float(value) |
|
887
e7169d6e6e45
added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents:
886
diff
changeset
|
1000 except ValueError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1001 raise TypeError('new property "%s" not numeric' % key) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1002 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1003 elif value is not None and isinstance(prop, hyperdb.Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1004 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1005 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1006 except ValueError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1007 raise TypeError('new property "%s" not an integer' % key) |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1008 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1009 elif value is not None and isinstance(prop, hyperdb.Boolean): |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1010 try: |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1011 int(value) |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1012 except ValueError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1013 raise TypeError('new property "%s" not boolean' % key) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1014 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1015 # make sure there's data where there needs to be |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1016 for key, prop in self.properties.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1017 if key in propvalues: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1018 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1019 if key == self.key: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1020 raise ValueError('key property "%s" is required' % key) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1021 if isinstance(prop, hyperdb.Multilink): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1022 propvalues[key] = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1023 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1024 # done |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1025 self.db.addnode(self.classname, newid, propvalues) |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1026 if self.do_journal: |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1027 self.db.addjournal(self.classname, newid, 'create', {}) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1028 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1029 return newid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1030 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1031 def get(self, nodeid, propname, default=_marker, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1032 """Get the value of a property on an existing node of this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1033 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1034 'nodeid' must be the id of an existing node of this class or an |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1035 IndexError is raised. 'propname' must be the name of a property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1036 of this class or a KeyError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1037 |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1038 'cache' exists for backward compatibility, and is not used. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1039 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1040 Attempts to get the "creation" or "activity" properties should |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1041 do the right thing. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1042 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1043 if propname == 'id': |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1044 return nodeid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1045 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1046 # get the node's dict |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1047 d = self.db.getnode(self.classname, nodeid) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1048 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1049 # check for one of the special props |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1050 if propname == 'creation': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1051 if 'creation' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1052 return d['creation'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1053 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1054 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1055 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1056 if journal: |
|
4342
94c992852f12
add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
4075
diff
changeset
|
1057 return journal[0][1] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1058 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1059 # on the strange chance that there's no journal |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1060 return date.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1061 if propname == 'activity': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1062 if 'activity' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1063 return d['activity'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1064 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1065 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1066 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1067 if journal: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1068 return self.db.getjournal(self.classname, nodeid)[-1][1] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1069 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1070 # on the strange chance that there's no journal |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1071 return date.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1072 if propname == 'creator': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1073 if 'creator' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1074 return d['creator'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1075 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1076 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1077 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1078 if journal: |
| 5809 | 1079 num_re = re.compile(r'^\d+$') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1080 value = journal[0][2] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1081 if num_re.match(value): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1082 return value |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1083 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1084 # old-style "username" journal tag |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1085 try: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1086 return self.db.user.lookup(value) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1087 except KeyError: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1088 # user's been retired, return admin |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1089 return '1' |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1090 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1091 return self.db.getuid() |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1092 if propname == 'actor': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1093 if 'actor' in d: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1094 return d['actor'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1095 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1096 raise ValueError('Journalling is disabled for this class') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1097 journal = self.db.getjournal(self.classname, nodeid) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1098 if journal: |
| 5809 | 1099 num_re = re.compile(r'^\d+$') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1100 value = journal[-1][2] |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1101 if num_re.match(value): |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1102 return value |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1103 else: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1104 # old-style "username" journal tag |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1105 try: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1106 return self.db.user.lookup(value) |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1107 except KeyError: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1108 # user's been retired, return admin |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1109 return '1' |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1110 else: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1111 return self.db.getuid() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1112 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1113 # get the property (raises KeyErorr if invalid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1114 prop = self.properties[propname] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1115 |
|
6179
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1116 if isinstance(prop, hyperdb.Multilink) and prop.computed: |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1117 cls = self.db.getclass(prop.rev_classname) |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1118 ids = cls.find(**{prop.rev_propname: nodeid}) |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1119 return ids |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1120 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1121 if propname not in d: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1122 if default is _marker: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1123 if isinstance(prop, hyperdb.Multilink): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1124 return [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1125 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1126 return None |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1127 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1128 return default |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1129 |
|
1014
8816534e6a1a
Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents:
1002
diff
changeset
|
1130 # return a dupe of the list so code doesn't get confused |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1131 if isinstance(prop, hyperdb.Multilink): |
|
6351
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1132 ids = d[propname][:] |
|
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1133 ids.sort(key=lambda x: int(x)) |
|
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1134 return ids |
|
1014
8816534e6a1a
Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents:
1002
diff
changeset
|
1135 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1136 return d[propname] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1137 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1138 def set(self, nodeid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1139 """Modify a property on an existing node of this class. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1140 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1141 'nodeid' must be the id of an existing node of this class or an |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1142 IndexError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1143 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1144 Each key in 'propvalues' must be the name of a property of this |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1145 class or a KeyError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1146 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1147 All values in 'propvalues' must be acceptable types for their |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1148 corresponding properties or a TypeError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1149 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1150 If the value of the key property is set, it must not collide with |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1151 other key strings or a ValueError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1152 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1153 If the value of a Link or Multilink property contains an invalid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1154 node id, a ValueError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1155 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1156 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1157 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1158 """ |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
1159 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1160 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
1161 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1162 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1163 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1164 for name, prop in self.getprops(protected=0).items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1165 if name in oldvalues: |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1166 continue |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1167 if isinstance(prop, hyperdb.Multilink): |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1168 oldvalues[name] = [] |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1169 else: |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1170 oldvalues[name] = None |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1171 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1172 self.fireReactors('set', nodeid, oldvalues) |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1173 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1174 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1175 def set_inner(self, nodeid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1176 """ Called by set, in-between the audit and react calls. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1177 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1178 if not propvalues: |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1179 return propvalues |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1180 |
| 7002 | 1181 if ('creator' in propvalues or 'actor' in propvalues or |
| 1182 'creation' in propvalues or 'activity' in propvalues): | |
|
5704
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
1183 raise KeyError('"creator", "actor", "creation" and ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1184 '"activity" are reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1185 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1186 if 'id' in propvalues: |
|
5378
35ea9b1efc14
Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5319
diff
changeset
|
1187 raise KeyError('"id" is reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1188 |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1189 for p in propvalues: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1190 prop = self.properties[p] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1191 if prop.computed: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1192 raise KeyError('"%s" is a computed property' % p) |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1193 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1194 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1195 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1196 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1197 node = self.db.getnode(self.classname, nodeid) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1198 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1199 raise IndexError |
| 5809 | 1200 num_re = re.compile(r'^\d+$') |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1201 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1202 # if the journal value is to be different, store it in here |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1203 journalvalues = {} |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1204 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1205 # omit quiet properties from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1206 quiet_props = [] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1207 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1208 # list() propvalues 'cos it might be modified by the loop |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1209 for propname, value in list(propvalues.items()): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1210 # check to make sure we're not duplicating an existing key |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1211 if propname == self.key and node[propname] != value: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1212 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1213 self.lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1214 except KeyError: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1215 pass |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1216 else: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1217 raise ValueError('node with key "%s" exists' % value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1218 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1219 # this will raise the KeyError if the property isn't valid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1220 # ... we don't use getprops() here because we only care about |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1221 # the writeable properties. |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1222 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1223 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1224 except KeyError: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1225 raise KeyError('"%s" has no property named "%s"' % ( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1226 self.classname, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1227 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1228 # if the value's the same as the existing value, no sense in |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1229 # doing anything |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1230 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1231 if value == current: |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1232 del propvalues[propname] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1233 continue |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1234 journalvalues[propname] = current |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1235 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1236 # do stuff based on the prop type |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1237 if isinstance(prop, hyperdb.Link): |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1238 link_class = prop.classname |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1239 # if it isn't a number, it's a key |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1240 if value is not None and not isinstance(value, type('')): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1241 raise ValueError('property "%s" link value be a string' % ( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1242 propname)) |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1243 if isinstance(value, type('')) and not num_re.match(value): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1244 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1245 value = self.db.classes[link_class].lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1246 except (TypeError, KeyError): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1247 raise IndexError('new property "%s": %s not a %s' % ( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1248 propname, value, prop.classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1249 |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1250 if (value is not None and |
|
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1251 not self.db.getclass(link_class).hasnode(value)): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1252 raise IndexError('%s has no node %s' % (link_class, |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1253 value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1254 |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1255 if self.do_journal and prop.do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1256 # register the unlink with the old linked node |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1257 if propname in node and node[propname] is not None: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1258 self.db.addjournal(link_class, node[propname], |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1259 'unlink', |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1260 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1261 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1262 # register the link with the newly linked node |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1263 if value is not None: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1264 self.db.addjournal(link_class, value, 'link', |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1265 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1266 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1267 elif isinstance(prop, hyperdb.Multilink): |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1268 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1269 value = [] |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
1270 if not hasattr(value, '__iter__') or isinstance(value, str): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1271 raise TypeError('new property "%s" not an iterable of' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1272 ' ids' % propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1273 link_class = self.properties[propname].classname |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1274 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1275 for entry in value: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1276 # if it isn't a number, it's a key |
|
7011
4d2625f10314
flake8 use isinstance rather than comparing types.
John Rouillard <rouilj@ieee.org>
parents:
7010
diff
changeset
|
1277 if not isinstance(entry, str): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1278 raise ValueError('new property "%s" link value ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1279 'must be a string' % propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1280 if not num_re.match(entry): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1281 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1282 entry = self.db.classes[link_class].lookup(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1283 except (TypeError, KeyError): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1284 raise IndexError( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1285 'new property "%s": %s not a %s' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1286 propname, entry, |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1287 self.properties[propname].classname)) |
|
6999
e5a28b8ded72
backout checkin 60b37f632601
John Rouillard <rouilj@ieee.org>
parents:
6998
diff
changeset
|
1288 l.append(entry) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1289 value = l |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1290 propvalues[propname] = value |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1291 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1292 # figure the journal entry for this property |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1293 add = [] |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1294 remove = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1295 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1296 # handle removals |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1297 if propname in node: |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1298 l = node[propname] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1299 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1300 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1301 for id in l[:]: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1302 if id in value: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1303 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1304 # register the unlink with the old linked node |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1305 if self.do_journal and self.properties[propname].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1306 self.db.addjournal(link_class, id, 'unlink', |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1307 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1308 l.remove(id) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1309 remove.append(id) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1310 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1311 # handle additions |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1312 for id in value: |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
1313 if not self.db.getclass(link_class).hasnode(id): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1314 raise IndexError('%s has no node %s' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1315 link_class, id)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1316 if id in l: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1317 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1318 # register the link with the newly linked node |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1319 if self.do_journal and self.properties[propname].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1320 self.db.addjournal(link_class, id, 'link', |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1321 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1322 l.append(id) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1323 add.append(id) |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1324 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1325 # figure the journal entry |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1326 l = [] |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1327 if add: |
|
967
dd35bab19dd9
use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents:
964
diff
changeset
|
1328 l.append(('+', add)) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1329 if remove: |
|
967
dd35bab19dd9
use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents:
964
diff
changeset
|
1330 l.append(('-', remove)) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1331 if l: |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1332 journalvalues[propname] = tuple(l) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1333 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1334 elif isinstance(prop, hyperdb.String): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1335 if value is not None and not isinstance(value, (str, unicode)): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1336 raise TypeError('new property "%s" not a ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1337 'string' % propname) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
1338 if prop.indexme: |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1339 self.db.indexer.add_text( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1340 (self.classname, nodeid, propname), |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
1341 value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1342 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1343 elif isinstance(prop, hyperdb.Password): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1344 if not isinstance(value, password.Password): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1345 raise TypeError('new property "%s" not a ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1346 'Password' % propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1347 propvalues[propname] = value |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1348 journalvalues[propname] = \ |
|
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1349 current and password.JournalPassword(current) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1350 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1351 elif value is not None and isinstance(prop, hyperdb.Date): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1352 if not isinstance(value, date.Date): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1353 raise TypeError('new property "%s" not a ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1354 'Date' % propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1355 propvalues[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1356 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1357 elif value is not None and isinstance(prop, hyperdb.Interval): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1358 if not isinstance(value, date.Interval): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1359 raise TypeError('new property "%s" not an ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1360 'Interval' % propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1361 propvalues[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1362 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1363 elif value is not None and isinstance(prop, hyperdb.Number): |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1364 try: |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1365 float(value) |
|
887
e7169d6e6e45
added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents:
886
diff
changeset
|
1366 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1367 raise TypeError('new property "%s" not ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1368 'numeric' % propname) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1369 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1370 elif value is not None and isinstance(prop, hyperdb.Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1371 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1372 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1373 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1374 raise TypeError('new property "%s" not ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1375 'numeric' % propname) |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1376 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1377 elif value is not None and isinstance(prop, hyperdb.Boolean): |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1378 try: |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1379 int(value) |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1380 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1381 raise TypeError('new property "%s" not ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1382 'boolean' % propname) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1383 |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1384 node[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1385 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1386 # record quiet properties to omit from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1387 if prop.quiet: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1388 quiet_props.append(propname) |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1389 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1390 # nothing to do? |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1391 if not propvalues: |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1392 return propvalues |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1393 |
|
3083
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1394 # update the activity time |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1395 node['activity'] = date.Date() |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1396 node['actor'] = self.db.getuid() |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1397 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1398 # do the set, and journal it |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1399 self.db.setnode(self.classname, nodeid, node) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1400 |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1401 if self.do_journal: |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1402 self.db.addjournal(self.classname, nodeid, 'set', journalvalues) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1403 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1404 # remove quiet properties from output |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1405 for propname in quiet_props: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1406 if propname in propvalues: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1407 del propvalues[propname] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1408 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1409 return propvalues |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1410 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1411 def retire(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1412 """Retire a node. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1413 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1414 The properties on the node remain available from the get() method, |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1415 and the node's id is never reused. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1416 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1417 Retired nodes are not returned by the find(), list(), or lookup() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1418 methods, and other nodes may reuse the values of their key properties. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1419 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1420 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1421 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1422 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1423 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1424 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1425 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1426 self.fireAuditors('retire', nodeid, None) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1427 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1428 node = self.db.getnode(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1429 node[self.db.RETIRED_FLAG] = 1 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1430 self.db.setnode(self.classname, nodeid, node) |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1431 if self.do_journal: |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1432 self.db.addjournal(self.classname, nodeid, 'retired', None) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1433 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1434 self.fireReactors('retire', nodeid, None) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1435 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1436 def restore(self, nodeid): |
|
5107
33ac07a7ef96
spelling error fixed in comment
John Rouillard <rouilj@ieee.org>
parents:
5100
diff
changeset
|
1437 """Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1438 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1439 Make node available for all operations like it was before retirement. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1440 """ |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1441 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1442 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1443 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1444 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1445 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1446 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1447 try: |
|
6998
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
1448 # eval for exception side effect |
|
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
1449 id = self.lookup(node[key]) # noqa: F841 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1450 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1451 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1452 else: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1453 raise KeyError("Key property (%s) of retired node clashes " |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1454 "with existing one (%s)" % (key, node[key])) |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1455 # Now we can safely restore node |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1456 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1457 del node[self.db.RETIRED_FLAG] |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1458 self.db.setnode(self.classname, nodeid, node) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1459 if self.do_journal: |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1460 self.db.addjournal(self.classname, nodeid, 'restored', None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1461 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1462 self.fireReactors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1463 |
|
1476
5a01e90b7dc9
fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents:
1467
diff
changeset
|
1464 def is_retired(self, nodeid, cldb=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1465 """Return true if the node is retired. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1466 """ |
|
1476
5a01e90b7dc9
fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents:
1467
diff
changeset
|
1467 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1468 if self.db.RETIRED_FLAG in node: |
|
940
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1469 return 1 |
|
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1470 return 0 |
|
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1471 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1472 def destroy(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1473 """Destroy a node. |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
1474 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1475 WARNING: this method should never be used except in extremely rare |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1476 situations where there could never be links to the node being |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1477 deleted |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1478 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1479 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1480 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1481 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1482 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1483 WARNING: really, use retire() instead |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1484 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1485 Well, I think that's enough warnings. This method exists mostly to |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1486 support the session storage of the cgi interface. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1487 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1488 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1489 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1490 self.db.destroynode(self.classname, nodeid) |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1491 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1492 # Locating nodes: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1493 def hasnode(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1494 """Determine if the given nodeid actually exists |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1495 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1496 return self.db.hasnode(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1497 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1498 def setkey(self, propname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1499 """Select a String property of this class to be the key property. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1500 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1501 'propname' must be the name of a String property of this class or |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1502 None, or a TypeError is raised. The values of the key property on |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1503 all existing nodes must be unique or a ValueError is raised. If the |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1504 property doesn't exist, KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1505 """ |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1506 prop = self.getprops()[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1507 if not isinstance(prop, hyperdb.String): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1508 raise TypeError('key properties must be String') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1509 self.key = propname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1510 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1511 def getkey(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1512 """Return the name of the key property for this class or None.""" |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1513 return self.key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1514 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1515 # TODO: set up a separate index db file for this? profile? |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1516 def lookup(self, keyvalue): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1517 """Locate a particular node by its key property and return its id. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1518 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1519 If this class has no key property, a TypeError is raised. If the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1520 'keyvalue' matches one of the values for the key property among |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1521 the nodes in this class, the matching node's id is returned; |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1522 otherwise a KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1523 """ |
|
969
ddcb527491ba
Consistent quoting
Richard Jones <richard@users.sourceforge.net>
parents:
967
diff
changeset
|
1524 if not self.key: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1525 raise TypeError('No key property set for ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1526 'class %s' % self.classname) |
|
7936
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1527 |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1528 # special notation for looking up the current database user |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1529 if keyvalue == '@current_user' and self.classname == 'user': |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1530 keyvalue = self.db.user.get(self.db.getuid(), self.key) |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1531 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1532 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1533 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1534 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1535 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1536 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1537 continue |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1538 if self.key not in node: |
|
2606
17eb5aeada7f
fix bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2601
diff
changeset
|
1539 continue |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1540 if node[self.key] == keyvalue: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1541 return nodeid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1542 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1543 cldb.close() |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1544 raise KeyError('No key (%s) value "%s" for "%s"' % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1545 self.key, keyvalue, self.classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1546 |
|
1103
db787cef1385
handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents:
1099
diff
changeset
|
1547 # change from spec - allows multiple props to match |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1548 def find(self, **propspec): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1549 """Get the ids of nodes in this class which link to the given nodes. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1550 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1551 'propspec' consists of keyword args propname=nodeid or |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1552 propname={nodeid:1, } |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1553 'propname' must be the name of a property in this class, or a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1554 KeyError is raised. That property must be a Link or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1555 Multilink property, or a TypeError is raised. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1556 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1557 Any node in this class whose 'propname' property links to any of |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1558 the nodeids will be returned. Examples:: |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1559 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1560 db.issue.find(messages='1') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1561 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1562 db.issue.find(messages=('1','3'), files=('7',)) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1563 db.issue.find(messages=['1','3'], files=['7']) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1564 """ |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1565 # shortcut |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1566 if not propspec: |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1567 return [] |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1568 |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1569 # validate the args |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1570 props = self.getprops() |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1571 for propname, itemids in propspec.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1572 # check the prop is OK |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1573 prop = props[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1574 if not isinstance(prop, hyperdb.Link) and not isinstance(prop, hyperdb.Multilink): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1575 raise TypeError("'%s' not a Link/Multilink " |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1576 "property" % propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1577 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1578 # ok, now do the find |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1579 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1580 l = [] |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1581 rev_multilinks = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1582 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1583 for id in self.getnodeids(db=cldb): |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1584 item = self.db.getnode(self.classname, id, db=cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1585 if self.db.RETIRED_FLAG in item: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1586 continue |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1587 for propname, itemids in propspec.items(): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1588 if not isinstance(itemids, dict): |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1589 if itemids is None or isinstance(itemids, type("")): |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1590 itemids = {itemids: 1} |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1591 else: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1592 itemids = dict.fromkeys(itemids) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1593 |
|
2450
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1594 # special case if the item doesn't have this property |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1595 if propname not in item: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1596 if None in itemids: |
|
2450
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1597 l.append(id) |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1598 break |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1599 continue |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1600 |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1601 # grab the property definition and its value on this item |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1602 prop = props[propname] |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1603 value = item[propname] |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1604 if isinstance(prop, hyperdb.Link) and value in itemids: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1605 l.append(id) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1606 break |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1607 elif isinstance(prop, hyperdb.Multilink): |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1608 if prop.rev_property: |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
1609 rev_multilinks.append((prop, itemids)) |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1610 continue |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1611 hit = 0 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1612 for v in value: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1613 if v in itemids: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1614 l.append(id) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1615 hit = 1 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1616 break |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1617 if hit: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1618 break |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1619 for prop, itemids in rev_multilinks: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1620 rprop = prop.rev_property |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1621 fun = l.append |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
1622 if isinstance(rprop, hyperdb.Multilink): |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1623 fun = l.extend |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1624 for id in itemids: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1625 fun(rprop.cls.get(id, rprop.name)) |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1626 if rev_multilinks: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1627 l = list(sorted(set(l))) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1628 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1629 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1630 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1631 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1632 def stringFind(self, **requirements): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1633 """Locate a particular node by matching a set of its String |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1634 properties in a caseless search. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1635 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1636 If the property is not a String property, a TypeError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1637 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1638 The return is a list of the id of all nodes that match. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1639 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1640 for propname in requirements: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1641 prop = self.properties[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1642 if not isinstance(prop, hyperdb.String): |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
1643 raise TypeError("'%s' not a String property" % propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1644 requirements[propname] = requirements[propname].lower() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1645 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1646 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1647 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1648 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1649 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1650 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1651 continue |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1652 for key, value in requirements.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1653 if key not in node: |
|
1296
34b1e6490e65
hardening for stringFind with missing props
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
1654 break |
|
905
502a5ae11cc5
Very close now. The cgi and mailgw now use the new security API.
Richard Jones <richard@users.sourceforge.net>
parents:
902
diff
changeset
|
1655 if node[key] is None or node[key].lower() != value: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1656 break |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1657 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1658 l.append(nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1659 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1660 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1661 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1662 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1663 def list(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1664 """ Return a list of the ids of the active nodes in this class. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1665 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1666 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1667 cn = self.classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1668 cldb = self.db.getclassdb(cn) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1669 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1670 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1671 node = self.db.getnode(cn, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1672 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1673 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1674 l.append(nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1675 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1676 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1677 l.sort() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1678 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1679 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1680 def getnodeids(self, db=None, retired=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1681 """ Return a list of ALL nodeids |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1682 |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1683 Set retired=None to get all nodes. Otherwise it'll get all the |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1684 retired or non-retired nodes, depending on the flag. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1685 """ |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1686 res = [] |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1687 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1688 # start off with the new nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1689 if self.classname in self.db.newnodes: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1690 res.extend(self.db.newnodes[self.classname]) |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1691 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1692 must_close = False |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1693 if db is None: |
|
1486
f5f60c75a458
added test for error in sqlite backend, and fixed *dbm backend error
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1694 db = self.db.getclassdb(self.classname) |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1695 must_close = True |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1696 try: |
|
5444
167f0d25ea8e
Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5395
diff
changeset
|
1697 res.extend(map(b2s, db.keys())) |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1698 |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1699 # remove the uncommitted, destroyed nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1700 if self.classname in self.db.destroyednodes: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1701 for nodeid in self.db.destroyednodes[self.classname]: |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
1702 if nodeid in db: |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1703 res.remove(nodeid) |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1704 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1705 # check retired flag |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1706 if retired is False or retired is True: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1707 l = [] |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1708 for nodeid in res: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1709 node = self.db.getnode(self.classname, nodeid, db) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1710 is_ret = self.db.RETIRED_FLAG in node |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1711 if retired == is_ret: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1712 l.append(nodeid) |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1713 res = l |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1714 finally: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1715 if must_close: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1716 db.close() |
|
7547
c8c4514f4c3e
issue685275 - show retired/unretire commands
John Rouillard <rouilj@ieee.org>
parents:
7393
diff
changeset
|
1717 |
|
c8c4514f4c3e
issue685275 - show retired/unretire commands
John Rouillard <rouilj@ieee.org>
parents:
7393
diff
changeset
|
1718 res.sort() |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1719 return res |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1720 |
|
7006
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1721 num_re = re.compile(r'^\d+$') |
|
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1722 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1723 def _filter(self, search_matches, filterspec, proptree, |
|
7006
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1724 num_re=num_re, retired=False, |
|
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1725 exact_match_spec=_marker): |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1726 """Return a list of the ids of the nodes in this class that |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1727 match the 'filter' spec, sorted by the group spec and then the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1728 sort spec. |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1729 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1730 "filterspec" is {propname: value(s)} |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1731 same for "exact_match_spec". The latter specifies exact matching |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1732 for String type while String types in "filterspec" are searched |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1733 for as case insensitive substring match. |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1734 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1735 "sort" and "group" are (dir, prop) where dir is '+', '-' or None |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1736 and prop is a prop name or None |
| 924 | 1737 |
| 4044 | 1738 "search_matches" is a sequence type or None |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1159
diff
changeset
|
1739 |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1740 "retired" specifies if we should find only non-retired nodes |
|
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1741 (default) or only retired node (value True) or all nodes. |
|
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1742 |
|
3455
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1743 The filter must match all properties specificed. If the property |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1744 value to match is a list: |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1745 |
|
3455
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1746 1. String properties must match all elements in the list, and |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1747 2. Other properties must match any of the elements in the list. |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1748 """ |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1749 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1750 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1751 |
|
7006
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1752 if exact_match_spec is _marker: |
|
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1753 exact_match_spec = {} |
|
11e2b0b1cc71
Replace mutable type and function call in arguments of function declaration.
John Rouillard <rouilj@ieee.org>
parents:
7005
diff
changeset
|
1754 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1755 cn = self.classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1756 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1757 # optimise filterspec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1758 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1759 props = self.getprops() |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1760 LINK = 'spec:link' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1761 MULTILINK = 'spec:multilink' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1762 STRING = 'spec:string' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1763 DATE = 'spec:date' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1764 INTERVAL = 'spec:interval' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1765 OTHER = 'spec:other' |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1766 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1767 for exact, filtertype in enumerate((filterspec, exact_match_spec)): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1768 for k, v in filtertype.items(): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1769 propclass = props[k] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1770 if isinstance(propclass, hyperdb.Link): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1771 if not isinstance(v, list): |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1772 v = [v] |
|
7936
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1773 if propclass.classname == 'user' and '@current_user' in v: |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1774 cu = self.db.getuid() |
|
a9b136565838
feat: issue1525113 - notation to filter by logged-in user
John Rouillard <rouilj@ieee.org>
parents:
7884
diff
changeset
|
1775 v = [x if x != "@current_user" else cu for x in v] |
|
6412
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1776 l.append((LINK, k, v)) |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1777 elif isinstance(propclass, hyperdb.Multilink): |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1778 # If it's a reverse multilink, we've already |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1779 # computed the ids of our own class. |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1780 if propclass.rev_property: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1781 l.append((OTHER, 'id', v)) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1782 else: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1783 # the value -1 is a special "not set" sentinel |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1784 if v in ('-1', ['-1']): |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1785 v = [] |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1786 elif not isinstance(v, list): |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1787 v = [v] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1788 l.append((MULTILINK, k, v)) |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1789 elif isinstance(propclass, hyperdb.String) and k != 'id': |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1790 if not isinstance(v, list): |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1791 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1792 for x in v: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1793 if exact: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1794 l.append((STRING, k, x)) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1795 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1796 # simple glob searching |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1797 x = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', x) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1798 x = x.replace('?', '.') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1799 x = x.replace('*', '.*?') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1800 l.append((STRING, k, re.compile(x, re.I))) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1801 elif isinstance(propclass, hyperdb.Date): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1802 try: |
| 6118 | 1803 ranges = [] |
| 1804 for d in v.split(','): | |
| 1805 if d == '-': | |
| 1806 ranges.append(None) | |
| 1807 continue | |
| 1808 date_rng = propclass.range_from_raw(d, self.db) | |
| 1809 ranges.append(date_rng) | |
| 1810 l.append((DATE, k, ranges)) | |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1811 except ValueError: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1812 # If range creation fails - ignore that search parameter |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1813 pass |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1814 elif isinstance(propclass, hyperdb.Interval): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1815 try: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1816 intv_rng = date.Range(v, date.Interval) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1817 l.append((INTERVAL, k, intv_rng)) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1818 except ValueError: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1819 # If range creation fails - ignore that search parameter |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1820 pass |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1821 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1822 elif isinstance(propclass, hyperdb.Boolean): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1823 if isinstance(v, str): |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1824 v = v.split(',') |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1825 if not isinstance(v, list): |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1826 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1827 bv = [] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1828 for val in v: |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1829 if isinstance(val, str): |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
1830 bv.append(propclass.from_raw(val)) |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1831 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1832 bv.append(val) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1833 l.append((OTHER, k, bv)) |
|
2453
0e2a0c2c8142
allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2450
diff
changeset
|
1834 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1835 elif k == 'id': |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1836 if not isinstance(v, list): |
|
4364
0e81742d0e2f
- unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4357
diff
changeset
|
1837 v = v.split(',') |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1838 l.append((OTHER, k, [str(int(val)) for val in v])) |
|
2453
0e2a0c2c8142
allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2450
diff
changeset
|
1839 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1840 elif isinstance(propclass, hyperdb.Number): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1841 if not isinstance(v, list): |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1842 try: |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1843 v = v.split(',') |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1844 except AttributeError: |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1845 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1846 l.append((OTHER, k, [float(val) for val in v])) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1847 |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1848 elif isinstance(propclass, hyperdb.Integer): |
|
7012
921bf9bdeea9
flake8 convert type comparisons to isinstance.
John Rouillard <rouilj@ieee.org>
parents:
7011
diff
changeset
|
1849 if not isinstance(v, list): |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1850 try: |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1851 v = v.split(',') |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1852 except AttributeError: |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1853 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1854 l.append((OTHER, k, [int(val) for val in v])) |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1855 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1856 filterspec = l |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1857 |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1858 # now, find all the nodes that pass filtering |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1859 matches = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1860 cldb = self.db.getclassdb(cn) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1861 t = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1862 try: |
| 924 | 1863 # TODO: only full-scan once (use items()) |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1864 for nodeid in self.getnodeids(cldb, retired=retired): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1865 node = self.db.getnode(cn, nodeid, cldb) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1866 # apply filter |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1867 for t, k, v in filterspec: |
|
1303
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1868 # handle the id prop |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1869 if k == 'id': |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1870 if nodeid not in v: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1871 break |
|
1303
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1872 continue |
|
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1873 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1874 # get the node value |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1875 nv = node.get(k, None) |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1876 |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1877 match = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1878 |
| 924 | 1879 # now apply the property filter |
| 1880 if t == LINK: | |
| 1881 # link - if this node's property doesn't appear in the | |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1882 # filterspec's nodeid list, skip it |
|
8241
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1883 try: |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1884 expr = Expression(v, is_link=True) |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1885 except ExpressionError as e: |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1886 e.context['class'] = cn |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1887 e.context['attr'] = k |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1888 raise |
|
6412
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1889 if expr.evaluate(nv): |
|
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1890 match = 1 |
| 924 | 1891 elif t == MULTILINK: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1892 # multilink - if any of the nodeids required by the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1893 # filterspec aren't in this node's property, then skip |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1894 # it |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1895 nv = node.get(k, []) |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
1896 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1897 # check for matching the absence of multilink values |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1898 if not v: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1899 match = not nv |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1900 else: |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4447
diff
changeset
|
1901 # otherwise, make sure this node has each of the |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1902 # required values |
|
8241
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1903 try: |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1904 expr = Expression(v) |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1905 except ExpressionError as e: |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1906 e.context['class'] = cn |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1907 e.context['attr'] = k |
|
741ea8a86012
fix: issue2551374. Error handling for filter expressions.
John Rouillard <rouilj@ieee.org>
parents:
8130
diff
changeset
|
1908 raise |
|
6396
75a53956cf13
Multilink expressions with simple "or"
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6352
diff
changeset
|
1909 if expr.evaluate(nv): |
|
75a53956cf13
Multilink expressions with simple "or"
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6352
diff
changeset
|
1910 match = 1 |
| 924 | 1911 elif t == STRING: |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1912 if nv is None: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1913 nv = '' |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1914 if is_us(v): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1915 # Exact match |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1916 match = (nv == v) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1917 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1918 # RE search |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1919 match = v.search(nv) |
| 6118 | 1920 elif t == DATE: |
| 1921 for x in v: | |
| 1922 if x is None or nv is None: | |
| 1923 if nv is None and x is None: | |
| 1924 match = 1 | |
| 1925 break | |
| 1926 continue | |
| 1927 elif x.to_value: | |
| 1928 if x.from_value <= nv <= x.to_value: | |
| 1929 match = 1 | |
| 1930 break | |
| 1931 else: | |
| 1932 if x.from_value <= nv: | |
| 1933 match = 1 | |
| 1934 break | |
| 1935 elif t == INTERVAL: | |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1936 if nv is None: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1937 match = v is None |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
1938 else: |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1939 if v.to_value: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1940 if v.from_value <= nv and v.to_value >= nv: |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1941 match = 1 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1942 else: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1943 if v.from_value <= nv: |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1944 match = 1 |
| 924 | 1945 elif t == OTHER: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1946 # straight value comparison for the other types |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1947 match = nv in v |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1948 if not match: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1949 break |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1950 else: |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1951 matches.append([nodeid, node]) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1952 |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1953 # filter based on full text search |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1954 if search_matches is not None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1955 k = [] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1956 for v in matches: |
| 4044 | 1957 if v[0] in search_matches: |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1958 k.append(v) |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1959 matches = k |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1960 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1961 # add sorting information to the proptree |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1962 JPROPS = {'actor': 1, 'activity': 1, 'creator': 1, 'creation': 1} |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1963 children = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1964 if proptree: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1965 children = proptree.sortable_children() |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1966 for pt in children: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1967 dir = pt.sort_direction |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1968 prop = pt.name |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1969 assert (dir and prop) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1970 propclass = props[prop] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1971 pt.sort_ids = [] |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1972 is_pointer = isinstance(propclass, (hyperdb.Link, |
|
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
1973 hyperdb.Multilink)) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1974 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1975 pt.sort_result = [] |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1976 try: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1977 # cache the opened link class db, if needed. |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1978 lcldb = None |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1979 # cache the linked class items too |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1980 lcache = {} |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1981 |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1982 for entry in matches: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1983 itemid = entry[-2] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1984 item = entry[-1] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1985 # handle the properties that might be "faked" |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1986 # also, handle possible missing properties |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1987 try: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1988 v = item[prop] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1989 except KeyError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1990 if prop in JPROPS: |
|
2437
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1991 # force lookup of the special journal prop |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1992 v = self.get(itemid, prop) |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1993 else: |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1994 # the node doesn't have a value for this |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1995 # property |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1996 v = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1997 if isinstance(propclass, hyperdb.Multilink): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1998 v = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1999 if prop == 'id': |
|
7001
a41d88b560f8
Remove whitespace before ( and [
John Rouillard <rouilj@ieee.org>
parents:
7000
diff
changeset
|
2000 v = int(itemid) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2001 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2002 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2003 pt.sort_result.append(v) |
|
2437
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
2004 continue |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2005 |
|
2285
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
2006 # missing (None) values are always sorted first |
|
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
2007 if v is None: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2008 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2009 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2010 pt.sort_result.append(v) |
|
2285
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
2011 continue |
|
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
2012 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2013 if isinstance(propclass, hyperdb.Link): |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2014 lcn = propclass.classname |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2015 link = self.db.classes[lcn] |
|
3476
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
2016 key = link.orderprop() |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2017 child = pt.propdict[key] |
|
7003
230852cb909f
Final whitespace cleanups.
John Rouillard <rouilj@ieee.org>
parents:
7002
diff
changeset
|
2018 if key != 'id': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2019 if v not in lcache: |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2020 # open the link class db if it's not already |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2021 if lcldb is None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2022 lcldb = self.db.getclassdb(lcn) |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2023 lcache[v] = self.db.getnode(lcn, v, lcldb) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2024 r = lcache[v][key] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2025 child.propdict[key].sort_ids.append(r) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2026 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2027 child.propdict[key].sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2028 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2029 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2030 r = propclass.sort_repr(pt.parent.cls, v, pt.name) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
2031 pt.sort_result.append(r) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2032 finally: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2033 # if we opened the link class db, close it now |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2034 if lcldb is not None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2035 lcldb.close() |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
2036 del lcache |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2037 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2038 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2039 |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2040 # pull the id out of the individual entries |
|
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2041 matches = [entry[-2] for entry in matches] |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
2042 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
2043 self.db.stats['filtering'] += (time.time() - start_t) |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
2044 return matches |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2045 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2046 def count(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2047 """Get the number of nodes in this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2048 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2049 If the returned integer is 'numnodes', the ids of all the nodes |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2050 in this class run from 1 to numnodes, and numnodes+1 will be the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2051 id of the next node to be created in this class. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2052 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2053 return self.db.countnodes(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2054 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2055 # Manipulating properties: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2056 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2057 def getprops(self, protected=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2058 """Return a dictionary mapping property names to property objects. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2059 If the "protected" flag is true, we include protected properties - |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2060 those which may not be modified. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2061 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2062 In addition to the actual properties on the node, these |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2063 methods provide the "creation" and "activity" properties. If the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2064 "protected" flag is true, we include protected properties - those |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2065 which may not be modified. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2066 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2067 d = self.properties.copy() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2068 if protected: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2069 d['id'] = hyperdb.String() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2070 d['creation'] = hyperdb.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2071 d['activity'] = hyperdb.Date() |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
2072 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2073 d['actor'] = hyperdb.Link('user') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2074 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2075 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2076 def addprop(self, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2077 """Add properties to this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2078 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2079 The keyword arguments in 'properties' must map names to property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2080 objects, or a TypeError is raised. None of the keys in 'properties' |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2081 may collide with the names of existing properties, or a ValueError |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2082 is raised before any properties have been added. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2083 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2084 for key in properties: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2085 if key in self.properties: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2086 raise ValueError(key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2087 self.properties.update(properties) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2088 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2089 def index(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2090 """ Add (or refresh) the node to search indexes """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2091 # find all the String properties that have indexme |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2092 for prop, propclass in self.getprops().items(): |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2093 if isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2094 # index them under (classname, nodeid, property) |
|
7393
9e612a39547a
Make anydbm indexer handle missing items the same as rdbms backend
John Rouillard <rouilj@ieee.org>
parents:
7211
diff
changeset
|
2095 self.db.indexer.add_text((self.classname, nodeid, prop), |
|
9e612a39547a
Make anydbm indexer handle missing items the same as rdbms backend
John Rouillard <rouilj@ieee.org>
parents:
7211
diff
changeset
|
2096 str(self.get(nodeid, prop))) |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2097 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2098 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2099 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2100 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2101 def export_list(self, propnames, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2102 """ Export a node - generate a list of CSV-able data in the order |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2103 specified by propnames for the given node. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2104 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2105 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2106 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2107 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2108 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2109 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2110 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2111 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2112 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2113 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2114 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2115 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2116 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2117 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2118 value = str(value) |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2119 l.append(repr_export(value)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2120 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2121 # append retired flag |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2122 l.append(repr_export(self.is_retired(nodeid))) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2123 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2124 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2125 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2126 def import_list(self, propnames, proplist): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2127 """ Import a node - all information including "id" is present and |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2128 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2129 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2130 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2131 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2132 Return the nodeid of the node imported. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2133 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2134 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2135 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2136 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2137 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2138 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2139 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2140 newid = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2141 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2142 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2143 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2144 |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2145 # Use eval_import to reverse the repr_export() used to |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2146 # output the CSV |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2147 value = eval_import(proplist[i]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2148 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2149 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2150 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2151 newid = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2152 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2153 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2154 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2155 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2156 d[self.db.RETIRED_FLAG] = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2157 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2158 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2159 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2160 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2161 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2162 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2163 if isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2164 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2165 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2166 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2167 elif isinstance(prop, hyperdb.Password): |
|
7211
506c86823abb
Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents:
7038
diff
changeset
|
2168 value = password.Password(encrypted=value, |
|
506c86823abb
Add config argument to more password.Password invocations.
John Rouillard <rouilj@ieee.org>
parents:
7038
diff
changeset
|
2169 config=self.db.config) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2170 d[propname] = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2171 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2172 # get a new id if necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2173 if newid is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2174 newid = self.db.newid(self.classname) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2175 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2176 # add the node and journal |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2177 self.db.addnode(self.classname, newid, d) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2178 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2179 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2180 def export_journals(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2181 """Export a class's journal - generate a list of lists of |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2182 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2183 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2184 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2185 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2186 No heading here - the columns are fixed. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2187 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2188 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2189 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2190 for nodeid in self.getnodeids(): |
|
6998
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
2191 for nodeid, date_, user, action, params in self.history( |
|
6997
60b37f632601
Flake8 fix whitespace/indent issues.
John Rouillard <rouilj@ieee.org>
parents:
6820
diff
changeset
|
2192 nodeid, enforceperm=False, skipquiet=False): |
|
6998
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
2193 date_ = date_.get_tuple() |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2194 if action == 'set': |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2195 export_data = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2196 for propname, value in params.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2197 if propname not in properties: |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2198 # property no longer in the schema |
|
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2199 continue |
|
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2200 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2201 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2202 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2203 if value is None: |
|
3826
bf2e9535da00
Journal and database testing.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3687
diff
changeset
|
2204 pass |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2205 elif isinstance(prop, hyperdb.Date): |
|
3166
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2206 # this is a hack - some dates are stored as strings |
|
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2207 if not isinstance(value, type('')): |
|
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2208 value = value.get_tuple() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2209 elif isinstance(prop, hyperdb.Interval): |
|
3390
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2210 # hack too - some intervals are stored as strings |
|
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2211 if not isinstance(value, type('')): |
|
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2212 value = value.get_tuple() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2213 elif isinstance(prop, hyperdb.Password): |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2214 value = str(value) |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2215 export_data[propname] = value |
| 3052 | 2216 params = export_data |
|
6998
e14b013ac83b
flake8: fix imports; fix shadowed import; silence unused var warning
John Rouillard <rouilj@ieee.org>
parents:
6997
diff
changeset
|
2217 r.append([repr_export(nodeid), repr_export(date_), |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2218 repr_export(user), repr_export(action), |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2219 repr_export(params)]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2220 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2221 |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2222 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2514
diff
changeset
|
2223 class FileClass(hyperdb.FileClass, Class): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2224 """This class defines a large chunk of data. To support this, it has a |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2225 mandatory String property "content" which is typically saved off |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2226 externally to the hyperdb. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2227 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2228 The default MIME type of this data is defined by the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2229 "default_mime_type" class attribute, which may be overridden by each |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2230 node if the class defines a "type" String property. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2231 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2232 def __init__(self, db, classname, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2233 """The newly-created class automatically includes the "content" |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2234 and "type" properties. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2235 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2236 if 'content' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2237 properties['content'] = hyperdb.String(indexme='yes') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2238 if 'type' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2239 properties['type'] = hyperdb.String() |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2240 Class.__init__(self, db, classname, **properties) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2241 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2242 def create(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2243 """ Snarf the "content" propvalue and store in a file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2244 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2245 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2246 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2247 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2248 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2249 # now remove the content property so it's not stored in the db |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2250 content = propvalues['content'] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2251 del propvalues['content'] |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2252 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2253 # make sure we have a MIME type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2254 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2255 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2256 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2257 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2258 |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3960
diff
changeset
|
2259 # store off the content as a file |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2260 self.db.storefile(self.classname, newid, None, bs2b(content)) |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3960
diff
changeset
|
2261 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2262 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2263 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2264 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2265 return newid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2266 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2267 def get(self, nodeid, propname, default=_marker, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2268 """ Trap the content propname and get it from the file |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2269 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2270 'cache' exists for backwards compatibility, and is not used. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2271 """ |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
2272 poss_msg = 'Possibly an access right configuration problem.' |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2273 if propname == 'content': |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2274 try: |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2275 return b2s(self.db.getfile(self.classname, nodeid, None)) |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
5232
diff
changeset
|
2276 except IOError as strerror: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2277 # XXX by catching this we don't see an error in the log. |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2278 return 'ERROR reading file: %s%s\n%s\n%s' % ( |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2279 self.classname, nodeid, poss_msg, strerror) |
|
5725
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2280 except UnicodeDecodeError as e: |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2281 # if content is not text (e.g. jpeg file) we get |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2282 # unicode error trying to convert to string in python 3. |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2283 # trap it and supply an error message. Include md5sum |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2284 # of content as this string is included in the etag |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2285 # calculation of the object. |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2286 return ('%s%s is not text, retrieve using ' |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2287 'binary_content property. mdsum: %s') % ( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2288 self.classname, nodeid, |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2289 md5(self.db.getfile( |
|
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2290 self.classname, nodeid, None)).hexdigest()) # nosec - bandit md5 use ok |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2291 elif propname == 'binary_content': |
|
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2292 return self.db.getfile(self.classname, nodeid, None) |
|
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2293 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2294 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2295 return Class.get(self, nodeid, propname, default) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2296 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2297 return Class.get(self, nodeid, propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2298 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2299 def set(self, itemid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2300 """ Snarf the "content" propvalue and update it in a file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2301 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2302 self.fireAuditors('set', itemid, propvalues) |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2303 |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2304 # create the oldvalues dict - fill in any missing values |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2305 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2306 for name, prop in self.getprops(protected=0).items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2307 if name in oldvalues: |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2308 continue |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2309 if isinstance(prop, hyperdb.Multilink): |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2310 oldvalues[name] = [] |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2311 else: |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2312 oldvalues[name] = None |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2313 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2314 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2315 content = None |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2316 if 'content' in propvalues: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2317 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2318 del propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2319 |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2320 # do the database update |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2321 propvalues = self.set_inner(itemid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2322 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2323 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2324 if content: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2325 # store and possibly index |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2326 self.db.storefile(self.classname, itemid, None, bs2b(content)) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2327 if self.properties['content'].indexme: |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2328 index_content = content |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2329 if bytes != str and isinstance(content, bytes): |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2330 index_content = content.decode('utf-8', errors='ignore') |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2331 mime_type = self.get(itemid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2332 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2333 index_content, mime_type) |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2334 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2335 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2336 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2337 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2338 return propvalues |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2339 |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2340 def index(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2341 """ Add (or refresh) the node to search indexes. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2342 |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2343 Use the content-type property for the content property. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2344 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2345 # find all the String properties that have indexme |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2346 for prop, propclass in self.getprops().items(): |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2347 if prop == 'content' and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2348 mime_type = self.get(nodeid, 'type', self.default_mime_type) |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2349 index_content = self.get(nodeid, 'binary_content') |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2350 if bytes != str and isinstance(index_content, bytes): |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2351 index_content = index_content.decode('utf-8', |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2352 errors='ignore') |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2353 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2354 index_content, mime_type) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2355 elif isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2356 # index them under (classname, nodeid, property) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2357 try: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2358 value = str(self.get(nodeid, prop)) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2359 except IndexError: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2360 # node has been destroyed |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2361 continue |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2362 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2363 |
|
7000
30cc3d6f1a3d
Flake8 fixes: spaces around modulo, need extra blank lines, underindent
John Rouillard <rouilj@ieee.org>
parents:
6999
diff
changeset
|
2364 |
|
1103
db787cef1385
handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents:
1099
diff
changeset
|
2365 # deviation from spec - was called ItemClass |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2366 class IssueClass(Class, roundupdb.IssueClass): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2367 # Overridden methods: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2368 def __init__(self, db, classname, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2369 """The newly-created class automatically includes the "messages", |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2370 "files", "nosy", and "superseder" properties. If the 'properties' |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2371 dictionary attempts to specify any of these properties or a |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2372 "creation" or "activity" property, a ValueError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2373 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2374 if 'title' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2375 properties['title'] = hyperdb.String(indexme='yes') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2376 if 'messages' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2377 properties['messages'] = hyperdb.Multilink("msg") |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2378 if 'files' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2379 properties['files'] = hyperdb.Multilink("file") |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2380 if 'nosy' not in properties: |
| 1002 | 2381 # note: journalling is turned off as it really just wastes |
| 2382 # space. this behaviour may be overridden in an instance | |
| 2383 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2384 if 'superseder' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2385 properties['superseder'] = hyperdb.Multilink(classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2386 Class.__init__(self, db, classname, **properties) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2387 |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
2388 # vim: set et sts=4 sw=4 : |
